Model deployment method and device, video auditing method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and provides a model deployment method, a video auditing method, a video auditing device, electronic equipment and a storage medium, aiming at a to-be-deployed target detection model, in the model deployment process, the calculation efficiency of the model is improved by reconstructing a topological structure of a feature extraction network, and the video auditing efficiency is improved. The data access efficiency is improved by optimizing the tensor layout of the feature extraction network, and meanwhile, an up-sampling layer and an element-by-element operation layer with a short connection path in the feature fusion network are combined into a single GPU kernel, so that multiple operations are completed in the single GPU kernel, any intermediate format conversion is not needed, and the data access efficiency is improved. Therefore, the deployed target detection model can efficiently run on the GPU.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to a model deployment method, a video review method, an apparatus, an electronic device, and a storage medium. Background Technology

[0002] In live streaming scenarios, the application of high-performance object detection models is increasing. For example, they first detect text in the screen (such as bullet comments, watermarks, or in-game text), and then use OCR (Optical Character Recognition) to recognize its content for tasks such as content review.

[0003] To meet the high real-time requirements of live streaming scenarios, trained object detection models need to be deployed on GPUs (Graphics Processing Units). However, the backbone networks of current high-performance object detection models generally use computational modules with complex nonlinear topologies. Such structures include multi-path branching and feature reuse mechanisms, making it difficult for existing general-purpose inference frameworks to efficiently fuse computational graphs. Their optimizers decompose continuous operations into multiple fine-grained, serially scheduled computational kernels, while introducing frequent kernel startup overhead and intermediate result read / write operations, resulting in low model efficiency on GPUs. Summary of the Invention

[0004] The purpose of this application is to provide a model deployment method, video review method, device, electronic device and storage medium that enable the deployed target detection model to run efficiently on the GPU.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, embodiments of this application provide a model deployment method, the method comprising: Obtain the target detection model to be deployed, the target detection model including a feature extraction network and a feature fusion network; The topology of the feature extraction network is reconstructed and its tensor layout is optimized; The upsampling layer and element-wise operation layer with short connection paths in the feature fusion network are merged into a single GPU kernel.

[0006] Optionally, the feature extraction network includes a convolutional module, a multi-layer pooling module, and a multi-gradient feature combination module; Reconstructing the topology of the feature extraction network and optimizing its tensor layout includes: The basic feature extraction sequences in the convolution module are merged into a single GPU kernel, and the input tensor layout of the splicing layer is adjusted to ensure memory continuity. The maximum pooling serial cascade structure in the multi-layer pooling module remains unchanged; All input tensors of the multi-gradient feature combination module are processed by a dedicated GPU kernel in the first custom plugin.

[0007] Optionally, the upsampling layer and element-wise operation layer with short connection paths in the feature fusion network are merged into a single GPU kernel, including: Identify upsampling layers and element-wise operation layers with short connection paths in the feature fusion network; The output tensor layout of the upsampling layer and the element-wise operation layer is unified to merge the upsampling layer and the element-wise operation layer into a single GPU kernel.

[0008] Optionally, the target detection model further includes a detection head, and the method further includes: Within a single GPU core, the probability distribution tensor output by the detection head is decoded into a bounding box coordinate tensor; Within a single GPU core, multiple raw prediction tensors containing the bounding box coordinates are decoded and filtered to output a detection result tensor.

[0009] Optionally, within a single GPU core, decoding the probability distribution tensor output by the detection head into a bounding box coordinate tensor includes: Using a dedicated GPU kernel in the second custom plugin, the probability distribution of each position in the probability distribution tensor output by the detection head is weighted and summed with a predefined coordinate vector to generate a bounding box coordinate tensor.

[0010] Optionally, within a single GPU core, multiple raw prediction tensors containing the bounding box coordinates are decoded and filtered to output a detection result tensor, including: Using a dedicated GPU kernel in the third custom plugin, threshold filtering and multi-class nonmaximum suppression are performed on multiple raw prediction tensors containing the bounding box coordinates, and the resulting detection result tensors are stored.

[0011] Secondly, embodiments of this application provide a video review method, the method comprising: Obtain the video frames to be reviewed from the live video stream; The target detection model deployed by the method in the first aspect above is used to input the video frame to be reviewed into the target detection model to determine the location of the text in the video to be reviewed. The video frames to be reviewed are reviewed based on the image of the location of the text in the video to be reviewed.

[0012] Thirdly, embodiments of this application provide a model deployment apparatus, the apparatus comprising: The model acquisition module is used to acquire the target detection model to be deployed, which includes a feature extraction network and a feature fusion network. The feature extraction network processing module is used to reconstruct the topology of the feature extraction network and optimize its tensor layout; The feature fusion network processing module is used to merge the upsampling layer and the element-wise operation layer with short connection paths in the feature fusion network into a single GPU core.

[0013] Fourthly, embodiments of this application provide a video review device, the device comprising: The video frame acquisition module is used to acquire video frames to be reviewed from the live video stream; The text detection module is used to input the video frame to be reviewed into the text target detection model deployed by the method in the first aspect above, and to determine the location of the text in the video to be reviewed; The review module is used to review the video frames to be reviewed based on the image of the location of the text in the video to be reviewed.

[0014] Fifthly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor can execute the machine-executable instructions to implement the model deployment method in the first aspect above, and / or the video review method in the second aspect above.

[0015] Sixthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the model deployment method in the first aspect above, and / or the video review method in the second aspect above.

[0016] Compared to existing technologies, the present application provides a model deployment method, video review method, device, electronic device, and storage medium. For the target detection model to be deployed, during the model deployment process, the computational efficiency of the model is improved by reconstructing the topology of the feature extraction network, and the data access efficiency is improved by optimizing the tensor layout of the feature extraction network. At the same time, the upsampling layer and element-wise operation layer with short connection paths in the feature fusion network are merged into a single GPU core, so that multiple operations can be completed in a single GPU core without any intermediate format conversion, thereby enabling the deployed target detection model to run efficiently on the GPU.

[0017] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 The diagram illustrates an application scenario in live streaming.

[0020] Figure 2 A flowchart illustrating a model deployment method provided in an embodiment of this application is shown.

[0021] Figure 3 The diagram shows a functional block diagram of a target detection model provided in an embodiment of this application.

[0022] Figure 4 A flowchart illustrating a video review method provided in an embodiment of this application is shown.

[0023] Figure 5 A block diagram of a model deployment apparatus provided in an embodiment of this application is shown.

[0024] Figure 6 A block diagram of a video review device provided in an embodiment of this application is shown.

[0025] Figure 7 A block diagram of an electronic device provided in an embodiment of this application is shown.

[0026] Icons: 10-Electronic device; 11-Processor; 12-Memory; 13-Bus; 100-Model deployment device; 101-Model acquisition module; 102-Feature extraction network processing module; 103-Feature fusion network processing module; 104-Post-processing module; 200-Video review device; 201-Video frame acquisition module; 202-Text detection module; 203-Review module. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can be arranged and designed in various different configurations.

[0028] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0029] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0030] The technical solutions provided in the embodiments of this application can be applied to, for example... Figure 1 The live streaming scenario shown includes a live streaming server and clients. The clients are divided into broadcaster clients and viewer clients. The broadcaster clients and viewer clients communicate with the live streaming server via the network, enabling the broadcaster on the broadcaster clients and the viewer clients to participate in real-time online live streaming. The terminal devices of the broadcaster clients and viewer clients can be, but are not limited to, various personal computers, laptops, smartphones, and tablets. The live streaming server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0031] In the aforementioned live streaming scenarios, the live streaming server may need to automatically identify text content in the video (such as bullet comments, watermarks, in-game text, etc.) for tasks such as content moderation. These identification tasks rely on object detection models, which can locate the position of text in the video and outline it with a bounding box.

[0032] In order to enable the object detection model to complete the task quickly and accurately in scenarios with extremely high speed requirements such as live streaming, the trained model needs to be deployed to the GPU of the live streaming server to run. GPU is a type of hardware specifically designed for parallel computing, which is suitable for processing large-scale data such as images and videos.

[0033] Currently, the industry has developed a series of general-purpose inference frameworks for efficiently deploying deep learning models on GPUs. However, when applying these general-purpose inference frameworks to object detection models in live streaming scenarios, existing solutions have some limitations, mainly reflected in the following aspects: 1. Based on the inference backend of the deep learning framework itself, this approach directly utilizes the framework's built-in just-in-time (JIT) compilation or accelerated linear algebra (XLA) technology for model optimization and deployment. While this method provides basic graph optimization and operator fusion capabilities, its optimization depth and hardware adaptability are typically inferior to dedicated inference engines. It struggles to handle complex custom operators, has limited fusion capabilities for non-standard network topologies, and cannot provide fine-grained kernel-level control and mixed-precision strategies. Therefore, its performance improvement is often limited and insufficient to meet the high real-time requirements of live streaming scenarios.

[0034] 2. The automated optimization process of the general inference framework involves directly inputting the trained model into the general inference framework after it has been transferred through an intermediate format. The framework then uses its automated graph optimization, operator fusion, and quantization functions to generate the inference engine. The automated process of the general inference framework is effective for standard networks (such as ResNet). However, for models with a large number of customized designs, its automatic graph optimizer may not be able to identify the best fusion strategy. It may retain the parts that cannot be identified or optimized as discrete, inefficient operators, or may not support them at all, resulting in construction failure.

[0035] Therefore, existing general inference frameworks cannot meet the deployment requirements of object detection models in live streaming scenarios.

[0036] To address the aforementioned issues, the model deployment method provided in this application improves the computational and data access efficiency of the target detection model by reconstructing the topology of the feature extraction network and optimizing its tensor layout. Furthermore, it merges multiple layers with short connection paths in the feature fusion network into a single GPU kernel, enabling multiple operations to be completed within this single kernel without any intermediate format conversion. This ensures that the deployed target detection model can run efficiently on the GPU. A detailed description follows.

[0037] The model deployment method provided in this application embodiment can be applied to... Figure 1 The live streaming server can also be used on the terminal devices (e.g., personal computers) of technicians who communicate with the live streaming server; this embodiment does not impose any restrictions on this.

[0038] Please refer to Figure 2 , Figure 2 This illustration shows a flowchart of a model deployment method provided in an embodiment of this application, which may include the following steps: S101, Obtain the target detection model to be deployed; S102, reconstruct the topology of the feature extraction network and optimize its tensor layout; S103 merges the upsampling layer and element-wise operation layer with short connection paths in the feature fusion network into a single GPU kernel.

[0039] In this embodiment, the target detection model can be a text detection model specifically customized for live OCR scenarios, capable of adapting to the characteristics of complex text forms and strong background interference in live scenarios. For example, such as Figure 3 As shown, the target detection model includes a feature extraction network, a feature fusion module, and a detection head. In a live OCR scenario, after the video frames to be reviewed extracted from the live video stream are input into the target detection model, the processing flow is as follows: 1. An initial feature map is extracted from the video frames to be reviewed by a feature extraction network, and then multi-scale spatial information and spatial context information are extracted from the initial feature map respectively; 2. The feature fusion module performs feature fusion on the final feature map input to the feature extraction network; 3. Utilize the detection head to predict the bounding box coordinates of text regions based on fused features.

[0040] In order to meet the accuracy and speed requirements of live OCR scenarios, the above target detection model is relatively complex in its architecture design, mainly reflected in: (1) the feature extraction network adopts a computation module with complex nonlinear topology (such as multi-path branching, feature caching and recombination), and (2) multiple layers in the feature fusion network adopt short connections, which changes the original network topology.

[0041] Existing general-purpose inference frameworks' automatic graph optimizers excel at fusing linear operator sequences, but their fusing capability fails for computation modules with complex nonlinear topologies. These modules are decomposed into a large number of inefficient, discrete computation kernel calls, resulting in significant kernel startup overhead and data read / write overhead. Furthermore, changes in network topology introduce numerous intermediate format conversions, requiring frequent read / write operations on intermediate results, leading to low efficiency of the model running on GPUs.

[0042] To ensure that the deployed object detection model can run efficiently on GPUs, the model structure needs to be optimized during the deployment process based on the existing general inference framework. This includes: 1. For the feature extraction network, improving computational efficiency by reconstructing its topology and improving data access efficiency by optimizing its tensor layout. Reconstructing the topology can involve operator fusion, while optimizing the tensor layout refers to adjusting the way input or output tensors are stored in memory; 2. For the feature fusion network, merging multiple layers with short connection paths into a single GPU kernel allows multiple operations to be completed within a single GPU kernel without any intermediate format conversion, further improving computational efficiency.

[0043] In practical applications, for the target detection model to be deployed, it is first converted into an intermediate model file. Then, the intermediate model file is optimized using graph editing tools, such as operator fusion. Finally, the optimized intermediate model file is converted into a model file supported by a general inference framework and run on the GPU.

[0044] Please continue to refer to Figure 3 The feature extraction network further includes a convolution module, a multi-level pooling module, and a multi-gradient feature combination module. The convolution module can reduce computational overhead while maintaining high accuracy. The multi-level pooling module can enhance the fusion of global and local features by combining pooling windows of different sizes. The multi-gradient feature combination module further enhances the feature representation capability by parallelizing multiple gradient flow branches, and can capture small and hidden text features very well.

[0045] To improve detection performance, feature extraction networks often employ an improved Cross-Stage Partial Networks (CSPDarknet) architecture, such as CSPDarknet-53. The convolutional module, multi-level pooling module, and multi-gradient feature combination module are respectively the CSP block, SPPF (Spatial Pyramid Pooling Fast) module, and C2F (CSP2 Fused, a cross-stage partial network combined with two fused convolutions) module.

[0046] Among them, the CSP block contains a complex feature graph splitting-processing-splitting topology. The SPPF module replaces the original parallel max pooling structure in the SPP (Spatial Pyramid Pooling Fast) module with a serially cascaded max pooling structure, while the C2F module introduces more complex feature flow branching and fusion. These structural changes place higher demands on the graph optimization and parallel execution scheduling of the general inference framework.

[0047] Based on this, in one optional implementation, the process of reconstructing the topology of the feature extraction network and optimizing its tensor layout in step S102 may include: S1021 merges the basic feature extraction sequences in the convolution module into a single GPU kernel and adjusts the input tensor layout of the splicing layer to ensure memory continuity; S1022, keep the maximum pooling serial cascade structure in the multi-layer pooling module unchanged; S1023 processes all input tensors of the multi-gradient feature combination module through a dedicated GPU kernel in the first custom plugin.

[0048] In step S1021, the basic feature extraction sequence can be a continuously computed sequence, i.e., containing multiple continuously computed layers, such as the convolution (Conv) -> batch normalization (BN) -> activation (SiLU) sequence in a CSP block. To avoid the general inference framework breaking down continuous operations into multiple fine-grained, serially scheduled computation kernels, this basic feature extraction sequence is merged into a single GPU kernel. Simultaneously, to prevent the general inference framework from inserting performance-degrading layers due to memory discontinuity during engine construction, the input tensor layout of the concatenation layer in the convolution module is adjusted; that is, the data storage format of the input tensors in memory is adjusted to ensure memory continuity.

[0049] Taking the CSP block as an example, during the model export stage, the intermediate model file of the CSP block is processed using graph editing tools, including: 1. Forcing the rewriting of the Convolution (Conv) -> Batch Normalization (BN) -> Activation (SiLU) sequence into a standard pattern that can be recognized by the general inference framework and deterministically fused into a single efficient CUDA Kernel. Simultaneously, the input tensors of the concatenation (Concat) layer are analyzed and adjusted, forcing them to be laid out continuously in memory (such as continuous NCHW), thereby eliminating the performance degradation caused by the insertion of Reformat (format conversion) or Shuffle (channel shuffling) layers due to memory discontinuity during engine construction by the general inference framework.

[0050] Among them, CUDA Kernel is a programming model core that utilizes GPUs for large-scale data parallel computing. It accelerates computing by organizing massive threads, providing memory collaboration mechanisms, and a unique execution model, breaking down computing tasks into tens of thousands of small tasks that are executed simultaneously.

[0051] NCHW is a format notation used to describe the memory layout of multidimensional data (usually tensors), commonly found in deep learning frameworks. Here, N represents the batch size, indicating the number of samples in a batch; C represents the number of channels, i.e., the number of feature layers per sample (e.g., the three color channels of an RGB image); H and W represent the height and width of the feature map, respectively. This format explicitly specifies the dimensional order of data in memory: data is first arranged by batch, then by channels, and finally by spatial dimensions (height and width). The NCHW layout is particularly suitable for optimizing convolution operations using GPUs, as it better utilizes the principle of locality and hardware parallel computing capabilities, thereby improving computational efficiency.

[0052] After completing the above operations, it is necessary to further verify the effectiveness of the optimized convolutional module and convert it into a file that can be run by the general inference framework. For example, build the engine using the general inference framework and check the logs. If, under the adjusted channel number configuration, all key convolutional layers successfully call the high-performance convolutional kernel of the general inference framework (such as Winograd or Implicit GEMM), and there are no records in the logs of falling back to the inefficient general GEMM implementation, then the verification is successful.

[0053] In step S1022, the serial cascaded max-pooling structure refers to the serial use of the same max-pooling layer (MaxPool) multiple times. Compared to using multiple max-pooling layers in parallel, this serial repetition of using the same block reduces computation and improves speed. However, this structure places higher demands on the graph optimization of general inference frameworks. The inventors analyzed and verified the serial cascaded max-pooling structure in the multi-layer pooling module using tools, confirming that its data flow exhibits efficient pipelined operation on the GPU, avoiding the scheduling overhead and memory bottlenecks of intermediate results that may arise from parallel structures. Therefore, this structure is maintained unchanged in model deployment.

[0054] In step S1023, in order to effectively capture small and subtle text features, the multi-gradient feature combination module introduces more complex feature flow branches and fusions, resulting in kernel startup overhead and data transmission latency between multiple operators. To address this issue, this embodiment designs a custom plugin that uses a dedicated GPU kernel to process all input tensors of the multi-gradient feature combination module.

[0055] Taking the C2F module as an example, a custom plugin is designed to address its complex multi-gradient flow branches. This plugin receives all the input tensors from the C2F module and completes all feature splitting, convolution calculation, batch normalization, activation, and final concatenation operations within a single CUDA Kernel. In this way, the original C2F subgraph is replaced by a single node call to this plugin, effectively eliminating kernel startup overhead and data transmission latency between multiple operators.

[0056] In one optional implementation, step S102, which merges the upsampling layer and element-wise operation layer with short connection paths in the feature fusion network into a single GPU kernel, includes: Find upsampling layers and element-wise operation layers with short connection paths in feature fusion networks; The output tensor layout of the upsampling layer and the element-wise operation layer is unified to merge the upsampling layer and the element-wise operation layer into a single GPU kernel.

[0057] In this embodiment, considering that text regions may appear at different scales in an image and have different aspect ratios and shapes, the feature fusion network, in order to better adapt to the diversity of text shapes (such as large differences in aspect ratio, font deformation, etc.), can combine high-level semantic features with low-level spatial features through a top-down path during feature fusion to generate multi-scale feature maps. This helps to capture text regions of different sizes. Then, the feature fusion is further strengthened through a bottom-up path to obtain the final fused features.

[0058] For example, the feature fusion module can employ a lightweight multi-scale feature pyramid network, which includes a PAN (Path Aggregation Network) module and an FPN (Feature Pyramid Network) module. Through the synergistic optimization of these two modules, efficient detection of multi-scale, complex-shaped text regions can be achieved.

[0059] When applying the feature fusion network described above, in order to significantly improve the feature fusion efficiency without increasing the computational load, the 1×1 convolutional compression layer before upsampling is eliminated. At the same time, a short connection path is established between FPN and PAN to ensure that feature information can be transmitted efficiently. However, this direct short connection method changes the original topology of FPN / PAN. The change in network topology will introduce a large number of intermediate format conversions, requiring frequent reading and writing of intermediate results.

[0060] Based on this, this embodiment unifies the output tensor layout of the upsampling layer and the element-wise operation layer, which have short connection paths, to merge the upsampling layer and the element-wise operation layer into a single GPU core. Taking the aforementioned lightweight multi-scale FPN+PAN structure as an example, graph analysis confirms that there are short connections between the upsampling layer (IResizeLayer) and the element-wise operation layer (IelementWiseLayer). To ensure fusion, the output tensor layout of the two is set to be the same, such as NCHW, so that the general inference framework can merge these two operations into a single computational core, avoiding any intermediate format conversion and improving computational efficiency.

[0061] Besides the aforementioned issue that the complexity of the model structure prevents existing general-purpose inference frameworks from meeting the deployment requirements of object detection models in live streaming scenarios, current deployment methods also suffer from the following problems: The model's post-processing workflow, including decoding bounding boxes from probability distributions and NMS (Non-Maximum Suppression), is essentially a computationally intensive algorithm. Traditional solutions place this logic on the CPU (Central Processing Unit), leading to two problems: 1. The massive amount of raw prediction tensors inferred by the model on the GPU needs to be copied to the CPU (Central Processing Unit), resulting in high data transmission latency; 2. The limited parallel capabilities of the CPU cannot efficiently process the massive number of candidate boxes, becoming the end-to-end performance bottleneck of the entire system and affecting the model's inference efficiency.

[0062] To address the aforementioned issues and improve model inference efficiency, the entire inference and post-processing process needs to be migrated to the GPU for execution. By GPU-based processing of all post-processing and decoding logic, CPU bottlenecks and CPU-GPU data interaction overhead are effectively eliminated.

[0063] Therefore, please refer to again Figure 2 The model deployment method provided in this application embodiment also includes steps S104 to S105.

[0064] S104 decodes the probability distribution tensor output by the detection head into a bounding box coordinate tensor within a single GPU core.

[0065] S105 decodes and filters multiple raw prediction tensors containing bounding box coordinates within a single GPU core, and outputs the detection result tensor.

[0066] The target detection model in this embodiment utilizes a decoupled detection head to separate coordinate regression and confidence prediction tasks, avoiding conflicts between tasks. It further improves detection accuracy by combining CIOU and DFL loss functions, and introduces a dynamic anchor box mechanism to expand the aspect ratio range of the anchor boxes. The combination of these techniques enables the target detection model to accurately and efficiently locate text regions of various shapes in highly complex live streaming scenarios, providing reliable input data for subsequent OCR recognition.

[0067] To address the aforementioned technical improvements and enhance model inference efficiency during deployment, the following tasks are required: 1. The loss function used during training must be removed during inference; 2. The discrete probability distribution output by the detection head must be efficiently decoded into bounding box coordinates; 3. The traditional CPU post-processing logic, including dynamic anchor box decoding, multi-scale prediction result aggregation, and final NMS, must be migrated to the GPU, with all processing performed within a single GPU core.

[0068] For the loss function, an automated script can be developed to accurately identify and remove all graph branches related to loss calculation before converting the model to a format supported by a general inference framework, such as after model training or after conversion to an intermediate model file, ensuring that the terminal node of the inference graph is the original prediction tensor of the model.

[0069] The decoding process, i.e., the implementation process of step S104, may include: using the dedicated GPU kernel in the second custom plugin, weighting and summing the probability distribution of each position in the probability distribution tensor output by the detection head with a predefined coordinate vector to generate a bounding box coordinate tensor.

[0070] In other words, this embodiment designs a custom plugin that receives the probability distribution tensor output by the detection head. Its internal dedicated GPU kernel performs a weighted summation of the probability distribution at each position in the probability distribution tensor with a predefined coordinate vector in parallel to generate a bounding box coordinate tensor.

[0071] For example, the detector outputs a probability distribution tensor of [B, 4, H, W, 17], where 4 represents the four sides of the bounding box, i.e., the distance from the center point to the top, bottom, left, and right sides; 17 is a characteristic of DFL (Distributed Softmax) models. Traditional models would directly regress a continuous floating-point value (e.g., the left margin is 10.5 pixels). However, in DFL, the model does not directly predict this value, but rather predicts the probability distribution of this value falling within a predefined range. Here, 17 means that this range is divided into 17 discrete intervals. For example, if the predefined distance range is 0 to 16 (a total of 17 integer values), then the model will predict 17 probability values ​​for the "left boundary distance," representing the probability of "distance being 0," "distance being 1," ..., "distance being 16." These 17 probability values, after being normalized by the Softmax function, sum to 1. Traditionally, this probability distribution tensor needs to be copied from the GPU to the CPU for computation, resulting in high data transfer latency.

[0072] This embodiment directly receives the probability distribution tensor [B, 4, H, W, 17] output by the detection head through a custom-designed plugin. Its internal dedicated GPU kernel (CUDA Kernel) performs a softmax operation on this 17-dimensional probability distribution (i.e., transforms it into weights), and then performs a weighted sum with a predefined vector representing possible coordinate values ​​(e.g., the dot product of the softmax result and a vector [0, 1, 2, ..., 16]). The final [B, H, W, 4] bounding box coordinate tensor is then calculated directly on the GPU, where 4 represents the decoded bounding box coordinates. In this way, the pixel-by-pixel loop decoding operation originally performed on the CPU is completely offloaded to the GPU, eliminating the massive data transfer between the CPU and GPU.

[0073] The post-processing process, namely the implementation process of step S105, may include: using a dedicated GPU kernel in the third custom plugin to perform threshold filtering and multi-class nonmaximum suppression on multiple original prediction tensors containing bounding box coordinates, and storing the obtained detection result tensors.

[0074] In other words, this embodiment designs a custom plugin that encapsulates the entire post-processing workflow into a single plugin. The input to this plugin is multiple raw prediction tensors output by the detection head (including confidence scores and bounding box coordinate tensors decoded in step S104). Its internal dedicated GPU core implements multi-stage parallel processing logic, including: 1) efficient threshold filtering; 2) optimized multi-class NMS algorithm; and 3) compact storage of the final detection results. The output of this plugin is a formatted final detection result tensor [Num_detections, 7], which can be directly used downstream. In [Num_detections, 7], Num_detections represents the total number of targets detected in the entire batch, and 7 represents the 7 attribute values ​​required for a detection result, namely, batch_idx (which image in the batch the detection box belongs to (index from 0 to B-1)), 4 coordinates, score confidence, and class (text). Through the above process, the originally fragmented and high-latency CPU post-processing workflow is transformed into a single, efficient GPU core call.

[0075] By deploying the object detection model to the GPU using the model deployment methods described above, it can be applied to tasks such as content moderation in live streaming scenarios.

[0076] Please refer to Figure 4 , Figure 4 This illustration shows a flowchart of a video review method provided in an embodiment of this application. This video review method can be applied to… Figure 1The live streaming server in the middle includes the following steps: S201, Obtain the video frames to be reviewed from the live video stream.

[0077] S202, input the video frame to be reviewed into the target detection model deployed by the method described in the aforementioned embodiments, and determine the location of the text in the video to be reviewed.

[0078] S203, Review the video frames based on the image of the location of the text in the video to be reviewed.

[0079] In this embodiment, video frames to be reviewed are first extracted from the live video stream for content review, avoiding frame-by-frame review and ensuring system performance. Then, the target detection model deployed by the method described above is called to perform text detection to determine the location of the text in the video to be reviewed. Next, text recognition is performed on the image of the location of the text in the video frame to be reviewed to improve the efficiency and accuracy of text recognition. Finally, multimodal sensitive semantic analysis is performed on the text recognition results to determine whether there is any illegal content. After illegal content is identified, the video frame to be reviewed and its information are marked and reported for review by the reviewers.

[0080] Compared with the prior art, the model deployment method provided in this application has the following advantages: First, by refactoring and deeply optimizing specific components in general object detection in live streaming scenarios, an effective performance improvement was achieved compared to the general optimization process. Secondly, it seamlessly integrates complex strategies (such as the DFL loss function) in the training phase with streamlined and efficient paths in the inference phase, and ensures the correctness and optimal performance of the inference logic through plug-in architecture. Third, by GPUing all post-processing and decoding logic, the CPU bottleneck and CPU-GPU data interaction overhead are effectively eliminated.

[0081] In order to perform the corresponding steps in the above method embodiments and various possible implementations, the following provides an implementation method for a model deployment device and a video review device.

[0082] Please refer to Figure 5 , Figure 5 The diagram shows a block diagram of a model deployment device 100 provided in an embodiment of this application, including: a model acquisition module 101, a feature extraction network processing module 102, and a feature fusion network processing module 103.

[0083] The model acquisition module 101 is used to acquire the target detection model to be deployed. The target detection model includes a feature extraction network and a feature fusion network. The feature extraction network processing module 102 is used to reconstruct the topology of the feature extraction network and optimize its tensor layout.

[0084] The feature fusion network processing module 103 is used to merge the upsampling layer and the element-wise operation layer with short connection paths in the feature fusion network into a single GPU core.

[0085] Optionally, the feature extraction network includes a convolutional module, a multi-layer pooling module, and a multi-gradient feature combination module; the feature extraction network processing module 102 is specifically used for: The basic feature extraction sequences in the convolutional module are merged into a single GPU kernel, and the input tensor layout of the stitching layer is adjusted to ensure memory continuity. Maintain the maximum pooling serial cascade structure in the multi-layer pooling module unchanged; All input tensors of the multi-gradient feature combination module are processed through a dedicated GPU kernel in the first custom plugin.

[0086] Optionally, the feature fusion network processing module 103 is specifically used for: Find upsampling layers and element-wise operation layers with short connection paths in feature fusion networks; The output tensor layout of the upsampling layer and the element-wise operation layer is unified to merge the upsampling layer and the element-wise operation layer into a single GPU kernel.

[0087] Optionally, the target detection model also includes a detection head, and the model deployment device 100 provided in this application embodiment also includes a post-processing module 104.

[0088] The post-processing module 104 is used to decode the probability distribution tensor output by the detection head into a bounding box coordinate tensor in a single GPU core; and to decode and filter multiple original prediction tensors containing bounding box coordinates in a single GPU core, and output the detection result tensor.

[0089] Optionally, the post-processing module 104 executes in a single GPU core to decode the probability distribution tensor output by the detection head into a bounding box coordinate tensor, including: using a dedicated GPU core in the second custom plugin to perform a weighted summation of the probability distribution at each position in the probability distribution tensor output by the detection head with a predefined coordinate vector to generate a bounding box coordinate tensor.

[0090] Optionally, the post-processing module 104 executes in a single GPU core to decode and filter multiple raw prediction tensors containing bounding box coordinates and output detection result tensors in the following ways: by using a dedicated GPU core in a third custom plugin to perform threshold filtering and multi-class nonmaximum suppression on multiple raw prediction tensors containing bounding box coordinates, and storing the obtained detection result tensors.

[0091] Please refer to Figure 6 , Figure 6 A block diagram of a video review device 200 provided in an embodiment of this application is shown, including: a video frame acquisition module 201, a text detection module 202, and a review module 203.

[0092] The video frame acquisition module 201 is used to acquire video frames to be reviewed from the live video stream.

[0093] The text detection module 202 is used to input the video frame to be reviewed into the text target detection model deployed by the method described in the foregoing embodiments, and to determine the location of the text in the video to be reviewed.

[0094] The review module 203 is used to review video frames based on the image of the location of the text in the video to be reviewed.

[0095] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the model deployment device 100 and the video review device 200 described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0096] Please refer to Figure 7 , Figure 7 A block diagram of an electronic device 10 provided in an embodiment of this application is shown. The electronic device 10 may be... Figure 1 The live streaming server in the video can also be a terminal device (e.g., a personal computer) of a technician who communicates with the live streaming server. The electronic device 10 includes a processor 11, a memory 12, and a bus 13, with the processor 11 connected to the memory 12 via the bus 13.

[0097] Memory 12 is used to store programs, for example Figure 5 The model deployment device 100 shown, or Figure 6 The video review device 200 shown. Taking the model deployment device 100 as an example, it includes at least one software function module that can be stored in the memory 12 in the form of software or firmware. After receiving the execution instruction, the processor 11 executes the program to implement the model deployment method disclosed in the foregoing embodiments.

[0098] The memory is used to store programs or data. The memory may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc.

[0099] Processor 11 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed through integrated logic circuits in the hardware of processor 11 or through software instructions. Processor 11 can be a general-purpose processor, including a Central Processing Unit (CPU), a Microcontroller Unit (MCU), a Complex Programmable Logic Device (CPLD), a Field Programmable Gate Array (FPGA), embedded ARM chips, etc.

[0100] Understandable. Figure 7 The structure shown is for illustrative purposes only; the electronic device 10 may also include components that are more advanced than those shown. Figure 7 The more or fewer components shown, or having the same Figure 7 The different configurations shown. Figure 7 The components shown can be implemented using hardware, software, or a combination thereof.

[0101] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by the processor 11, implements the model deployment method or video review method described in the foregoing embodiments.

[0102] This application also provides a program product that, when executed by the processor 11, implements the model deployment method or video review method described in the foregoing embodiments.

[0103] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0104] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0105] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0106] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A model deployment method, characterized by, The method comprises: obtaining a target detection model to be deployed, the target detection model comprising a feature extraction network and a feature fusion network; reconstructing the topology of the feature extraction network and optimizing the tensor layout thereof; merging the upsampling layer and the element-wise operation layer with a short connection path in the feature fusion network into a single GPU kernel.

2. The model deployment method of claim 1, wherein, The feature extraction network comprises a convolution module, a multi-layer pooling module, and a multi-gradient feature combination module; reconstructing the topology of the feature extraction network and optimizing the tensor layout thereof comprises: merging the base feature extraction sequence in the convolution module into a single GPU kernel, and adjusting the input tensor layout of the splicing layer to ensure memory continuity; keeping the maximum pooling serial cascade structure in the multi-layer pooling module unchanged; processing all input tensors of the multi-gradient feature combination module through a special GPU kernel in a first custom plug-in.

3. The model deployment method of claim 1, wherein, merging the upsampling layer and the element-wise operation layer with a short connection path in the feature fusion network into a single GPU kernel comprises: finding the upsampling layer and the element-wise operation layer with a short connection path in the feature fusion network; unifying the output tensor layout of the upsampling layer and the element-wise operation layer to merge the upsampling layer and the element-wise operation layer into a single GPU kernel.

4. The model deployment method of any one of claims 1-3, wherein, The target detection model further comprises a detection head, and the method further comprises: decoding a probability distribution tensor output by the detection head into a bounding box coordinate tensor in a single GPU kernel; decoding and filtering a plurality of original prediction tensors containing the bounding box coordinates in a single GPU kernel to output a detection result tensor.

5. The model deployment method of claim 4, wherein, decoding a probability distribution tensor output by the detection head into a bounding box coordinate tensor in a single GPU kernel comprises: weighting and summing the probability distribution of each position in the probability distribution tensor output by the detection head with a predefined coordinate vector through a special GPU kernel in a second custom plug-in to generate a bounding box coordinate tensor.

6. The model deployment method of claim 4, wherein, decoding and filtering a plurality of original prediction tensors containing the bounding box coordinates in a single GPU kernel to output a detection result tensor comprises: threshold filtering and multi-class non-maximum suppression on the plurality of original prediction tensors containing the bounding box coordinates through a special GPU kernel in a third custom plug-in, and storing the obtained detection result tensor.

7. A method of video auditing, the method comprising: The method comprises: obtaining a video frame to be audited from a live video stream; inputting the video frame to be audited into the target detection model deployed through the method of any one of claims 1-6 to determine the position of the text in the video frame to be audited; auditing the video frame to be audited based on the image at the position of the text in the video frame to be audited.

8. A model deployment apparatus characterized by comprising: The device comprises: a model obtaining module configured to obtain a target detection model to be deployed, the target detection model comprising a feature extraction network and a feature fusion network; a feature extraction network processing module configured to reconstruct the topology of the feature extraction network and optimize the tensor layout thereof; a feature extraction network processing module configured to reconstruct the topology of the feature extraction network and optimize the tensor layout thereof; The feature fusion network processing module is configured to combine the up-sampling layer and the element-by-element operation layer with a short connection path in the feature fusion network into a single GPU kernel.

9. A video auditing apparatus characterized by comprising: The device comprises: The video frame acquisition module is configured to acquire a to-be-audited video frame from a live video stream. The text detection module is configured to input the to-be-audited video frame into a text detection model deployed by the model deployment method of any one of claims 1-6, and determine the position of the text in the to-be-audited video frame. The auditing module is configured to audit the to-be-audited video frame based on the image of the position of the text in the to-be-audited video frame.

10. An electronic device, comprising: The processor and the memory, wherein the memory stores machine executable instructions capable of being executed by the processor, and the processor is capable of executing the machine executable instructions to implement the model deployment method of any one of claims 1-6 or the video auditing method of claim 7.

11. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the model deployment method of any one of claims 1-6 or the video auditing method of claim 7.

Images