A multi-target detection method and related equipment

Abstract

The present application discloses a multi-target detection method and related equipment, the method comprising: obtaining a monitoring scene image; inputting the monitoring scene image into a pre-trained multi-target detection model to generate a target recognition result for the monitoring scene image, wherein the multi-target detection model is composed of a detection output network, four multi-level aggregation bottleneck networks, a fast spatial pyramid pooling network, two upsampling networks and two branch processing networks, with a large number of sample monitoring scene images annotated with target objects as training data, and multi-head training based on a loss function composed of a rectangular box center point error, a prediction box width and height error, a prediction box confidence loss and a prediction box category loss. The application can significantly speed up the detection speed, is easier to deploy on resource-constrained devices, reduces hardware costs and energy consumption, and has strong robustness.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and more specifically, to a multi-target detection method and related equipment. Background Art

[0002] In today's digital age, intelligent monitoring systems have been widely used in many fields such as traffic monitoring, public safety, and smart homes. In these scenarios, multi-target detection technology, as a key link in intelligent monitoring systems, undertakes the important mission of quickly and accurately identifying various objects (including pedestrians, vehicles, animals, etc.) from continuous video streams. It plays an important role in ensuring traffic safety, maintaining social order, and improving the level of intelligent life.

[0003] However, when the existing target detection methods are actually applied to monitoring scenarios, they expose many problems that need to be solved. First, from the perspective of computing resources, video surveillance equipment often needs to be deployed on embedded or edge devices, which have relatively limited computing resources. However, the current mainstream target detection algorithms, such as Faster R-CNN and YOLO series, have high requirements for computing resources due to their complex network structure and large-scale parameter calculations. This makes it difficult to meet the real-time processing requirements required by the monitoring scenarios during actual operation, which seriously restricts the response speed of the monitoring system and makes it impossible to detect and warn various targets in a timely and effective manner.

[0004] Secondly, in terms of real-time performance and energy efficiency optimization, many monitoring application scenarios have clearly put forward millisecond-level detection response time requirements, while expecting to minimize computing costs and thus reduce hardware power consumption. Unfortunately, existing detection methods often ignore energy efficiency optimization in the pursuit of high detection accuracy, and it is difficult to achieve a balance between the two. Even if some methods attempt to make improvements, they still cannot effectively optimize energy efficiency while ensuring detection accuracy, which undoubtedly limits the large-scale promotion and long-term stable operation of intelligent monitoring systems.

[0005] In order to overcome these challenges, researchers have actively explored a variety of technical paths, including model pruning, quantization, and knowledge distillation, aiming to lightweight the target detection algorithm so that it can better adapt to resource-constrained hardware environments. Although these methods reduce the computational and storage requirements of the model to a certain extent, it is found in practice that the model lightweighting process is often accompanied by a significant decrease in detection accuracy. In particular, for small targets and occluded targets that frequently appear in monitoring scenes, the detection effect is more seriously affected. This greatly reduces the reliability of the lightweight model in practical applications and cannot meet the strict requirements of the monitoring system for detection accuracy.

[0006] In summary, the existing multi-target detection technology has problems such as limited computing resources, difficulty in balancing real-time and energy efficiency, and serious loss of accuracy after model lightweighting when facing the actual needs of intelligent monitoring systems. Therefore, there is an urgent need for a new solution that can achieve efficient and accurate multi-target detection in a resource-constrained hardware environment and maintain good robustness during the model lightweighting process. Summary of the invention

[0007] In view of this, the present application provides a multi-target detection method and related equipment, which can significantly speed up the detection speed, is easier to deploy on resource-constrained devices, reduces hardware costs and energy consumption, and has strong robustness.

[0008] A multi-target detection method, comprising:

[0009] Acquire monitoring scene images;

[0010] The monitoring scene image is input into a pre-trained multi-target detection model to generate a target recognition result for the monitoring scene image, wherein the multi-target detection model is composed of a detection output network, four multi-level aggregation bottleneck networks, a fast spatial pyramid pooling network, two upsampling networks and two branch processing networks, and a large number of sample monitoring scene images marked with target objects are used as training data. The model is obtained by multi-head training based on a loss function composed of a rectangular box center point error, a prediction box width and height error, a prediction box confidence loss and a prediction box category loss.

[0011] Optionally, the detection output network, the four multi-level aggregation bottleneck networks, the fast spatial pyramid pooling network and the two upsampling networks in the multi-target detection model are connected in series in sequence;

[0012] The aggregate analysis results of the second multi-level aggregate bottleneck network are input to the second upper adoption network, and the aggregate analysis results of the third multi-level aggregate bottleneck network are input to the first upper adoption network;

[0013] The upsampling analysis results of the two upsampling networks are input into the first branch processing network, and the pyramid pooling result of the fast spatial pyramid pooling network and the branch processing result of the first branch processing network are input into the second branch processing network.

[0014] Optionally, the detection output network includes two detection output modules;

[0015] The multi-level aggregation bottleneck network includes a multi-level aggregation bottleneck module and a detection output module;

[0016] The fast spatial pyramid pooling network includes a fast spatial pyramid pooling module and a detection output module;

[0017] The upsampling network includes an upsampling layer, a connection layer, and a multi-level aggregation bottleneck module connected in sequence;

[0018] The branch processing network includes a detection output module, a connection layer and a multi-level aggregation bottleneck module which are connected in sequence.

[0019] Optionally, the multi-level aggregation bottleneck module includes six detection output layers and a connection layer;

[0020] The input node is connected to the first detection output layer, the first detection output layer is connected to the second detection output layer and the third detection output layer respectively, and the second detection output layer is connected to the fourth detection output layer and the fifth detection output layer in sequence;

[0021] The connection layer receives the detection results of the second detection output layer, the detection results of the third detection output layer, and the summary detection results obtained by adding the detection results of the second detection output layer and the fifth detection output layer element by element, and inputs them to the connected sixth detection output layer after a splicing operation.

[0022] Optionally, the fast spatial pyramid pooling module includes three maximum pooling layers and one connection layer;

[0023] The input nodes are respectively connected to the first maximum pooling layer, and the first maximum pooling layer, the second maximum pooling layer and the third maximum pooling layer are connected in series in sequence;

[0024] The connection layer receives the input data of the first maximum pooling layer and the pooling results of the three maximum pooling layers and performs a splicing operation.

[0025] Optionally, the detection output module is composed of a convolution layer, a batch normalization layer and an activation function layer connected in series.

[0026] A multi-target detection device, comprising:

[0027] An image acquisition unit, used to acquire images of monitoring scenes;

[0028] The target detection unit is used to input the monitoring scene image into a pre-trained multi-target detection model to generate a target recognition result for the monitoring scene image, wherein the multi-target detection model is composed of a detection output network, four multi-level aggregation bottleneck networks, a fast spatial pyramid pooling network, two upsampling networks and two branch processing networks, and uses a large number of sample monitoring scene images marked with target objects as training data, and is obtained by multi-head training based on a loss function composed of a rectangular box center point error, a prediction box width and height error, a prediction box confidence loss and a prediction box category loss.

[0029] A multi-target detection device, comprising a memory and a processor;

[0030] The memory is used to store programs;

[0031] The processor is used to execute the program to implement each step of the multi-target detection method as described in any one of the above items.

[0032] A readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, each step of the multi-target detection method as described in any one of the above items is implemented.

[0033] A computer program product comprises a computer program, wherein when the computer program is run by a processor, each step of the multi-target detection method as described above is executed.

[0034] It can be seen from the above technical solutions that a multi-target detection method and related equipment provided by the embodiment of the present application adopts a specific network structure consisting of a detection output network, a multi-level aggregation bottleneck network, a fast spatial pyramid pooling network, an upsampling network, and a branch processing network, and cooperates with corresponding optimization strategies. Compared with the existing target detection method, it can significantly speed up the detection speed, meet the strict requirements of the millisecond detection response time in the monitoring scene, and promptly identify various target objects such as pedestrians, vehicles, animals, etc. in the monitoring scene image, and ensure the timeliness and effectiveness of monitoring. The overall network structure design and training method of the present application make the scale of the model reasonably optimized. Compared with traditional target detection models that require large computing resources, under the premise of ensuring detection performance, the model of the present invention has a smaller volume, reduces the occupation of resources such as hardware storage, is easier to be deployed on resource-constrained devices, and reduces hardware costs and energy consumption.

[0035] In addition, the present application has strong robustness and can maintain good detection effects when facing complex situations such as small targets and occluded targets that are common in monitoring scenarios. It effectively overcomes the problem that some existing models have a significant decrease in detection accuracy for such targets during the lightweight process, and demonstrates strong adaptability and stability, ensuring the reliable execution of multi-target detection tasks in various actual monitoring scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0037] Figure 1 A flowchart of a multi-target detection method disclosed in an embodiment of the present application;

[0038] Figure 2 A schematic diagram of the model structure of a multi-target detection model disclosed in an embodiment of the present application;

[0039] Figure 3 A schematic diagram of another model structure of a multi-target detection model disclosed in an embodiment of the present application;

[0040] Figure 4 A schematic diagram of the structure of a multi-level aggregation bottleneck module disclosed in an embodiment of the present application;

[0041] Figure 5 A schematic diagram of the structure of a fast spatial pyramid pooling module disclosed in an embodiment of the present application;

[0042] Figure 6 A schematic diagram of the structure of a detection output module disclosed in an embodiment of the present application;

[0043] Figure 7 A schematic diagram of a multi-target detection device disclosed in an embodiment of the present application;

[0044] Figure 8 This is a hardware structure block diagram of a multi-target detection device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0046] The present application can be used in many general or special computing device environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multi-processor devices, distributed computing environments including any of the above devices or equipment, etc.

[0047] An embodiment of the present application provides a multi-target detection method, which can be applied to platforms or applications such as monitoring systems, and can also be applied to various computer terminals or smart terminals. The execution entity can be a processor or server of the computer terminal or smart terminal.

[0048] Next, the present application scheme is introduced. The present application proposes the following technical scheme, please see below for details.

[0049] Figure 1 This is a flowchart of a multi-target detection method disclosed in an embodiment of the present application.

[0050] like Figure 1 As shown, the method may include:

[0051] Step S1, obtaining a monitoring scene image.

[0052] Specifically, high-definition cameras deployed in the monitoring area continuously shoot the monitoring scene at a fixed frame rate to obtain a series of continuous raw image data. These cameras have functions such as autofocus, auto exposure and image enhancement, and can adapt to different lighting conditions and weather conditions to ensure that the collected images are clear, complete and have sufficient detail information, providing a high-quality input data source for subsequent target detection. At the same time, the acquired images are preprocessed in real time, including but not limited to image denoising, grayscale and image size normalization, and the image resolution is uniformly adjusted to the standard size required by the model, so that they can be more efficiently input into the multi-target detection model for analysis and processing.

[0053] Step S2: input the monitoring scene image into a pre-trained multi-target detection model to generate a target recognition result for the monitoring scene image.

[0054] Specifically, the multi-target detection model consists of a detection output network, four multi-level aggregation bottleneck networks, a fast spatial pyramid pooling network, two upsampling networks and two branch processing networks. It uses a large number of sample surveillance scene images labeled with target objects as training data, and is obtained through multi-head training based on a loss function composed of rectangular box center point error, prediction box width and height error, prediction box confidence loss and prediction box category loss.

[0055] In the model training phase, a large number of sample surveillance scene images with target objects labeled from different surveillance scenes are collected. These samples cover various common target categories (such as pedestrians, vehicles, animals, etc.) and different target postures, sizes and occlusion conditions to ensure the diversity and comprehensiveness of the training data. Multi-head training is performed based on the loss function consisting of the rectangular box center point error, the prediction box width and height error, the prediction box confidence loss and the prediction box category loss. In each round of training, the model calculates the errors between the prediction box and the true annotation box through forward propagation, and updates and optimizes the model parameters through the back propagation algorithm based on these errors. After multiple rounds of repeated training, the model can accurately learn the feature representation and positioning information of different targets. In practical applications, it can quickly and accurately generate target recognition results for the input surveillance scene images, including the target category, location (presented in the form of rectangular box coordinates) and confidence score, providing strong support for subsequent monitoring decisions and event responses.

[0056] The following is a detailed introduction to the model structure of the multi-target detection model.

[0057] Figure 2 A schematic diagram of the model structure of a multi-target detection model disclosed in an embodiment of the present application.

[0058] like Figure 2 As shown, the detection output network, the four multi-level aggregation bottleneck networks, the fast spatial pyramid pooling network and the two upsampling networks in the multi-target detection model are connected in series in sequence;

[0059] The aggregate analysis results of the second multi-level aggregate bottleneck network are input to the second upper adoption network, and the aggregate analysis results of the third multi-level aggregate bottleneck network are input to the first upper adoption network;

[0060] The upsampling analysis results of the two upsampling networks are input into the first branch processing network, and the pyramid pooling result of the fast spatial pyramid pooling network and the branch processing result of the first branch processing network are input into the second branch processing network.

[0061] Specifically, the detection output network, the four multi-level aggregation bottleneck networks, the fast spatial pyramid pooling network and the two upsampling networks are sequentially connected in series. In this process, the image data first enters the first multi-level aggregation bottleneck network, which performs preliminary feature extraction and aggregation analysis on the input data through its unique structure, and passes the processed results to the next multi-level aggregation bottleneck network. After continuous processing by the four multi-level aggregation bottleneck networks, the features of the data are gradually refined and enhanced.

[0062] Next, the second multi-level aggregation bottleneck network (the fourth multi-level aggregation bottleneck network connected in sequence in the model), after completing its own aggregation analysis, inputs its results into the second upsampling network (the second one connected to the upsampling network in the model), and the aggregation analysis results of the third multi-level aggregation bottleneck network are input into the first upsampling network. This connection method enables feature information at different levels to be introduced into the upsampling process at the appropriate stage, which helps to restore the detailed information of the image and better integrate features of different scales, providing rich feature representation for subsequent accurate detection.

[0063] Then, after the two upsampling networks complete their respective upsampling analyses, their results are input into the first branch processing network. The first branch processing network integrates and further analyzes the information from the two upsampling networks to mine potential target-related features and information associations.

[0064] At the same time, the fast spatial pyramid pooling network performs a pyramid pooling operation on the data processed by the previous multi-level aggregation bottleneck network, the purpose of which is to pool and fuse features at different scales to obtain a feature map with multi-scale information. The pyramid pooling result is input into the second branch processing network together with the branch processing result of the first branch processing network. The second branch processing network, as a key integration node in the model, comprehensively fuses and analyzes feature information from different paths, and finally passes the processed results to the detection output network, which generates detailed recognition results such as the category, position, and confidence of the target object in the monitoring scene image, thereby completing the entire multi-target detection process and achieving efficient and accurate detection of various targets in the monitoring scene.

[0065] This network structure design fully utilizes the advantages of each network component through reasonable module connection and information transmission mechanism, effectively improving the model's detection capability and accuracy for multiple targets in complex monitoring scenarios, enabling it to better adapt to the diverse needs in practical applications.

[0066] Among them, further, such as Figure 3 As shown, the detection output network includes two detection output modules (CBL);

[0067] The multi-level aggregation bottleneck network includes a multi-level aggregation bottleneck module (MLA) and a detection output module (CBL);

[0068] The fast spatial pyramid pooling network includes a fast spatial pyramid pooling module (SPPF) and a detection output module (CBL);

[0069] The upsampling network includes an upsampling layer (Upsanple), a connection layer (Concat) and a multi-level aggregation bottleneck module (MLA) connected in sequence;

[0070] The branch processing network includes a detection output module (CBL), a connection layer (Concat) and a multi-level aggregation bottleneck module (MLA) which are connected in sequence.

[0071] Specifically, the detection output network contains two detection output modules, which jointly perform the final analysis and decision on the feature information processed by the entire model to generate accurate recognition results of targets in the monitored scene images, including key information such as the target category, location, and confidence level. It can effectively process complex feature data from other network modules to ensure the reliability and accuracy of the output results.

[0072] The multi-level aggregation bottleneck network integrates a multi-level aggregation bottleneck module and a detection output module. As the core part of the network, the multi-level aggregation bottleneck module can effectively extract and condense the key feature information in the image through multi-level feature aggregation and bottleneck structure design, and optimize the convolution parameters while keeping the feature map resolution unchanged, thereby improving the feature expression ability and computational efficiency. The detection output module performs preliminary detection and analysis on the features processed by the aggregation bottleneck module at a specific stage of each multi-level aggregation bottleneck network, providing more targeted and valuable feature information for subsequent network processing. It also plays a certain role in monitoring and feedback of intermediate results, which helps to stabilize the training and performance improvement of the entire model.

[0073] The fast spatial pyramid pooling network consists of a fast spatial pyramid pooling module and a detection output module. The fast spatial pyramid pooling module uses the maximum pooling layer to achieve feature fusion, and can efficiently pool features at different scales to obtain feature maps with multi-scale information, which is essential for detecting target objects of different sizes. The role of the detection output module here is to further screen and organize the features after pooling, extract features closely related to the target, and transmit more discriminative information to the subsequent network, thereby enhancing the model's perception of the target and positioning accuracy.

[0074] The upsampling network includes an upsampling layer, a connection layer, and a multi-level aggregation bottleneck module connected in sequence. The upsampling layer is responsible for enlarging the size of the input feature map and restoring some detailed information of the image to make up for the information that may be lost in the previous processing. The connection layer plays a role of transition and information integration, effectively connecting and transferring information between the output of the upsampling layer and the subsequent multi-level aggregation bottleneck module to ensure the fluency and integrity of the data. After receiving the features after upsampling and connection processing, the multi-level aggregation bottleneck module aggregates and optimizes the features again to further improve the quality and expression ability of the features, and provide richer and more valuable feature data for subsequent network processing, thereby better supporting target detection tasks.

[0075] The branch processing network includes a detection output module, a connection layer and a multi-level aggregation bottleneck module connected in sequence. The detection output module first performs preliminary detection and analysis on the input feature information, extracts possible target-related features and potential target area information, and provides important clues and directions for subsequent processing. The connection layer effectively integrates and transmits the results of the detection output module with the input of other networks to ensure that feature information from different sources can be fully integrated and utilized in the branch processing network. The multi-level aggregation bottleneck module performs deep aggregation and optimization processing on the features integrated by the connection layer, further explores the association and potential information between the features, improves the model's understanding and processing capabilities for complex target scenes, and ultimately provides high-quality feature output for the entire multi-target detection model to achieve accurate identification and positioning of targets in monitoring scenes.

[0076] Multi-level aggregation bottleneck module (MLA):

[0077] like Figure 4 As shown, the multi-level aggregation bottleneck module includes six detection output layers (CBL-c*k*s*) and a connection layer (Concat);

[0078] The input node is connected to the first detection output layer, the first detection output layer is connected to the second detection output layer and the third detection output layer respectively, and the second detection output layer is connected to the fourth detection output layer and the fifth detection output layer in sequence;

[0079] The connection layer receives the detection results of the second detection output layer, the detection results of the third detection output layer, and the summary detection results obtained by adding the detection results of the second detection output layer and the fifth detection output layer element by element, and inputs them to the connected sixth detection output layer after a splicing operation.

[0080] Specifically, the multi-level aggregation bottleneck module plays a key role in feature extraction and integration in the entire multi-target detection model. The module is mainly composed of six detection output layers and one connection layer, and the layers work together through specific connection methods.

[0081] First, the input node passes the data to the first detection output layer. As the initial processing unit of the data, the first detection output layer has a specific convolution kernel and parameter settings, which is used to perform preliminary feature extraction and screening on the input data to highlight the key information and potential features in the image. Subsequently, the first detection output layer establishes connections with the second detection output layer and the third detection output layer respectively, and transmits the preliminarily processed data in a shunt, so that further feature mining and analysis can be performed in parallel on different paths.

[0082] After receiving the data from the first detection output layer, the second detection output layer continues to process the data in depth through its own convolution operation and feature transformation, and then connects the fourth detection output layer and the fifth detection output layer in sequence to form a continuous feature processing chain. In this chain, each layer extracts and optimizes the features more finely based on the previous layer, gradually enhancing the semantic information and discrimination ability of the features, for example, by adjusting the size, number, and step size of the convolution kernel to adapt to the feature extraction requirements of different scales and complexities.

[0083] At the same time, the connection layer serves as a data aggregation and integration center, which receives data from multiple sources, including the detection results of the second detection output layer, the detection results of the third detection output layer, and the summary detection results obtained by adding the detection results of the second detection output layer and the fifth detection output layer element by element. This data collection and integration method makes full use of the feature information on different paths, retains the details of local features, and integrates the contextual information of global features, thereby improving the richness and completeness of features.

[0084] Finally, the connection layer concatenates these integrated features to form a complete feature vector and inputs it to the connected sixth detection output layer. As the last layer of the multi-level aggregation bottleneck module, the sixth detection output layer performs the final processing and optimization on the input feature vector and outputs a feature map with high expressiveness and discrimination, providing high-quality input data for subsequent network modules, thereby ensuring the performance and accuracy of the entire multi-target detection model.

[0085] Fast Spatial Pyramid Pooling Module (SPPF):

[0086] like Figure 5 As shown, the fast spatial pyramid pooling module includes three maximum pooling layers (maxp-k*s*p*) and one connection layer (Concat);

[0087] The input nodes are respectively connected to the first maximum pooling layer, and the first maximum pooling layer, the second maximum pooling layer and the third maximum pooling layer are connected in series in sequence;

[0088] The connection layer receives the input data of the first maximum pooling layer and the pooling results of the three maximum pooling layers and performs a splicing operation.

[0089] Specifically, the fast spatial pyramid pooling module aims to efficiently process input data by extracting and fusing features at different scales to form feature information. The module consists of three maximum pooling layers and one connection layer.

[0090] First, the input nodes transfer the data to the first maximum pooling layer. As a downsampling operation, the maximum pooling layer can reduce the dimension and amount of calculation of the data while retaining the significant features. The first maximum pooling layer performs a pooling operation on the input data according to the pre-set pooling kernel size, extracts the maximum value feature in the local area, and thus highlights the key texture and shape information in the image.

[0091] Next, the output of the first maximum pooling layer is used as the input of the second maximum pooling layer, which also performs pooling operations according to the established parameter configuration to further screen the features and enhance the robustness and representativeness of the features. Subsequently, the output of the second maximum pooling layer is passed to the third maximum pooling layer. After the three layers of maximum pooling operations connected in series, the key features of the data at different scales are gradually extracted and integrated to form a multi-scale feature representation, which is of great significance for detecting target objects of different sizes and shapes.

[0092] At the same time, the connection layer receives the original input data from the first maximum pooling layer and the pooling results of the three maximum pooling layers, and concatenates these data. In this way, the connection layer fuses the feature information at different scales, retaining the detailed information in the original data and incorporating the high-level features after pooling, so that the output feature vector contains rich multi-scale information, which can better cope with the diversity and complexity of target objects in the target detection process.

[0093] Finally, the feature data after splicing through the connection layer is passed to the subsequent network modules, providing strong feature support for the entire multi-target detection model, helping to improve the model's detection accuracy and recall rate of targets, especially when facing targets of different sizes and postures in complex monitoring scenarios, enabling more accurate identification and positioning.

[0094] The design of the fast spatial pyramid pooling module effectively improves the performance of the model in multi-target detection tasks by rationally utilizing the downsampling characteristics of the maximum pooling layer and the feature fusion function of the connection layer, enabling it to better adapt to various complex situations in practical applications.

[0095] Detection output module (CBL):

[0096] like Figure 6 As shown in the figure, the detection output module is composed of a convolutional layer (conv), a batch normalization layer (bn) and an activation function layer (leaky-relu) connected in series.

[0097] Specifically, the detection output module is responsible for the final analysis and decision-making of the input feature data to generate accurate target detection results. This module consists of a series of convolutional layers, batch normalization layers, and activation function layers, which work together to complete feature conversion and information extraction.

[0098] First, the convolution layer, as the basic unit of feature extraction, performs convolution operations by sliding a set of learnable convolution kernels on the input feature map. The parameters of the convolution kernel are continuously optimized and adjusted during the model training process to extract specific feature patterns related to the target in the input data, such as edge, texture, shape and other information, thereby enhancing the expressiveness and discriminability of the features and providing high-quality feature input for subsequent processing.

[0099] Next, the batch normalization layer normalizes the feature data output by the convolutional layer. During the training process, it calculates the mean and variance of each batch of data and standardizes the features to make the data distribution more stable, accelerate the convergence of the model, and reduce the risk of overfitting to a certain extent. In this way, the batch normalization layer helps to improve the training efficiency and generalization ability of the model, so that the model can show more stable performance in different data sets and scenarios.

[0100] Finally, the activation function layer introduces nonlinear transformation to the feature data after batch normalization. Common activation functions include Leaky ReLU (Leaky Rectified Linear Unit). The role of the activation function is to introduce nonlinear factors to the model, so that the model can learn more complex feature representations and decision boundaries, thereby improving the model's adaptability and classification capabilities for complex target scenes. After being processed by the activation function, the feature data is further converted into more discriminative information, providing strong support for the final target detection result output.

[0101] Through the collaborative work of the convolutional layers, batch normalization layers, and activation function layers connected in series, the detection output module can effectively process and analyze the input features and convert them into target detection results with clear semantic information, including target category prediction, location information, and confidence scores, thereby providing a solid guarantee for the accuracy and reliability of the entire multi-target detection model.

[0102] It can be seen from the above technical solutions that a multi-target detection method and related equipment provided by the embodiment of the present application adopts a specific network structure consisting of a detection output network, a multi-level aggregation bottleneck network, a fast spatial pyramid pooling network, an upsampling network, and a branch processing network, and cooperates with corresponding optimization strategies. Compared with the existing target detection method, it can significantly speed up the detection speed, meet the strict requirements of the millisecond detection response time in the monitoring scene, and promptly identify various target objects such as pedestrians, vehicles, animals, etc. in the monitoring scene image, and ensure the timeliness and effectiveness of monitoring. The overall network structure design and training method of the present application make the scale of the model reasonably optimized. Compared with traditional target detection models that require large computing resources, under the premise of ensuring detection performance, the model of the present invention has a smaller volume, reduces the occupation of resources such as hardware storage, is easier to be deployed on resource-constrained devices, and reduces hardware costs and energy consumption.

[0103] In addition, the present application has strong robustness and can maintain good detection effects when facing complex situations such as small targets and occluded targets that are common in monitoring scenarios. It effectively overcomes the problem that some existing models have a significant decrease in detection accuracy for such targets during the lightweight process, and demonstrates strong adaptability and stability, ensuring the reliable execution of multi-target detection tasks in various actual monitoring scenarios.

[0104] In addition, for Figure 3 The parameters of each layer and the output size of the multi-target detection model shown in the following table:

[0105]

[0106] A multi-target detection device provided in an embodiment of the present application is described below. The multi-target detection device described below and the multi-target detection method described above can refer to each other.

[0107] See also Figure 7 , Figure 7 A schematic diagram of a multi-target detection device disclosed in an embodiment of the present application.

[0108] like Figure 7 As shown, the multi-target detection device may include:

[0109] An image acquisition unit 110 is used to acquire a monitoring scene image;

[0110] The target detection unit 120 is used to input the monitoring scene image into a pre-trained multi-target detection model to generate a target recognition result for the monitoring scene image, wherein the multi-target detection model is composed of a detection output network, four multi-level aggregation bottleneck networks, a fast spatial pyramid pooling network, two upsampling networks and two branch processing networks, and uses a large number of sample monitoring scene images marked with target objects as training data, and is obtained by multi-head training based on a loss function composed of a rectangular box center point error, a prediction box width and height error, a prediction box confidence loss and a prediction box category loss.

[0111] It can be seen from the above technical solutions that a multi-target detection method and related equipment provided by the embodiment of the present application adopts a specific network structure consisting of a detection output network, a multi-level aggregation bottleneck network, a fast spatial pyramid pooling network, an upsampling network, and a branch processing network, and cooperates with corresponding optimization strategies. Compared with the existing target detection method, it can significantly speed up the detection speed, meet the strict requirements of the millisecond detection response time in the monitoring scene, and promptly identify various target objects such as pedestrians, vehicles, animals, etc. in the monitoring scene image, and ensure the timeliness and effectiveness of monitoring. The overall network structure design and training method of the present application make the scale of the model reasonably optimized. Compared with traditional target detection models that require large computing resources, under the premise of ensuring detection performance, the model of the present invention has a smaller volume, reduces the occupation of resources such as hardware storage, is easier to be deployed on resource-constrained devices, and reduces hardware costs and energy consumption.

[0112] In addition, the present application has strong robustness and can maintain good detection effects when facing complex situations such as small targets and occluded targets that are common in monitoring scenarios. It effectively overcomes the problem that some existing models have a significant decrease in detection accuracy for such targets during the lightweight process, and demonstrates strong adaptability and stability, ensuring the reliable execution of multi-target detection tasks in various actual monitoring scenarios.

[0113] Optionally, the detection output network, the four multi-level aggregation bottleneck networks, the fast spatial pyramid pooling network and the two upsampling networks in the multi-target detection model are connected in series in sequence;

[0114] The aggregate analysis results of the second multi-level aggregate bottleneck network are input to the second upper adoption network, and the aggregate analysis results of the third multi-level aggregate bottleneck network are input to the first upper adoption network;

[0115] The upsampling analysis results of the two upsampling networks are input into the first branch processing network, and the pyramid pooling result of the fast spatial pyramid pooling network and the branch processing result of the first branch processing network are input into the second branch processing network.

[0116] Optionally, the detection output network includes two detection output modules;

[0117] The multi-level aggregation bottleneck network includes a multi-level aggregation bottleneck module and a detection output module;

[0118] The fast spatial pyramid pooling network includes a fast spatial pyramid pooling module and a detection output module;

[0119] The upsampling network includes an upsampling layer, a connection layer, and a multi-level aggregation bottleneck module connected in sequence;

[0120] The branch processing network includes a detection output module, a connection layer and a multi-level aggregation bottleneck module which are connected in sequence.

[0121] Optionally, the multi-level aggregation bottleneck module includes six detection output layers and a connection layer;

[0122] The input node is connected to the first detection output layer, the first detection output layer is connected to the second detection output layer and the third detection output layer respectively, and the second detection output layer is connected to the fourth detection output layer and the fifth detection output layer in sequence;

[0123] The connection layer receives the detection results of the second detection output layer, the detection results of the third detection output layer, and the summary detection results obtained by adding the detection results of the second detection output layer and the fifth detection output layer element by element, and inputs them to the connected sixth detection output layer after a splicing operation.

[0124] Optionally, the fast spatial pyramid pooling module includes three maximum pooling layers and one connection layer;

[0125] The input nodes are respectively connected to the first maximum pooling layer, and the first maximum pooling layer, the second maximum pooling layer and the third maximum pooling layer are connected in series in sequence;

[0126] The connection layer receives the input data of the first maximum pooling layer and the pooling results of the three maximum pooling layers and performs a splicing operation.

[0127] Optionally, the detection output module is composed of a convolution layer, a batch normalization layer and an activation function layer connected in series.

[0128] The multi-target detection apparatus provided in the embodiments of the present application can be applied to multi-target detection equipment. Figure 8The hardware structure diagram of the multi-target detection device is shown in FIG. Figure 8 ,The hardware structure of the multi-target detection device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4;

[0129] In the embodiment of the present application, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 communicate with each other through the communication bus 4;

[0130] The processor 1 may be a central processing unit CPU, or an application-specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention, etc.;

[0131] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), etc., such as at least one disk memory;

[0132] The memory stores a program, and the processor can call the program stored in the memory, wherein the program is used to:

[0133] Acquire monitoring scene images;

[0134] The monitoring scene image is input into a pre-trained multi-target detection model to generate a target recognition result for the monitoring scene image, wherein the multi-target detection model is composed of a detection output network, four multi-level aggregation bottleneck networks, a fast spatial pyramid pooling network, two upsampling networks and two branch processing networks, and a large number of sample monitoring scene images marked with target objects are used as training data. The model is obtained by multi-head training based on a loss function composed of a rectangular box center point error, a prediction box width and height error, a prediction box confidence loss and a prediction box category loss.

[0135] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0136] The embodiment of the present application further provides a readable storage medium, which may store a program suitable for execution by a processor, wherein the program is used to:

[0137] Acquire monitoring scene images;

[0138] The monitoring scene image is input into a pre-trained multi-target detection model to generate a target recognition result for the monitoring scene image, wherein the multi-target detection model is composed of a detection output network, four multi-level aggregation bottleneck networks, a fast spatial pyramid pooling network, two upsampling networks and two branch processing networks, and a large number of sample monitoring scene images marked with target objects are used as training data. The model is obtained by multi-head training based on a loss function composed of a rectangular box center point error, a prediction box width and height error, a prediction box confidence loss and a prediction box category loss.

[0139] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0140] The present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, the execution method is as follows:

[0141] Acquire monitoring scene images;

[0142] The monitoring scene image is input into a pre-trained multi-target detection model to generate a target recognition result for the monitoring scene image, wherein the multi-target detection model is composed of a detection output network, four multi-level aggregation bottleneck networks, a fast spatial pyramid pooling network, two upsampling networks and two branch processing networks, and a large number of sample monitoring scene images marked with target objects are used as training data. The model is obtained by multi-head training based on a loss function composed of a rectangular box center point error, a prediction box width and height error, a prediction box confidence loss and a prediction box category loss.

[0143] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0144] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0145] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0146] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-target detection method, characterized in that: include: Acquire monitoring scene images; The monitoring scene image is input into a pre-trained multi-target detection model to generate a target recognition result for the monitoring scene image, wherein the multi-target detection model is composed of a detection output network, four multi-level aggregation bottleneck networks, a fast spatial pyramid pooling network, two upsampling networks and two branch processing networks, and a large number of sample monitoring scene images marked with target objects are used as training data. The model is obtained by multi-head training based on a loss function composed of a rectangular box center point error, a prediction box width and height error, a prediction box confidence loss and a prediction box category loss.

2. The method according to claim 1, characterized in that The detection output network, the four multi-level aggregation bottleneck networks, the fast spatial pyramid pooling network and the two upsampling networks in the multi-target detection model are connected in series in sequence; The aggregate analysis results of the second multi-level aggregate bottleneck network are input to the second upper adoption network, and the aggregate analysis results of the third multi-level aggregate bottleneck network are input to the first upper adoption network; The upsampling analysis results of the two upsampling networks are input into the first branch processing network, and the pyramid pooling result of the fast spatial pyramid pooling network and the branch processing result of the first branch processing network are input into the second branch processing network.

3. The method according to claim 1 or 2, characterized in that: The detection output network includes two detection output modules; The multi-level aggregation bottleneck network includes a multi-level aggregation bottleneck module and a detection output module; The fast spatial pyramid pooling network includes a fast spatial pyramid pooling module and a detection output module; The upsampling network includes an upsampling layer, a connection layer, and a multi-level aggregation bottleneck module connected in sequence; The branch processing network includes a detection output module, a connection layer and a multi-level aggregation bottleneck module which are connected in sequence.

4. The method according to claim 3, characterized in that The multi-level aggregation bottleneck module includes six detection output layers and a connection layer; The input node is connected to the first detection output layer, the first detection output layer is connected to the second detection output layer and the third detection output layer respectively, and the second detection output layer is connected to the fourth detection output layer and the fifth detection output layer in sequence; The connection layer receives the detection results of the second detection output layer, the detection results of the third detection output layer, and the summary detection results obtained by adding the detection results of the second detection output layer and the fifth detection output layer element by element, and inputs them to the connected sixth detection output layer after a splicing operation.

5. The method according to claim 3, characterized in that: The fast spatial pyramid pooling module includes three maximum pooling layers and one connection layer; The input nodes are respectively connected to the first maximum pooling layer, and the first maximum pooling layer, the second maximum pooling layer and the third maximum pooling layer are connected in series in sequence; The connection layer receives the input data of the first maximum pooling layer and the pooling results of the three maximum pooling layers and performs a splicing operation.

6. The method according to claim 3, characterized in that The detection output module is composed of a convolution layer, a batch normalization layer and an activation function layer connected in series.

7. A multi-target detection device, characterized in that: include: An image acquisition unit, used to acquire images of monitoring scenes; The target detection unit is used to input the monitoring scene image into a pre-trained multi-target detection model to generate a target recognition result for the monitoring scene image, wherein the multi-target detection model is composed of a detection output network, four multi-level aggregation bottleneck networks, a fast spatial pyramid pooling network, two upsampling networks and two branch processing networks, and uses a large number of sample monitoring scene images marked with target objects as training data, and is obtained by multi-head training based on a loss function composed of a rectangular box center point error, a prediction box width and height error, a prediction box confidence loss and a prediction box category loss.

8. A multi-target detection device, characterized in that: including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the multi-target detection method according to any one of claims 1 to 6.

9. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the multi-target detection method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, each step of the multi-target detection method according to any one of claims 1 to 6 is executed.

Images