Target detection method, device and equipment and readable storage medium
By adding a dynamic graph message passing module to the backbone feature extraction module of the YOLOv6 network, combined with feature fusion and prediction modules, the problem of low accuracy of the YOLOv6 network is solved, and more efficient object detection is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- E-SURFING DIGITAL LIFE TECH CO LTD
- Filing Date
- 2022-12-20
- Publication Date
- 2026-05-12
AI Technical Summary
The YOLOv6 network is not accurate enough in object detection and has insufficient capture capability.
A Dynamic Graph Message Passing (DGMN) module is added to the EfficientRep Backbone backbone feature extraction module. By predicting the dynamic filters and affinity transformation matrices of feature nodes in video frames, feature fusion is performed in conjunction with the Rep-PAN Neck module, and the detection results are generated using the Head module.
It improves the target detection model's ability to capture more detailed and key features, thereby enhancing the accuracy and reliability of detection.
Smart Images
Figure CN115908853B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of Internet technology, and more specifically, to a target detection method, apparatus, device, and readable storage medium. Background Technology
[0002] Object detection is a fundamental problem in computer vision, tasked with identifying key objects in images or videos and determining their category and location. Due to its fundamental nature, object detection has numerous applications in various fields, including autonomous vehicles, medical imaging, urban security, identity verification, robot navigation, and motion analysis.
[0003] In recent years, the YOLOv6 network has been proposed for object detection. The YOLOv6 network consists of an EfficientRep Backbone module for feature extraction, a Rep-PAN Neck module for feature fusion, and a prediction head module. On the COCO dataset, YOLOv6 outperforms other algorithms of similar size in both accuracy and speed. However, despite its superior performance compared to other YOLO networks in object detection, YOLOv6's relatively low capture capability results in insufficient accuracy in object detection; that is, the accuracy of object detection networks in YOLOv6 is still not high enough. Summary of the Invention
[0004] In view of this, this application provides a target detection method, apparatus, device, and readable storage medium to improve the accuracy of target detection networks when performing target detection.
[0005] To achieve the above objectives, the following solution is proposed:
[0006] A target detection method, comprising:
[0007] Acquire video frames and an object detection model. The object detection model includes a backbone module, a feature fusion Rep-PAN Neck module, and a prediction head module. The backbone module is an EfficientRep Backbone module that adds a Dynamic Graph Message Passing (DGMN) module after the stem module.
[0008] The Backbone module is used to predict the transformation matrix of dynamic filters and affinity relationships for each feature node in the video frame, and random walk sampling is performed based on the transformation matrix to extract detailed features and key features;
[0009] The Rep-PAN Neck module is used to fuse the detailed features and key features to obtain fused data;
[0010] Using the Head module, the fused data is analyzed to generate detection results corresponding to the video frame. The detection results are either an empty set or video frames marked with the region and category of the target object. The target object is the object that needs to be focused on in the scene corresponding to the video frame.
[0011] Optionally, the step of using the Backbone module to predict the transformation matrix of dynamic filters and affinity relationships for each feature node of the video frame includes:
[0012] The video frames are downsampled using the stem module to obtain a sampled feature map;
[0013] Using the DGMN module, based on the sampled feature map, the dynamic association between each feature node is predicted, and based on the dynamic association between each feature node, the transformation matrix of the dynamic filter and affinity relationship of each feature node is jointly predicted.
[0014] Optionally, the DGMN module includes a uniform walk sampler and a random walk sampler;
[0015] The step of using the DGMN module to predict the dynamic associations between the various feature nodes based on the sampled feature map includes:
[0016] The DGMN module is used to sample feature nodes of the sampled feature map based on the uniform walk sampler and the random walk sampler, and the node index is returned.
[0017] Based on the self-attention mechanism and the node index, the dynamic association between each feature node is predicted.
[0018] Optionally, obtaining the target detection model includes:
[0019] Acquire multiple training video frames and the initial object detection model for each scenario;
[0020] In each training video frame, the region where the training object is located is marked, and the training category corresponding to the training object is marked, to obtain the training image corresponding to the initial object detection model. The training image is a training video frame containing the marked labels.
[0021] The initial object detection model is iteratively trained using each of the training images to obtain a trained initial object detection model, which is the latest object detection model.
[0022] Optionally, in each training video frame, the region where the training object is located is marked, and the training category corresponding to the training object is marked, to obtain the training image corresponding to the initial object detection model, including:
[0023] Based on the scene corresponding to each training video frame, determine the training category that needs to be focused on for the training video frame, and find the training object corresponding to the training category in the training video frame;
[0024] The training category of the training object and the region where the training object is located are marked on the training video frames containing the training object to obtain the training image corresponding to the initial object detection model.
[0025] Optionally, the step of iteratively training the initial object detection model using each of the training images to obtain the trained initial object detection model includes:
[0026] The training images are combined proportionally to obtain a training set, a test set, and a validation set.
[0027] The initial object detection model is iteratively trained using the training set, test set, and validation set to obtain the trained initial object detection model.
[0028] Optionally, the step of iteratively training the initial object detection model using the training set, test set, and validation set to obtain the trained initial object detection model includes:
[0029] The initial target detection model is iteratively trained using the training images in the training set, the number of iterations is recorded, and the initial target detection model after each iteration is obtained.
[0030] Generate an identifier for each of the initial object detection models after iterative training, wherein the identifier corresponds one-to-one with the initial object detection model after iterative training;
[0031] Using the validation set, the average accuracy of the initial target detection model after each iteration of training is calculated until the number of iterations is greater than a preset first threshold, or the average accuracy is higher than a preset second threshold. The identifier of the latest iteration of training initial target detection model is then used as the target identifier.
[0032] The process involves repeatedly returning to the initial target detection model and iteratively training it using the training images in the training set to obtain multiple target identifiers.
[0033] Using the test set, the initial target detection model with the highest average accuracy after iterative training is selected from the initial target detection models corresponding to each target identifier as the initial target detection model after training.
[0034] A target detection device, comprising:
[0035] The acquisition unit is used to acquire video frames and an object detection model. The object detection model includes a backbone module, a feature fusion Rep-PAN Neck module, and a prediction head module. The backbone module is an EfficientRep Backbone module that adds a Dynamic Graph Message Passing (DGMN) module after the stem module.
[0036] The extraction unit is used to predict the transformation matrix of dynamic filters and affinity relationships for each feature node of the video frame using the Backbone module, and to perform random walk sampling based on the transformation matrix to extract detailed features and key features.
[0037] The fusion unit is used to fuse the detailed features and key features using the Rep-PAN Neck module to obtain fused data;
[0038] The analysis unit is used to analyze the fused data using the Head module and generate the detection result corresponding to the video frame. The detection result is an empty set or a video frame marked with the region and category of the target object. The target object is the object that needs to be focused on in the scene corresponding to the video frame.
[0039] A target detection device, comprising a memory and a processor;
[0040] The memory is used to store programs;
[0041] The processor is used to execute the program to implement the various steps of the target detection method described above.
[0042] A readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the target detection method described above.
[0043] As can be seen from the above technical solution, the target detection method provided in this application can acquire video frames and a target detection model. The target detection model includes a backbone module, a feature fusion Rep-PAN Neck module, and a prediction head module. The backbone module is an EfficientRep Backbone module that adds a Dynamic Graph Message Passing (DGMN) module after the stem module. The backbone module predicts the transformation matrix of the dynamic filter and affinity relationship for each feature node in the video frame, and performs random walk sampling based on the transformation matrix to extract detailed and key features. Thus, this application can use the backbone module with added DGMN module to predict the transformation matrix corresponding to each feature node, thereby capturing remote dependencies, improving the capture capability of the backbone module, and achieving the extraction of more detailed features and features requiring special attention. The Rep-PAN Neck module... The Neck module fuses the detailed features and key features to obtain fused data. The Head module analyzes this fused data to generate detection results for the video frame. These detection results are either empty sets or video frames labeled with the region and category of the target object. The target object is the object of interest within the scene corresponding to the video frame. Thus, the target detection model of this application can utilize the extracted detailed and key features to better detect targets in video frames, resulting in better and more accurate detection results. Therefore, the target detection model proposed in this application possesses high capture capability, which can improve the accuracy and reliability of target detection. Attached Figure Description
[0044] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0045] Figure 1 This is a flowchart of a target detection method disclosed in an embodiment of this application;
[0046] Figure 2 A schematic diagram of the structure of a Backbone module provided in an embodiment of this application;
[0047] Figure 3 This is a structural block diagram of a target detection device disclosed in an embodiment of this application;
[0048] Figure 4This is a hardware structure block diagram of a target detection device disclosed in an embodiment of this application. Detailed Implementation
[0049] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0050] The target detection method provided in this application can be applied to a wide range of general-purpose or special-purpose computing device environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor devices, and distributed computing environments including any of the above devices.
[0051] Next, combine Figure 1 The target detection method of this application is described in detail, including the following steps:
[0052] Step S1: Obtain video frames and object detection model.
[0053] Specifically, the target detection model may include a backbone module, a feature fusion Rep-PAN Neck module, and a prediction head module. The backbone module is the EfficientRep Backbone module, which adds a Dynamic Graph Message Passing (DGMN) module after the stem module.
[0054] The DGMN module has two key dynamic features. First, it dynamically samples the neighborhood of feature nodes. Intuitively, the object detection network with the DGMN module can effectively collect remote context by selecting only a subset of the most relevant nodes in the video frame. Second, it dynamically predicts filtering weights and correlation matrices based on the sampled feature nodes. These weights and correlation matrices are used to propagate information between feature nodes through message passing.
[0055] The EfficientRep Backbone module is designed based on the Rep operator, which enables the EfficientRep Backbone module to efficiently utilize hardware computing power while also having strong representation capabilities.
[0056] Figure 2 This is a schematic diagram of the structure of a Backbone module provided in an embodiment of this application.
[0057] Figure 2In this context, stem represents the stem module, DGMN layer represents the DGMN module, and RepConv, stride=2 represents the RepConv layer with stride=2.
[0058] Reference Figure 2 It can be seen that the Backbone module of this application also includes multiple RepConv layers and RepBlock, as well as the efficient SimSPPF.
[0059] Step S2: Use the Backbone module to predict the transformation matrix of dynamic filters and affinity relationships for each feature node of the video frame, and perform random walk sampling based on the transformation matrix to extract detailed features and key features.
[0060] Specifically, video frames can be input into the Backbone module, and the stem module and DGMN module in the Backbone module can be used to predict the remote dependencies between the captured feature nodes of the video frames.
[0061] After obtaining the dependencies, the transformation matrices of dynamic filters and affinity relationships for each feature node can be predicted using the dependencies, and detailed features and key features can be extracted based on the transformation matrices of dynamic filters and affinity relationships between each feature node.
[0062] The formula for random walk sampling based on the transformation matrix is as follows:
[0063]
[0064] in, and For each feature node in the transformation matrix The matrix transformation parameters are learned separately, i is the number of the i-th feature node, j is the number of the j-th sampling node of the random walk, q is the q-th sampling rate, P is the dimension of the space, K is the number of uniform sampling nodes, and C is the number of feature channels.
[0065] Step S3: Using the Rep-PAN Neck module, the detailed features and key features are fused to obtain fused data.
[0066] Specifically, the Rep-PAN Neck module can be used to effectively fuse the extracted detailed features and key features, resulting in fused data.
[0067] Step S4: Using the Head module, analyze the fused data and generate the detection results corresponding to the video frame.
[0068] Specifically, the Head module can be used to parse the fused data, obtain the parsing results, and generate the detection results corresponding to the video frame based on the parsing results.
[0069] The detection result can be empty, indicating that there is no target object that needs to be focused on in the video frame. Similarly, the detection result can be a video frame marked with the area and category of the target object.
[0070] Marker boxes, such as rectangles, can be generated in video frames. The area corresponding to the marker box is the area where the target object is located, and the category corresponding to the target object is marked in the video frame.
[0071] The target object can be any object that needs to be monitored in the scene corresponding to the video frame. For example, if the scene is a garbage monitoring scene, then the objects that need to be monitored are items such as garbage bags, plastic bags, and plastic bottles.
[0072] As can be seen from the above technical solutions, the embodiments of this application provide a target detection method. It utilizes a Backbone module with an added DGMN module to predict the transformation matrix corresponding to each feature node, thereby capturing long-range dependencies and improving the Backbone module's capture capability. Furthermore, it leverages the EfficientRep Backbone module, which possesses efficient representation capabilities, to extract more detailed features and those requiring special attention. Subsequently, it uses the Rep-PAN Neck module, which balances accuracy and speed and has strong multi-scale feature fusion capabilities, for data fusion, ensuring both accuracy and high efficiency. Finally, the Head module generates the detection results. Therefore, this application can utilize the Backbone module with efficient dependency capture capabilities, the Rep-PAN Neck module with strong multi-scale feature fusion capabilities, and the Head module for target detection. In summary, the efficient dependency capture capability and strong multi-scale feature fusion capability improve the accuracy of the detection results of this application.
[0073] Furthermore, the object detection model of this application has been experimentally verified to improve the accuracy of object detection while maintaining the original training and inference efficiency, resulting in high economic benefits. Moreover, the Backbone module of this application can be used as a preprocessing flow for object detection or as a downstream task in any computer vision, demonstrating high versatility.
[0074] In some embodiments of this application, the process of obtaining the target detection model in step S1 is described in detail, and the steps are as follows:
[0075] S10. Obtain multiple training video frames and the initial object detection model for each scene.
[0076] Specifically, each scenario can be any scenario that requires the use of an object detection model, such as garbage detection scenarios and insect and rodent pest detection scenarios.
[0077] The training video frames can be video frames captured in various scenarios.
[0078] The initial object detection model may include an initial Backbone module, an initial Rep-PAN Neck module, and an initial Head module.
[0079] S11. In each training video frame, mark the region where the training object is located and mark the training category corresponding to the training object to obtain the training image corresponding to the initial object detection model. The training image is a training video frame containing the marked labels.
[0080] Specifically, in each training video frame, the region where the training object exists and the training category of the existing training object are marked.
[0081] After labeling is completed, multiple training video frames containing labeled tags can be obtained as training images.
[0082] S12. Using each of the training images, iteratively train the initial target detection model to obtain a trained initial target detection model, which is the latest target detection model.
[0083] Specifically, the initial object detection model can be iteratively trained using training images, and the parameters of the initial object detection model can be adjusted until the trained object detection model is obtained.
[0084] As can be seen from the above technical solutions, this application provides an optional method for obtaining an object detection model through training. By using the above method, the reliability and accuracy of the object detection model can be further improved, and an object detection model that meets the requirements can be obtained, thereby improving the reliability of this application.
[0085] In some embodiments of this application, the process of step S11, which involves marking the region where the training object is located in each training video frame and marking the training category corresponding to the training object, to obtain the training image corresponding to the initial object detection model, is described in detail below:
[0086] S110. Based on the scene corresponding to each training video frame, determine the training category that needs to be focused on for the training video frame, and find the training object corresponding to the training category in the training video frame.
[0087] Specifically, the training category corresponding to the scene can be determined based on the scene corresponding to the training video frame, and the training category corresponding to the scene is the training category that the training video frame needs to focus on.
[0088] You can find the training objects that correspond to the training categories that you need to focus on in the training video frames.
[0089] Different scenarios correspond to different training categories. For example, the training categories for the garbage detection scenario may include, but are not limited to, plastic bags, beverage bottles, beverage boxes, and cigarette butts. The training categories for the insect and rodent detection scenario may include, but are not limited to, rats, mosquitoes, and cockroaches.
[0090] S111. Mark the training category of the training object and the region where the training object is located on the training video frame containing the training object to obtain the training image corresponding to the initial target detection model.
[0091] Specifically, the training category of the training object and the region of the training object can be labeled on the training video frame. After labeling, the training image is obtained.
[0092] As can be seen from the above technical solutions, the embodiments of this application provide an optional method for obtaining training images. Through the above method, the efficiency of obtaining training images and the efficiency of obtaining object detection models can be further improved.
[0093] In some embodiments of this application, the process of step S12, which involves iteratively training the initial object detection model using each of the training images to obtain the trained initial object detection model, is described in detail below:
[0094] S120. Combine the training images proportionally to obtain a training set, a test set, and a validation set.
[0095] Specifically, the training images can be divided into a training set, a test set, and a validation set. The training set contains the most training images, followed by the validation set, and the test set contains the fewest training images.
[0096] The training set, test set, and validation set all contain training images corresponding to multiple scenes.
[0097] S121. Using the training set, test set, and validation set, the initial target detection model is iteratively trained to obtain the trained initial target detection model.
[0098] Specifically, the initial object detection model can be iteratively trained using training images in the training set, the average accuracy of the initial object detection model can be calculated using training images in the validation set, and the initial object detection model with the highest average accuracy can be selected as the initial object detection model after training using the test set.
[0099] As can be seen from the above technical solution, this embodiment provides an optional method for iteratively training an initial object detection model. Through the above method, the initial object detection model can be iteratively trained using the training set, the test set, and the validation set, and the initial object detection model with the highest average accuracy can be selected to better obtain the trained initial object detection model.
[0100] In some embodiments of this application, the process of iteratively training the initial object detection model using the training set, test set, and validation set to obtain the trained initial object detection model is described in detail below:
[0101] S1210. The initial target detection model is iteratively trained using the training images in the training set, the number of iterations is recorded, and the initial target detection model after each iteration is obtained.
[0102] Specifically, the training images in the training set are sequentially input into the initial object detection model to iteratively train the initial object detection model, resulting in the initial object detection model after each iteration.
[0103] While performing iterative training, the number of iterations of the initial object detection model is recorded.
[0104] S1211. Generate an identifier for each of the initial target detection models after iterative training, wherein the identifier corresponds one-to-one with the initial target detection model after iterative training.
[0105] Specifically, a unique identifier is determined for the initial object detection model after each iteration of training.
[0106] S1212. Using the verification set, calculate the average accuracy of the initial target detection model after each iteration of training until the number of iterations is greater than a preset first threshold, or the average accuracy is higher than a preset second threshold, and use the identifier of the latest iteration of training initial target detection model as the target identifier.
[0107] Specifically, the training images in the validation set are sequentially input into the initial object detection model after each iteration of training. The average accuracy of the initial object detection model after each iteration of training is calculated until the number of recorded iterations exceeds the first threshold or the average accuracy is greater than the second threshold. The identifier of the latest iteration of training initial object detection model is then used as the target identifier.
[0108] The first and second thresholds can be preset according to actual needs. When higher accuracy is required, the first and second thresholds can be larger.
[0109] S1213. Repeat step S1210 multiple times to obtain multiple target identifiers.
[0110] Specifically, steps S1210-S1212 can be repeated multiple times to obtain target labels for the initial target detection model after multiple iterations of training.
[0111] S1214. Using the test set, select the initial target detection model with the highest average accuracy from the initial target detection models trained iteratively for each record's identifier as the initial target detection model after training.
[0112] Specifically, the training images in the test set are sequentially input into the initial target detection model corresponding to each target identifier to obtain the average accuracy of the initial target detection model after each iteration of training. The initial target detection model with the highest average accuracy after each iteration of training is selected as the initial target detection model after training.
[0113] As can be seen from the above technical solution, this embodiment provides an optional method for obtaining the initial target detection model after training by utilizing a training set, a test set, and a validation set. The above method can further improve the average accuracy of the target detection model trained in this application, thereby improving the target detection accuracy of this application.
[0114] In some embodiments of this application, the process of predicting the dynamic filter and affinity transformation matrix of each feature node of the video frame using the Backbone module in step S2 is described as follows:
[0115] S20. The video frame is downsampled using the stem module to obtain a sampling feature map.
[0116] Specifically, the stem module is used to quickly downsample video frames, resulting in a sampled feature map that retains strong feature representation capabilities while reducing a large number of parameters in the video frame.
[0117] S21. Using the DGMN module, based on the sampled feature map, predict the dynamic association between each feature node, and based on the dynamic association between each feature node, jointly predict the transformation matrix of the dynamic filter and affinity relationship of each feature node.
[0118] Specifically, the stem module outputs the sampled feature map to the DGMN module.
[0119] The DGMN module can be used to sample feature nodes in a sampled feature map to obtain sampling results. Based on the sampling results, it can predict the dynamic association between each feature node and jointly predict the transformation matrix of dynamic filter and affinity relationship for each feature node based on the dynamic association between each feature node.
[0120] The dynamic associations between various feature nodes can be input into a 3×3 convolutional layer. This convolutional layer is used to jointly predict the dynamic filter and affinity transformation matrix of each feature node based on the dynamic associations between various feature nodes.
[0121] As can be seen from the above technical solution, this embodiment provides an optional method for predicting the transformation matrix using the Backbone module. Through this method, the stem module in the Backbone module can be further used to generate a sampled feature map, and the DGMN module can be used to process the sampled feature map to obtain the long-range dependencies between each feature node, and a transformation matrix can be constructed based on these long-range dependencies. The transformation matrix extracted through the above process can better extract the detailed and key features of the sampled feature map.
[0122] In some embodiments of this application, the process of predicting the dynamic association between each feature node using the DGMN module based on the sampled feature map in step S21 is described in detail, and the steps are as follows:
[0123] S210. The DGMN module is used to sample feature nodes of the sampled feature map based on the uniform walk sampler and the random walk sampler, and the node index is returned.
[0124] Specifically, the uniform walk sampler and random walk sampler in the DGMN module can be used to sample the feature nodes of the sampled feature map, and the index of the sampled feature node is returned after sampling.
[0125] The node index indicates the location of the feature node corresponding to that node index, and the node index can be coordinate data.
[0126] S211. Based on the self-attention mechanism and the node index, predict the dynamic association between each feature node.
[0127] Specifically, the self-attention mechanism of the DGMN module and the generated node index can be used to predict the dynamic associations between various feature nodes.
[0128] Dynamic associations can be dynamic remote dependencies.
[0129] As can be seen from the above technical solution, this embodiment provides an optional method for predicting the dynamic association between each feature node using the DGMN module. The above method can better utilize the self-attention mechanism of the DGMN module to obtain the dynamic association, further improving the practicality and reliability of this application.
[0130] The target detection device provided in the embodiments of this application is described below. The target detection device described below and the target detection method described above can be referred to in correspondence.
[0131] See Figure 3 , Figure 3 This is a schematic diagram of the structure of a target detection device disclosed in an embodiment of this application.
[0132] like Figure 3 As shown, the target detection device may include:
[0133] Acquisition unit 1 is used to acquire video frames and target detection model. The target detection model includes a backbone module, a feature fusion Rep-PAN Neck module, and a prediction head module. The backbone module is the EfficientRep Backbone module, which adds a dynamic graph message passing (DGMN) module after the stem module.
[0134] Extraction unit 2 is used to predict the transformation matrix of dynamic filters and affinity relationships for each feature node of the video frame using the Backbone module, and to perform random walk sampling based on the transformation matrix to extract detailed features and key features;
[0135] Fusion unit 3 is used to fuse the detailed features and key features using the Rep-PAN Neck module to obtain fused data;
[0136] Analysis unit 4 is used to analyze the fused data using the Head module and generate detection results corresponding to the video frame. The detection results are either an empty set or video frames marked with the region and category of the target object. The target object is the object that needs to be focused on in the scene corresponding to the video frame.
[0137] Optionally, the extraction unit may include:
[0138] The sampling feature map acquisition unit is used to downsample the video frame using the stem module to obtain a sampling feature map;
[0139] The dynamic association prediction unit is used to use the DGMN module to predict the dynamic association between each feature node based on the sampled feature map, and to jointly predict the transformation matrix of dynamic filter and affinity relationship for each feature node based on the dynamic association between each feature node.
[0140] Optionally, the dynamic association prediction unit may include:
[0141] The node index generation unit is used to use the DGMN module to sample feature nodes of the sampled feature map based on the uniform walk sampler and the random walk sampler, and return the node index.
[0142] The node index utilization unit is used to predict the dynamic association between various feature nodes based on the self-attention mechanism and the node index.
[0143] Optionally, the acquisition unit may include:
[0144] The training video frame acquisition unit is used to acquire multiple training video frames and the initial object detection model in various scenes.
[0145] A category labeling unit is used to label the region where the training object is located in each training video frame and label the training category corresponding to the training object to obtain the training image corresponding to the initial object detection model. The training image is a training video frame containing labeling.
[0146] The model training unit is used to iteratively train the initial object detection model using each of the training images to obtain a trained initial object detection model, which is the latest object detection model.
[0147] Optionally, the category tag unit may include:
[0148] The training object lookup unit is used to determine the training category that needs to be focused on for each training video frame based on the scene corresponding to each training video frame, and to find the training object corresponding to the training category in the training video frame.
[0149] A region labeling unit is used to label the training category of the training object and the region where the training object is located on training video frames containing the training object, so as to obtain the training image corresponding to the initial object detection model.
[0150] Optionally, the model training unit may include:
[0151] The training image combination unit is used to combine the various training images proportionally to obtain a training set, a test set, and a validation set.
[0152] The set utilization unit is used to iteratively train the initial target detection model using the training set, the test set, and the validation set to obtain the trained initial target detection model.
[0153] Optionally, the collection utilization unit may include:
[0154] The first set utilization unit is used to iteratively train the initial target detection model using the training images in the training set, record the number of iterations, and obtain the initial target detection model after each iteration.
[0155] The second set utilization unit is used to generate an identifier for each of the initial target detection models after iterative training, and the identifier corresponds one-to-one with the initial target detection models after iterative training;
[0156] The third set utilization unit is used to use the verification set to calculate the average accuracy of the initial target detection model after each iteration of training until the number of iterations is greater than a preset first threshold, or the average accuracy is higher than a preset second threshold, and the identifier of the latest obtained initial target detection model after iteration training is used as the target identifier.
[0157] The fourth set utilization unit is used to repeatedly return to the step of iteratively training the initial target detection model using the training images in the training set to obtain multiple target labels;
[0158] The fifth set utilization unit is used to select the initial target detection model with the highest average accuracy from the iteratively trained initial target detection models corresponding to each target identifier using the test set as the initial target detection model after training.
[0159] The target detection device provided in this application embodiment can be applied to target detection equipment, such as PC terminals, cloud platforms, servers, and server clusters. Optionally, Figure 4 The hardware structure block diagram of the target detection device is shown. (Refer to...) Figure 4 The hardware structure of the target detection device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4;
[0160] In this embodiment of the application, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4;
[0161] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.
[0162] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;
[0163] The memory stores a program, which the processor can call. The program is used for:
[0164] Acquire video frames and an object detection model. The object detection model includes a backbone module, a feature fusion Rep-PAN Neck module, and a prediction head module. The backbone module is an EfficientRep Backbone module that adds a Dynamic Graph Message Passing (DGMN) module after the stem module.
[0165] The Backbone module is used to predict the transformation matrix of dynamic filters and affinity relationships for each feature node in the video frame, and random walk sampling is performed based on the transformation matrix to extract detailed features and key features;
[0166] The Rep-PAN Neck module is used to fuse the detailed features and key features to obtain fused data;
[0167] Using the Head module, the fused data is analyzed to generate detection results corresponding to the video frame. The detection results are either an empty set or video frames marked with the region and category of the target object. The target object is the object that needs to be focused on in the scene corresponding to the video frame.
[0168] Optionally, the refined and extended functions of the program can be referred to the above description.
[0169] This application embodiment also provides a readable storage medium that can store a program suitable for execution by a processor, the program being used for:
[0170] Acquire video frames and an object detection model. The object detection model includes a backbone module, a feature fusion Rep-PAN Neck module, and a prediction head module. The backbone module is an EfficientRep Backbone module that adds a Dynamic Graph Message Passing (DGMN) module after the stem module.
[0171] The Backbone module is used to predict the transformation matrix of dynamic filters and affinity relationships for each feature node in the video frame, and random walk sampling is performed based on the transformation matrix to extract detailed features and key features;
[0172] The Rep-PAN Neck module is used to fuse the detailed features and key features to obtain fused data;
[0173] Using the Head module, the fused data is analyzed to generate detection results corresponding to the video frame. The detection results are either an empty set or video frames marked with the region and category of the target object. The target object is the object that needs to be focused on in the scene corresponding to the video frame.
[0174] Optionally, the refined and extended functions of the program can be referred to the above description.
[0175] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0176] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0177] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. The various embodiments of this application can be combined with each other. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A target detection method, characterized in that, include: Acquire video frames and an object detection model. The object detection model includes a backbone module, a feature fusion Rep-PAN Neck module, and a prediction head module. The backbone module is an EfficientRep Backbone module that adds a Dynamic Graph Message Passing (DGMN) module after the stem module. The Backbone module is used to predict the transformation matrix of dynamic filters and affinity relationships for each feature node in the video frame, and random walk sampling is performed based on the transformation matrix to extract detailed features and key features; The Rep-PAN Neck module is used to fuse the detailed features and key features to obtain fused data; Using the Head module, the fused data is analyzed to generate the detection result corresponding to the video frame. The detection result is an empty set or a video frame marked with the region and category of the target object. The target object is the object that needs to be focused on in the scene corresponding to the video frame. The transformation matrix for predicting the dynamic filters and affinity relationships of each feature node in the video frame using the Backbone module includes: The video frames are downsampled using the stem module to obtain a sampled feature map; Using the DGMN module, based on the sampled feature map, the dynamic association between each feature node is predicted, and based on the dynamic association between each feature node, the transformation matrix of the dynamic filter and affinity relationship of each feature node is jointly predicted.
2. The target detection method according to claim 1, characterized in that, The DGMN module includes a uniform walk sampler and a random walk sampler; The step of using the DGMN module to predict the dynamic associations between the various feature nodes based on the sampled feature map includes: The DGMN module is used to sample feature nodes of the sampled feature map based on the uniform walk sampler and the random walk sampler, and the node index is returned. Based on the self-attention mechanism and the node index, the dynamic association between each feature node is predicted.
3. The target detection method according to claim 1, characterized in that, The acquisition of the target detection model includes: Acquire multiple training video frames and the initial object detection model for each scenario; In each training video frame, the region where the training object is located is marked, and the training category corresponding to the training object is marked, to obtain the training image corresponding to the initial object detection model. The training image is a training video frame containing the marked labels. The initial object detection model is iteratively trained using each of the training images to obtain a trained initial object detection model, which is the latest object detection model.
4. The target detection method according to claim 3, characterized in that, In each training video frame, the region where the training object is located is marked, and the training category corresponding to the training object is marked, to obtain the training image corresponding to the initial object detection model, including: Based on the scene corresponding to each training video frame, determine the training category that needs to be focused on for the training video frame, and find the training object corresponding to the training category in the training video frame; The training category of the training object and the region where the training object is located are marked on the training video frames containing the training object to obtain the training image corresponding to the initial object detection model.
5. The target detection method according to claim 3, characterized in that, The step of iteratively training the initial object detection model using each of the training images to obtain the trained initial object detection model includes: The training images are combined proportionally to obtain a training set, a test set, and a validation set. The initial object detection model is iteratively trained using the training set, test set, and validation set to obtain the trained initial object detection model.
6. The target detection method according to claim 5, characterized in that, The step of iteratively training the initial object detection model using the training set, test set, and validation set to obtain the trained initial object detection model includes: The initial target detection model is iteratively trained using the training images in the training set, the number of iterations is recorded, and the initial target detection model after each iteration is obtained. Generate an identifier for each of the initial object detection models after iterative training, wherein the identifier corresponds one-to-one with the initial object detection model after iterative training; Using the validation set, the average accuracy of the initial target detection model after each iteration of training is calculated until the number of iterations is greater than a preset first threshold, or the average accuracy is higher than a preset second threshold. The identifier of the latest iteration of training initial target detection model is then used as the target identifier. The process involves repeatedly returning to the initial target detection model and iteratively training it using the training images in the training set to obtain multiple target identifiers. Using the test set, the initial target detection model with the highest average accuracy after iterative training is selected from the initial target detection models corresponding to each target identifier as the initial target detection model after training.
7. A target detection device, characterized in that, include: The acquisition unit is used to acquire video frames and an object detection model. The object detection model includes a backbone module, a feature fusion Rep-PAN Neck module, and a prediction head module. The backbone module is an EfficientRep Backbone module that adds a Dynamic Graph Message Passing (DGMN) module after the stem module. The extraction unit is used to predict the transformation matrix of dynamic filters and affinity relationships for each feature node of the video frame using the Backbone module, and to perform random walk sampling based on the transformation matrix to extract detailed features and key features. The fusion unit is used to fuse the detailed features and key features using the Rep-PAN Neck module to obtain fused data; The analysis unit is used to analyze the fused data using the Head module and generate the detection result corresponding to the video frame. The detection result is an empty set or a video frame marked with the region and category of the target object. The target object is the object that needs to be focused on in the scene corresponding to the video frame. The transformation matrix for predicting the dynamic filters and affinity relationships of each feature node in the video frame using the Backbone module includes: The video frames are downsampled using the stem module to obtain a sampled feature map; Using the DGMN module, based on the sampled feature map, the dynamic association between each feature node is predicted, and based on the dynamic association between each feature node, the transformation matrix of the dynamic filter and affinity relationship of each feature node is jointly predicted.
8. A target detection device, characterized in that, Including memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement the various steps of the target detection method as described in any one of claims 1-6.
9. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the target detection method as described in any one of claims 1-6.