Behavior Detection Method and System Based on Object Detection Model
By improving the YOLOv10 model and combining it with one-to-one and one-to-many detection heads, the accuracy and real-time performance issues of fall detection in complex backgrounds have been resolved, achieving efficient and accurate fall behavior detection.
Patent Information
- Application Number
- CN202411288053.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-13
- Publication Date
- 2025-11-14
- Estimated Expiration
- 2044-09-13
AI Technical Summary
Existing fall detection methods suffer from detection delays in complex background environments and cannot accurately identify fall behavior. In particular, when the visual detection environment is poor, errors in skeleton coordinate extraction lead to inaccurate detection results. Furthermore, the detection model is not updated in a timely manner, and the detection accuracy needs to be improved.
An improved YOLOv10 model is adopted, adding one-to-one and one-to-many detection heads. Background and target candidate boxes are processed through a behavior recognition model, and prediction is performed using the one-to-one detection head. Combined with a feature extraction network and a prediction network, the pose of the tracking target is determined, avoiding NMS post-processing and improving detection speed and accuracy.
It achieves efficient and accurate detection of fall behavior in complex backgrounds, improves the applicability and real-time performance of the detection model, reduces computational load, and enhances robustness.
Smart Images
Figure CN119229531B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of behavior recognition technology, and more specifically to a behavior detection method and system based on a target detection model. Background Technology
[0002] In modern society, most young people need to work during the day, which may leave only elderly people over 65 years old at home. There are generally two reasons why the elderly are prone to falls. The first is due to their own illness, such as sudden coma caused by heart disease, sudden coma caused by cerebral infarction, or ALS. The second is physical injury caused by tripping over objects at home due to not paying attention while walking.
[0003] However, existing fall detection methods suffer from delays in detecting falls due to complex background environments, sometimes even failing to recognize them. Furthermore, in poor visual detection environments, such as changes in lighting or the angle at which an elderly person stands, skeleton coordinates are easily extracted incorrectly or not at all, severely impacting the effectiveness of behavior detection and leading to inaccurate results. In addition, skeleton coordinates only reflect static information and cannot comprehensively reflect fall behavior; moreover, the detection model is not updated in a timely and effective manner, remaining in its initial modeling state, thus requiring further improvement in detection accuracy.
[0004] Therefore, how to provide a behavior detection method and system based on a target detection model that is both efficient and accurate is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0005] In view of this, the present invention provides a behavior detection method and system based on a target detection model to solve the problems of the prior art.
[0006] To achieve the above objectives, the present invention adopts the following technical solution:
[0007] This invention discloses a behavior detection method based on an object detection model, comprising:
[0008] Acquire the captured target video, obtain the target video to be analyzed from the target video, divide the target video to be analyzed into N segments, and randomly extract one frame from each segment as the image to be identified, where N is an integer greater than 1;
[0009] An improved YOLOv10 model is constructed, which adds a one-to-one head to the traditional YOLOv10 model. During training, the two one-to-one heads and one one-to-many head are optimized together with the model, allowing the backbone and neck to enjoy the rich supervision provided by the one-to-many assignment. During inference, the one-to-many head is discarded, and the two one-to-one heads are used for prediction. Two candidate regions at different scales are obtained in each frame of the image: a background candidate box and a target candidate box. The target candidate box includes a first tracking target, and the background candidate box includes the image containing the target candidate box containing the location of the first tracking target to be predicted.
[0010] The background candidate box and the target candidate box are input into the first feature extraction network in the behavior recognition model to obtain the first feature map of the background candidate box and the second feature map of the target candidate box.
[0011] The first feature map and the second feature map are input into the first prediction network in the behavior recognition model to obtain the first prediction map corresponding to the first prediction network. The first prediction network includes M key point prediction units in sequence. The first prediction map is used to indicate the predicted position of the first tracking target on the background candidate box, where M is an integer greater than 1. According to the first prediction map, the position of the first tracking target is marked on the background candidate box.
[0012] The pose of the first tracked target is determined based on the changes in the position of the first tracked target in consecutive frames.
[0013] Preferably, in the above-mentioned behavior detection method based on the object detection model, the improved YOLOv10 model includes: adding a one-to-one detection head to the head network based on the traditional YOLOv10 model; the head network includes a first one-to-one detection head, a second one-to-one detection head, and a one-to-many detection head;
[0014] The first and second one-to-one detection heads retain the same structure and optimization objective as the original one-to-many branch. During training, the three detection heads are optimized together with the model. During inference, the one-to-many detection head is discarded, and the first and second one-to-one detection heads are used to make predictions to obtain background candidate boxes and target candidate boxes.
[0015] Preferably, in the above-described behavior detection method based on an object detection model, inputting the first feature map and the second feature map into the first prediction network of the behavior recognition model to obtain the first prediction map corresponding to the first prediction network includes:
[0016] The input data is processed by each keypoint prediction unit in the first prediction network to obtain the third feature map of the target candidate box and the fusion map corresponding to the background candidate box. The input data of the (l+1)th keypoint prediction unit is the fusion map and the third feature map obtained by the lth keypoint prediction unit, l∈{1,2,…M}. The fusion map is used to indicate the predicted keypoints of the first tracking target on the background candidate box.
[0017] The feature map obtained from the Mth keypoint prediction unit is processed by reducing the number of channels to obtain the first prediction map.
[0018] Preferably, in the above-described behavior detection method based on an object detection model, the input data is processed using each keypoint prediction unit in the first prediction network to obtain the third feature map of the target candidate box and the fusion map corresponding to the background candidate box, including:
[0019] The first convolution operation unit of the first key point prediction unit of the first prediction network is used to perform convolution operation on the first target map to obtain the third feature map of the target candidate box;
[0020] The second convolution operation unit of the first key point prediction unit is used to perform convolution operation on the second target image to obtain the fourth feature map of the background candidate box;
[0021] Using the cross-correlation operation unit in the first key point prediction unit, cross-correlation operation is performed on the third feature map and the fourth feature map to obtain a fused map;
[0022] Wherein, the first keypoint prediction unit is the s-th keypoint prediction unit of the first prediction network, s∈{1,2,…M}. When s=1, the first target map is the first feature map and the second target map is the second feature map; when s≠1, the first target map is the fusion map input to the first keypoint prediction unit and the second target map is the third feature map input to the first keypoint prediction unit.
[0023] Preferably, in the above-described behavior detection method based on the target detection model, before using the cross-correlation operation unit in the first keypoint prediction unit to perform cross-correlation operation on the third feature map and the fourth feature map to obtain the fused map, the method further includes:
[0024] The third feature map is input into the third convolution operation unit in the first key point prediction unit using the first convolution operation unit;
[0025] The data size of the third feature map is adjusted using the third convolution operation unit, and the ratio between the adjusted data size of the third feature map and the data size of the fourth feature map is within a predetermined range.
[0026] The third feature map, after data size adjustment, is input into the cross-correlation calculation unit.
[0027] Preferably, in the above-mentioned behavior detection method based on the object detection model, a first one-to-one detection head and a second one-to-one detection head are added to the head network, and a CIB module is introduced. The kernel size of the second 3×3 depth convolution of the CIB module of the first one-to-one detection head is increased to 7×7 to obtain background candidate boxes.
[0028] On the other hand, the present invention discloses a behavior detection system based on an object detection model, wherein the method includes:
[0029] The acquisition module acquires the captured target video, obtains the target video to be analyzed from the target video, divides the target video to be analyzed into N segments, and randomly extracts one frame from each segment as the image to be identified, where N is an integer greater than 1;
[0030] The module constructs an improved YOLOv10 model. This improved YOLOv10 model adds a one-to-one head to the traditional YOLOv10 model. During training, the two one-to-one heads and one one-to-many head are optimized together with the model, allowing the backbone and neck to enjoy the rich supervision provided by the one-to-many assignment. During inference, the one-to-many head is discarded, and the two one-to-one heads are used for prediction. Two candidate regions at different scales are obtained for each frame: a background candidate box and a target candidate box. The target candidate box includes a first tracking target, and the background candidate box includes the image containing the target candidate box, indicating the location of the first tracking target to be predicted.
[0031] The input module inputs the background candidate box and the target candidate box into the first feature extraction network in the behavior recognition model to obtain the first feature map of the background candidate box and the second feature map of the target candidate box;
[0032] The processing module inputs the first feature map and the second feature map into the first prediction network in the behavior recognition model to obtain a first prediction map corresponding to the first prediction network. The first prediction network includes M key point prediction units in sequence. The first prediction map is used to indicate the predicted position of the first tracking target on the background candidate box, where M is an integer greater than 1. According to the first prediction map, the position of the first tracking target is marked on the background candidate box.
[0033] The output module determines the pose of the first tracking target based on the changes in the position of the first tracking target in consecutive frames.
[0034] Preferably, in the above-mentioned behavior detection system based on the target detection model, the head network in the construction module includes a first one-to-one detection head, a second one-to-one detection head, and a one-to-many detection head;
[0035] The first and second one-to-one detection heads retain the same structure and optimization objective as the original one-to-many branch. During training, the three detection heads are optimized together with the model. During inference, the one-to-many detection head is discarded, and the first and second one-to-one detection heads are used to make predictions to obtain background candidate boxes and target candidate boxes.
[0036] Preferably, in the above-mentioned behavior detection system based on the object detection model, a first one-to-one detection head and a second one-to-one detection head are added to the head network, and a CIB module is introduced. The kernel size of the second 3×3 depth convolution of the CIB module of the first one-to-one detection head is increased to 7×7 to obtain background candidate boxes.
[0037] As can be seen from the above technical solution, compared with the prior art, this invention discloses a behavior detection method and system based on an object detection model. It creatively integrates two one-to-one detection heads and one one-to-many detection heads in the head part of an improved YOLOv10 model. This utilizes the rich supervision signals from the one-to-many detection heads to train and optimize model parameters, ensuring model accuracy, while employing one-to-one detection heads for inference avoids NMS post-processing, thus improving inference speed. Specifically, two one-to-one detection heads are used to obtain background candidate boxes and target candidate boxes respectively. These boxes are then processed by a behavior recognition model to obtain a first prediction map corresponding to a first prediction network. Based on the first prediction map, the position of the first tracking target is marked on the background candidate boxes. The pose of the first tracking target is determined based on the changes in its position across consecutive frames. This achieves target tracking on the background candidate boxes. For different scenarios, it eliminates the need to determine different anchor box selection schemes, resulting in better applicability of the target tracking method. Attached Figure Description
[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0039] Figure 1An improved YOLOv10 model structure diagram provided for this invention;
[0040] Figure 2 The structure diagram of the behavior recognition model provided by the present invention. Detailed Implementation
[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0042] Embodiments of the present invention disclose a behavior detection method based on an object detection model, such as... Figure 1 As shown, it includes:
[0043] Acquire the captured target video, obtain the target video to be analyzed from the target video, divide the target video to be analyzed into N segments, and randomly extract one frame from each segment as the image to be identified, where N is an integer greater than 1;
[0044] An improved YOLOv10 model is constructed, which adds a one-to-one head to the traditional YOLOv10 model. During training, the two one-to-one heads and one one-to-many head are optimized together with the model, allowing the backbone and neck to enjoy the rich supervision provided by the one-to-many assignment. During inference, the one-to-many head is discarded, and the two one-to-one heads are used for prediction. Two candidate regions at different scales are obtained in each frame of the image: a background candidate box and a target candidate box. The target candidate box includes a first tracking target, and the background candidate box includes the image containing the target candidate box containing the location of the first tracking target to be predicted.
[0045] The background candidate box and the target candidate box are input into the first feature extraction network in the behavior recognition model to obtain the first feature map of the background candidate box and the second feature map of the target candidate box.
[0046] The first feature map and the second feature map are input into the first prediction network in the behavior recognition model to obtain the first prediction map corresponding to the first prediction network. The first prediction network includes M key point prediction units in sequence. The first prediction map is used to indicate the predicted position of the first tracking target on the background candidate box, where M is an integer greater than 1. According to the first prediction map, the position of the first tracking target is marked on the background candidate box.
[0047] The pose of the first tracked target is determined based on the changes in the position of the first tracked target in consecutive frames.
[0048] Furthermore, the key points of the first tracking target's posture are detected to obtain candidate bounding box information and 17 key points, each of which consists of three parameters: x, y, and c. If the confidence level of a key point is lower than a preset confidence threshold, the key point is considered occluded. The coordinates of the candidate bounding box, the coordinates of the 17 key points, and the occlusion status of the key points are stored in the trajectory's database. If other trajectories are detected, this trajectory is matched with other trajectories. If the match is successful, the two trajectories are merged. If the match fails, this trajectory is saved in the historical trajectories. According to actual needs, the 17 key points of the first tracking target detected in this trajectory, along with their corresponding confidence levels, are used to recognize key actions.
[0049] Based on human posture (skeleton key points) information, this method solves the problem of trajectory mismatch caused by pedestrian occlusion during trajectory tracking, while reducing the amount of computation, improving the running speed, and providing a more robust, real-time, and scalable solution.
[0050] To further optimize the above technical solution, the improved YOLOv10 model includes: adding a one-to-one detection head to the head network based on the traditional YOLOv10 model; the head network includes a first one-to-one detection head, a second one-to-one detection head, and a one-to-many detection head;
[0051] The first and second one-to-one detection heads retain the same structure and optimization objective as the original one-to-many branch. During training, the three detection heads are optimized together with the model. During inference, the one-to-many detection head is discarded, and the first and second one-to-one detection heads are used to make predictions to obtain background candidate boxes and target candidate boxes.
[0052] To further optimize the above technical solution, the first feature map and the second feature map are input into the first prediction network in the behavior recognition model to obtain the first prediction map corresponding to the first prediction network, including:
[0053] The input data is processed by each keypoint prediction unit in the first prediction network to obtain the third feature map of the target candidate box and the fusion map corresponding to the background candidate box. The input data of the (l+1)th keypoint prediction unit is the fusion map and the third feature map obtained by the lth keypoint prediction unit, l∈{1,2,…M}. The fusion map is used to indicate the predicted keypoints of the first tracking target on the background candidate box.
[0054] The feature map obtained from the Mth keypoint prediction unit is processed by reducing the number of channels to obtain the first prediction map.
[0055] To further optimize the above technical solution, each keypoint prediction unit in the first prediction network is used to process the input data to obtain the third feature map of the target candidate box and the fusion map corresponding to the background candidate box, including:
[0056] The first convolution operation unit of the first key point prediction unit of the first prediction network is used to perform convolution operation on the first target map to obtain the third feature map of the target candidate box;
[0057] The second convolution operation unit of the first key point prediction unit is used to perform convolution operation on the second target image to obtain the fourth feature map of the background candidate box;
[0058] Using the cross-correlation operation unit in the first key point prediction unit, cross-correlation operation is performed on the third feature map and the fourth feature map to obtain a fused map;
[0059] Wherein, the first keypoint prediction unit is the s-th keypoint prediction unit of the first prediction network, s∈{1,2,…M}. When s=1, the first target map is the first feature map and the second target map is the second feature map; when s≠1, the first target map is the fusion map input to the first keypoint prediction unit and the second target map is the third feature map input to the first keypoint prediction unit.
[0060] To further optimize the above technical solution, before performing cross-correlation operations on the third feature map and the fourth feature map using the cross-correlation unit in the first key point prediction unit to obtain the fused map, the method further includes:
[0061] The third feature map is input into the third convolution operation unit in the first key point prediction unit using the first convolution operation unit;
[0062] The data size of the third feature map is adjusted using the third convolution operation unit, and the ratio between the adjusted data size of the third feature map and the data size of the fourth feature map is within a predetermined range.
[0063] The third feature map, after data size adjustment, is input into the cross-correlation calculation unit.
[0064] Specifically, the number of first prediction networks in the behavior recognition model is N, where N is a positive integer.
[0065] The following example uses N=3, and combines... Figure 2 This disclosure describes an embodiment.
[0066] like Figure 2 As shown, the target candidate box is input into the first feature extraction sub-network to obtain three first feature maps of the target candidate box, and the background candidate box is input into the second feature extraction sub-network to obtain three second feature maps of the background candidate box.
[0067] The three first feature maps are input into the three first prediction networks, and the three second feature maps are input into the three first prediction networks, resulting in a first prediction map for each first prediction network. Since there are three first prediction networks in total, a total of three first prediction maps are obtained.
[0068] Then, the three first prediction heatmaps are added together using the addition operation unit to obtain the first target prediction map.
[0069] Then, based on the first target prediction map, the position of the first tracked target is marked on the background candidate box.
[0070] The input data is processed by each keypoint prediction unit in the first prediction network to obtain the third feature map of the target candidate box and the fusion map corresponding to the background candidate box. The input data of the (l+1)th keypoint prediction unit is the fusion map and the third feature map obtained by the lth keypoint prediction unit, l∈{1,2,…M}. The fusion map is used to indicate the predicted keypoints of the first tracking target on the background candidate box.
[0071] The first prediction map is obtained by reducing the number of channels in the fusion map obtained from the Mth keypoint prediction unit.
[0072] Specifically, the first prediction map is obtained by using a channel reduction operation unit to reduce the number of channels in the response map.
[0073] For example, if the fused image has 512 channels, a channel reduction operation unit can be used to reduce the number of channels in the fused image, resulting in a first prediction image with 5 channels. By performing this channel reduction operation, a first prediction image with fewer channels is obtained. Therefore, the position of the first tracking target on the background candidate box can be determined based on the first prediction image.
[0074] Optionally, a single downchannel operation unit includes P convolution operation units, where P is a positive integer.
[0075] The first prediction image is obtained by using a channel reduction operation unit to reduce the number of channels in the fused image.
[0076] This can include: performing P convolution operations on the fused image using P convolutional units to reduce the number of channels, thereby obtaining a prediction image. The number of channels in the prediction image can be equal to the number of channels in the label image used when training the action recognition model.
[0077] To further optimize the above technical solution, a first one-to-one detection head and a second one-to-one detection head are added to the head network, and a CIB module is introduced. The kernel size of the second 3×3 depth convolution of the CIB module in the first one-to-one detection head is increased to 7×7 to obtain background candidate boxes.
[0078] Another embodiment of the present invention discloses a behavior detection system based on an object detection model, which applies the above method and includes:
[0079] The acquisition module acquires the captured target video, obtains the target video to be analyzed from the target video, divides the target video to be analyzed into N segments, and randomly extracts one frame from each segment as the image to be identified, where N is an integer greater than 1;
[0080] The module constructs an improved YOLOv10 model. This improved YOLOv10 model adds a one-to-one head to the traditional YOLOv10 model. During training, the two one-to-one heads and one one-to-many head are optimized together with the model, allowing the backbone and neck to enjoy the rich supervision provided by the one-to-many assignment. During inference, the one-to-many head is discarded, and the two one-to-one heads are used for prediction. Two candidate regions at different scales are obtained for each frame: a background candidate box and a target candidate box. The target candidate box includes a first tracking target, and the background candidate box includes the image containing the target candidate box, indicating the location of the first tracking target to be predicted.
[0081] The input module inputs the background candidate box and the target candidate box into the first feature extraction network in the behavior recognition model to obtain the first feature map of the background candidate box and the second feature map of the target candidate box;
[0082] The processing module inputs the first feature map and the second feature map into the first prediction network in the behavior recognition model to obtain a first prediction map corresponding to the first prediction network. The first prediction network includes M key point prediction units in sequence. The first prediction map is used to indicate the predicted position of the first tracking target on the background candidate box, where M is an integer greater than 1. According to the first prediction map, the position of the first tracking target is marked on the background candidate box.
[0083] The output module determines the pose of the first tracking target based on the changes in the position of the first tracking target in consecutive frames.
[0084] To further optimize the above technical solution, the head network in the construction module includes a first one-to-one detection head, a second one-to-one detection head, and a one-to-many detection head.
[0085] The first and second one-to-one detection heads retain the same structure and optimization objective as the original one-to-many branch. During training, the three detection heads are optimized together with the model. During inference, the one-to-many detection head is discarded, and the first and second one-to-one detection heads are used to make predictions to obtain background candidate boxes and target candidate boxes.
[0086] To further optimize the above technical solution, a first one-to-one detection head and a second one-to-one detection head are added to the head network, and a CIB module is introduced. The kernel size of the second 3×3 depth convolution of the CIB module in the first one-to-one detection head is increased to 7×7 to obtain background candidate boxes.
[0087] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.
[0088] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A behavior detection method based on an object detection model, characterized in that, include: Acquire the captured target video, obtain the target video to be analyzed from the target video, divide the target video to be analyzed into N segments, and randomly extract one frame from each segment as the image to be identified, where N is an integer greater than 1; An improved YOLOv10 model is constructed, which adds a one-to-one head to the traditional YOLOv10 model. During training, the two one-to-one heads and one one-to-many head are optimized together with the model, allowing the backbone and neck to enjoy the rich supervision provided by the one-to-many assignment. During inference, the one-to-many head is discarded, and the two one-to-one heads are used for prediction. Two candidate regions at different scales are obtained in each frame of the image: a background candidate box and a target candidate box. The target candidate box includes a first tracking target, and the background candidate box includes the image containing the target candidate box containing the location of the first tracking target to be predicted. The background candidate box and the target candidate box are input into the first feature extraction network in the behavior recognition model to obtain the first feature map of the background candidate box and the second feature map of the target candidate box. The first feature map and the second feature map are input into the first prediction network in the behavior recognition model to obtain the first prediction map corresponding to the first prediction network. The first prediction network includes M key point prediction units in sequence. The first prediction map is used to indicate the predicted position of the first tracking target on the background candidate box, where M is an integer greater than 1. According to the first prediction map, the position of the first tracking target is marked on the background candidate box. The pose of the first tracked target is determined based on the changes in the position of the first tracked target in consecutive frames; The system detects key points of the first tracked target's posture, acquiring candidate bounding boxes and 17 key points, each composed of three parameters: x, y, and c. If the confidence level of a key point is lower than a preset confidence threshold, it is considered occluded. The coordinates of the candidate bounding boxes, the coordinates of the 17 key points, and the occlusion status of the key points are stored in the trajectory's database. If other trajectories are detected, this trajectory is matched with them. If the match is successful, the two trajectories are merged. If the match fails, this trajectory is saved in the historical trajectories. Based on actual needs, the 17 key points of the first tracked target and their corresponding confidence levels saved in this trajectory are used to recognize key actions.
2. The behavior detection method based on the target detection model according to claim 1, characterized in that, The improved YOLOv10 model includes: adding a one-to-one detection head to the head network based on the traditional YOLOv10 model; the head network includes a first one-to-one detection head, a second one-to-one detection head, and a one-to-many detection head; The first and second one-to-one detection heads retain the same structure and optimization objective as the original one-to-many branch. During training, the three detection heads are optimized together with the model. During inference, the one-to-many detection head is discarded, and the first and second one-to-one detection heads are used to make predictions to obtain background candidate boxes and target candidate boxes.
3. The behavior detection method based on the target detection model according to claim 1, characterized in that, The first feature map and the second feature map are input into the first prediction network in the behavior recognition model to obtain the first prediction map corresponding to the first prediction network, including: The input data is processed by each keypoint prediction unit in the first prediction network to obtain the third feature map of the target candidate box and the fusion map corresponding to the background candidate box. The input data of the (l+1)th keypoint prediction unit is the fusion map and the third feature map obtained by the lth keypoint prediction unit, l∈{1,2,…M}. The fusion map is used to indicate the predicted keypoints of the first tracking target on the background candidate box. The feature map obtained from the Mth keypoint prediction unit is processed by reducing the number of channels to obtain the first prediction map.
4. The behavior detection method based on the target detection model according to claim 1, characterized in that, The input data is processed using each keypoint prediction unit in the first prediction network to obtain the third feature map of the target candidate box and the fused map corresponding to the background candidate box, including: The first convolution operation unit of the first key point prediction unit of the first prediction network is used to perform convolution operation on the first target map to obtain the third feature map of the target candidate box; The second convolution operation unit of the first key point prediction unit is used to perform convolution operation on the second target image to obtain the fourth feature map of the background candidate box; Using the cross-correlation operation unit in the first key point prediction unit, cross-correlation operation is performed on the third feature map and the fourth feature map to obtain a fused map; Wherein, the first keypoint prediction unit is the s-th keypoint prediction unit of the first prediction network, s∈{1,2,…M}. When s=1, the first target map is the first feature map and the second target map is the second feature map; when s≠1, the first target map is the fusion map input to the first keypoint prediction unit and the second target map is the third feature map input to the first keypoint prediction unit.
5. The behavior detection method based on the target detection model according to claim 4, characterized in that, Before performing cross-correlation operations on the third feature map and the fourth feature map using the cross-correlation unit in the first keypoint prediction unit to obtain the fused map, the method further includes: The third feature map is input into the third convolution operation unit in the first key point prediction unit using the first convolution operation unit; The data size of the third feature map is adjusted using the third convolution operation unit, and the ratio between the adjusted data size of the third feature map and the data size of the fourth feature map is within a predetermined range. The third feature map, after data size adjustment, is input into the cross-correlation calculation unit.
6. The behavior detection method based on the target detection model according to claim 1, characterized in that, A first one-to-one detection head and a second one-to-one detection head are added to the head network, and a CIB module is introduced. The kernel size of the second 3×3 depth convolution of the CIB module in the first one-to-one detection head is increased to 7×7 to obtain background candidate boxes.
7. A behavior detection system based on an object detection model, employing the behavior detection method based on an object detection model according to any one of claims 1-6, characterized in that, include: The acquisition module acquires the captured target video, obtains the target video to be analyzed from the target video, divides the target video to be analyzed into N segments, and randomly extracts one frame from each segment as the image to be identified, where N is an integer greater than 1; The module constructs an improved YOLOv10 model. This improved YOLOv10 model adds a one-to-one head to the traditional YOLOv10 model. During training, the two one-to-one heads and one one-to-many head are optimized together with the model, allowing the backbone and neck to enjoy the rich supervision provided by the one-to-many assignment. During inference, the one-to-many head is discarded, and the two one-to-one heads are used for prediction. Two candidate regions at different scales are obtained for each frame: a background candidate box and a target candidate box. The target candidate box includes a first tracking target, and the background candidate box includes the image containing the target candidate box, indicating the location of the first tracking target to be predicted. The input module inputs the background candidate box and the target candidate box into the first feature extraction network in the behavior recognition model to obtain the first feature map of the background candidate box and the second feature map of the target candidate box; The processing module inputs the first feature map and the second feature map into the first prediction network in the behavior recognition model to obtain a first prediction map corresponding to the first prediction network. The first prediction network includes M key point prediction units in sequence. The first prediction map is used to indicate the predicted position of the first tracking target on the background candidate box, where M is an integer greater than 1. According to the first prediction map, the position of the first tracking target is marked on the background candidate box. The output module determines the pose of the first tracked target based on the changes in the position of the first tracked target in consecutive frames; The system detects key points of the first tracked target's posture, acquiring candidate bounding boxes and 17 key points, each composed of three parameters: x, y, and c. If the confidence level of a key point is lower than a preset confidence threshold, it is considered occluded. The coordinates of the candidate bounding boxes, the coordinates of the 17 key points, and the occlusion status of the key points are stored in the trajectory's database. If other trajectories are detected, this trajectory is matched with them. If the match is successful, the two trajectories are merged. If the match fails, this trajectory is saved in the historical trajectories. Based on actual needs, the 17 key points of the first tracked target and their corresponding confidence levels saved in this trajectory are used to recognize key actions.
8. The behavior detection system based on the target detection model according to claim 7, characterized in that, The construction module includes a head network, comprising a first one-to-one detection head, a second one-to-one detection head, and a one-to-many detection head; The first and second one-to-one detection heads retain the same structure and optimization objective as the original one-to-many branch. During training, the three detection heads are optimized together with the model. During inference, the one-to-many detection head is discarded, and the first and second one-to-one detection heads are used to make predictions to obtain background candidate boxes and target candidate boxes.
9. The behavior detection system based on the target detection model according to claim 7, characterized in that, A first one-to-one detection head and a second one-to-one detection head are added to the head network, and a CIB module is introduced. The kernel size of the second 3×3 depth convolution of the CIB module in the first one-to-one detection head is increased to 7×7 to obtain background candidate boxes.
Citation Information
Patent Citations
Target tracking method and device, electronic equipment and storage medium
CN113763415A