Shielding target tracking method and device, equipment and storage medium
By adopting the occlusion target tracking method in AI visual motion capture technology, and using feature map detection and mask repair technology, the problem of insufficient motion capture accuracy and robustness under the occlusion problem is solved, and more efficient occlusion target tracking and motion capture effects are achieved.
Patent Information
- Application Number
- CN202411582329.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-06
- Publication Date
- 2025-03-25
AI Technical Summary
The existing AI visual motion capture technology has problems of insufficient accuracy and robustness when dealing with occlusion problems, resulting in unsatisfactory or inaccurate motion capture results.
A occlusion target tracking method is proposed. By obtaining the feature map of video frame data, occlusion target detection process is performed, an initial mask feature map is generated, and the final mask feature map is generated through mask repair processing, and the motion trajectory of the occlusion target is tracked based on the mask.
It effectively improves the accuracy and efficiency of occlusion target tracking, reduces the impact of background noise on subsequent tracking, saves computing resources, and reduces the calculation of irrelevant areas, thereby achieving improved accuracy and robustness of motion capture.
Smart Images
Figure CN119693413A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image detection technology, and in particular to a method, device, equipment and storage medium for tracking an occluded target. Background Art
[0002] AI visual motion capture technology refers to the use of computer vision and artificial intelligence algorithms to capture and analyze the movement of objects or people. AI visual motion capture technology is widely used in the medical field. For example, in medical education, medical students or medical staff can use motion capture systems based on AI visual motion capture technology to learn and simulate various medical operations or surgical procedures to improve their skills and experience; in medical diagnosis, for some diseases or abnormal conditions that are difficult to detect or diagnose through routine examinations or laboratory tests, motion capture systems can be used to analyze and identify patients' movement characteristics or abnormal behaviors, thereby providing early screening or early warning.
[0003] At present, AI visual motion capture technology has some challenges and difficulties in dealing with occlusion problems, resulting in unsatisfactory or inaccurate motion capture results. For example, when the captured target is occluded by other objects or people, the motion capture system may not be able to correctly identify and track the target, or when the captured target is similar in color to the background or other objects, the motion capture system may not be able to effectively distinguish between targets and non-targets, affecting the accuracy and robustness of motion capture.
[0004] In the related technology, multiple cameras are usually used or combined with depth cameras, infrared cameras, etc. to obtain multi-view or multi-modal data, so as to enhance the adaptability of the motion capture system to occlusion by fusing multi-view or multi-modal data. However, fusing multi-view or multi-modal data requires adding more hardware equipment and computing resources, which increases costs.
[0005] In related technologies, methods based on key points or skeletons are also used to simplify the representation and tracking of targets by motion capture systems. For example, human posture estimation models such as OpenPose, AlphaPose, and HRNet are used to detect and identify the key points or skeletons of targets, thereby estimating the motion information of targets based on the positions of the key points or skeletons. However, methods based on key points or skeletons may not be able to fully express the shape and details of the target, and are less effective for targets with severe occlusion or complex postures, affecting the tracking accuracy of the target.
[0006] In related technologies, there are also methods based on image segmentation to improve the segmentation and recognition capabilities of motion capture systems for targets. For example, image segmentation models such as DeepLab and U-Net are used to assign each pixel in the image to a category or an instance, thereby determining the boundary and shape of the target based on the category or instance at the pixel level. However, methods based on image segmentation may not be able to accurately distinguish the boundaries between different categories or instances, and have poor effects on small objects or low-resolution images, affecting the tracking accuracy of the target.
[0007] Therefore, how to effectively improve the accuracy of occluded target tracking while saving costs to improve the accuracy and robustness of motion capture has become a technical problem that needs to be solved urgently. Summary of the invention
[0008] The main purpose of the embodiments of the present application is to propose a method, device, equipment and storage medium for tracking an occluded target, aiming to save costs and effectively improve the accuracy of tracking an occluded target, so as to improve the accuracy and robustness of motion capture.
[0009] To achieve the above object, a first aspect of an embodiment of the present application provides a method for tracking an occluded target, the method comprising:
[0010] Get the feature map of video frame data;
[0011] Performing occluded target detection processing on the feature map to obtain the occluded target and the position of the occluded target in the feature map;
[0012] Based on the position, masking the occluded object on the feature map to obtain a first mask feature map, wherein the first mask feature map includes an initial mask of the occluded object;
[0013] Performing a repair process on the initial mask on the first mask feature map to obtain a second mask feature map, wherein the second mask feature map includes a mask of the occluded object;
[0014] The motion trajectory of the occluded object is tracked based on the mask on the second mask feature map.
[0015] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a device for tracking an obstructed target, the device comprising:
[0016] A feature map acquisition module, used to acquire a feature map of video frame data;
[0017] An occluded target detection module is used to perform occluded target detection processing on the feature map to obtain the occluded target and the position of the occluded target in the feature map;
[0018] An occluded object mask module, configured to perform mask processing on the occluded object on the feature map based on the position to obtain a first mask feature map, wherein the first mask feature map includes an initial mask of the occluded object;
[0019] A mask repair module, configured to repair the initial mask on the first mask feature map to obtain a second mask feature map, wherein the second mask feature map includes a mask of the occluded object;
[0020] An occluded target tracking module is used to track the motion trajectory of the occluded target based on the mask on the second mask feature map.
[0021] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes a computer device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the method described in the first aspect when executing the computer program.
[0022] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.
[0023] The present application proposes an occluded target tracking method, apparatus, device and storage medium. The occluded target tracking method obtains a feature map of video frame data; performs occluded target detection processing on the feature map to obtain the occluded target and the position of the occluded target in the feature map; based on the position, performs mask processing on the occluded target on the feature map to obtain a first mask feature map, wherein the first mask feature map includes an initial mask of the occluded target; performs repair processing on the initial mask on the first mask feature map to obtain a second mask feature map, wherein the second mask feature map includes a mask of the occluded target; and tracks the motion trajectory of the occluded target based on the mask on the second mask feature map. In this way, by performing occluded target detection processing on the feature map, the occluded target and its position can be accurately identified, so that based on the position, by performing mask processing on the occluded target, the boundary and shape of the occluded target can be accurately characterized, effectively addressing the occlusion problem of the occluded target, and then by repairing the initial mask of the occluded target, the integrity of the boundary and shape of the occluded target can be better characterized, thereby reducing the impact of background noise on subsequent tracking, so as to concentrate computing resources and only focus on important masks, and realize tracking of the motion trajectory of the occluded target at the mask level, reducing the calculation of irrelevant areas, thereby saving costs and effectively improving the tracking accuracy and efficiency of the occluded target. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1is a flow chart of an occluded target tracking method provided by an embodiment of the present application;
[0025] Figure 2 It is a schematic diagram of an architecture for tracking an occluded target provided in an embodiment of the present application;
[0026] Figure 3 yes Figure 1 A detailed flowchart of step S102 in FIG.
[0027] Figure 4 is a schematic block diagram of an occluded target tracking device provided in an embodiment of the present application;
[0028] Figure 5 It is a schematic block diagram of the structure of the computer device provided in the embodiment of the present application. DETAILED DESCRIPTION
[0029] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0030] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0032] First, some nouns involved in this application are analyzed:
[0033] YOLOv5 network: It is an efficient and advanced deep network model in the field of object detection and is a new version of the YOLO (You Only Look Once) series. YOLOv5 inherits the advantages of the YOLO series and is optimized to have better real-time detection performance and higher accuracy.
[0034] Mask R-CNN network: is a deep learning network for instance segmentation, which extends Faster R-CNN by adding a segmentation mask prediction function on the basis of object detection. Mask R-CNN performs well in instance segmentation tasks, can perform object detection and segmentation at the same time, and can be used for a variety of visual tasks such as object detection, instance segmentation, and key point detection.
[0035] Generative Adversarial Network (GAN): is a deep learning framework that consists of two neural networks, the generator and the discriminator. They are trained through an adversarial process to generate realistic data.
[0036] Object tracking (Deep Learning-based SORT, DeepSort) network: is a target tracking algorithm network that combines deep learning and classic tracking technology. It mainly realizes efficient and accurate real-time target tracking in complex scenes. The DeepSort network has significantly improved accuracy and efficiency and is an important tool in the field of target tracking.
[0037] Based on this, the embodiments of the present application provide a method, device, equipment and storage medium for tracking an occluded target, aiming to save costs and effectively improve the accuracy of tracking an occluded target, so as to improve the accuracy and robustness of motion capture.
[0038] The occluded target tracking method, device, equipment and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the occluded target tracking method in the embodiments of the present application is described.
[0039] The embodiments of the present application can acquire and process relevant data based on business security technology. Among them, business security (Artificial Intelligence, AI) is the theory, method, technology and application system of using digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0040] Business security basic technologies generally include sensors, dedicated business security chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. Business security software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.
[0041] The occluded target tracking method provided in the embodiment of the present application relates to the field of image detection technology. The occluded target tracking method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, and can also be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and business security platforms; the software can be an application that implements the occluded target tracking method, such as an insurance service platform for corporate customers developed by a financial institution, but is not limited to the above forms.
[0042] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.
[0043] Figure 1 is an optional flowchart of the occluded target tracking method provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S105.
[0044] Step S101, obtaining a feature map of video frame data.
[0045] The video stream data for tracking occluded targets affected by occlusion can be collected by an image acquisition device such as a camera. The occluded targets refer to various objects that are partially or completely occluded by other objects, environmental elements (such as walls) or other occluders. The occluded targets include pedestrians, vehicles, animals, equipment and other categories. The occluded targets can be dynamic.
[0046] In order to facilitate tracking of dynamic occluded targets, the video stream data is divided into continuous single video frame data, and multi-level feature maps are extracted from the video frame data, wherein the feature maps cover important information in the video frame data.
[0047] Step S102, performing occluded target detection processing on the feature map to obtain the occluded target and the position of the occluded target in the feature map.
[0048] See also Figure 2 , Figure 2 This is a schematic diagram of the architecture for tracking an occluded target according to an embodiment of the present application. Figure 2 As shown, the architecture includes a dynamic occlusion detector, a dynamic occlusion segmenter, a dynamic occlusion restorer and a dynamic occlusion tracker. The embodiment of the present application combines the dynamic occlusion detector, the dynamic occlusion segmenter, the dynamic occlusion restorer and the dynamic occlusion tracker to track the motion trajectory of the occluded target.
[0049] Among them, the feature map is processed by occlusion target detection through a dynamic occlusion detector to obtain at least one occlusion target and the position of the occlusion target in the feature map.
[0050] Since the dynamic occlusion detector has high detection speed, accuracy, robustness and adaptability, the occlusion mark detection processing of the feature map through the dynamic occlusion detector can improve the accuracy, efficiency and convenience of detecting the occluded target and its position, so that the movement of the occluded target can still be quickly tracked when the occluded target is occluded.
[0051] Step S103: Based on the position, mask processing is performed on the occluded target on the feature map to obtain a first mask feature map, wherein the first mask feature map includes an initial mask of the occluded target.
[0052] Among them, through the dynamic occlusion segmentor, based on the position of the occluded target in the feature map, the occluded target is masked on the feature map to obtain a first mask feature map, wherein the first mask feature map contains an initial mask of the occluded target.
[0053] Since the dynamic occlusion segmenter has high segmentation speed, accuracy, robustness and adaptability, the dynamic occlusion segmenter can quickly and accurately segment the occluded target on the feature map and generate an initial mask of the occluded target, so that the initial mask can clearly and accurately characterize the boundary and shape of the occluded target.
[0054] Step S104, performing repair processing on the initial mask on the first mask feature map to obtain a second mask feature map, wherein the second mask feature map includes a mask of the occluded target.
[0055] Among them, the initial mask of the occluded target is repaired on the first mask feature map through the dynamic occlusion restorer to obtain the second mask feature map, and the second mask feature map contains the mask of the occluded target.
[0056] Since the dynamic occlusion restorer has a high mask optimization performance, the initial mask of the occluded target can be repaired by the dynamic occlusion restorer, which can improve the quality of the mask of the occluded target and make the mask of the occluded target more clearly and accurately represent the complete boundary and shape of the occluded target.
[0057] Step S105, tracking the motion trajectory of the occluded target based on the mask on the second mask feature map.
[0058] Wherein, a dynamic occlusion tracker is used to track the motion trajectory of the occluded target based on the mask on the second mask feature map.
[0059] Since the dynamic occlusion tracker combines deep learning and traditional filtering technology, it can efficiently and accurately track targets in dynamic scenes, especially when the target is occluded. In this way, the dynamic occlusion tracker can track the occluded target at the mask level, eliminate the boundary blur problem of the occluded target, and effectively improve the tracking accuracy of the occluded target.
[0060] In summary, through the multi-level mask processing and tracking mechanism, the accuracy, robustness and efficiency of occluded target tracking are improved, which can provide strong support for practical applications. For example, it can effectively improve the ability to cope with target occlusion problems in application scenarios such as medical education and medical diagnosis in the medical field.
[0061] The occluded target tracking method provided in the above embodiment obtains a feature map of video frame data; performs occluded target detection processing on the feature map to obtain the occluded target and the position of the occluded target in the feature map; based on the position, performs mask processing on the occluded target on the feature map to obtain a first mask feature map, wherein the first mask feature map includes an initial mask of the occluded target; performs repair processing on the initial mask on the first mask feature map to obtain a second mask feature map, wherein the second mask feature map includes a mask of the occluded target; and tracks the motion trajectory of the occluded target based on the mask on the second mask feature map. In this way, by performing occluded target detection processing on the feature map, the occluded target and its position can be accurately identified, so that based on the position, by performing mask processing on the occluded target, the boundary and shape of the occluded target can be accurately characterized, effectively addressing the occlusion problem of the occluded target, and then by repairing the initial mask of the occluded target, the integrity of the boundary and shape of the occluded target can be better characterized, thereby reducing the impact of background noise on subsequent tracking, so as to concentrate computing resources and only focus on important masks, and realize tracking of the motion trajectory of the occluded target at the mask level, reducing the calculation of irrelevant areas, thereby saving costs and effectively improving the tracking accuracy and efficiency of the occluded target.
[0062] See also Figure 3In some embodiments, step S102 may include but is not limited to steps S1021 to S1024.
[0063] Step S1021, performing occlusion target detection processing on the feature map through a dynamic occlusion detector to obtain a bounding box and the confidence, offset, coordinates and category probability of the bounding box;
[0064] Step S1022, selecting candidate bounding boxes from the bounding boxes according to the confidence level;
[0065] Step S1023, classifying the candidate bounding boxes according to the confidence and category probability of the candidate bounding boxes to obtain the occluded target;
[0066] Step S1024: Adjust the position of the candidate bounding box according to the offset and coordinates of the candidate bounding box to obtain a position.
[0067] Exemplarily, the dynamic occlusion detector can be a trained YOLOv5 network. The dynamic occlusion detector can use single stage, full convolution, anchor-independent, multi-scale perception, adaptive adjustment and other methods to improve detection speed, accuracy, robustness and adaptability. At the same time, it can detect on input feature maps of any size and scale, and can automatically adjust the size and scale of the output bounding box to adapt to targets of different scales and shapes. Among them, the bounding box is a rectangular box used to represent the position of the target in the feature map, and the bounding box is defined by four coordinates (xmin, ymin, xmax, ymax). In addition, the dynamic occlusion detector can detect multiple categories or multiple instances on a feature map at the same time, and can distinguish the overlap or occlusion relationship between different categories or instances, thereby effectively handling the dynamic occlusion problem.
[0068] Based on this, the feature map can be input into the dynamic occlusion detector for occlusion target detection processing, and at least two bounding boxes and their confidence, offset, coordinates and category probabilities are first obtained, wherein the confidence refers to the degree of confidence of the dynamic occlusion detector in the existence of the occluded target, the offset refers to the position offset of the bounding box relative to the anchor box, and the category probability refers to the probability of the bounding box in each category; then the confidence of each bounding box is compared with a preset confidence threshold, and the bounding boxes with confidence less than the preset confidence threshold are filtered, so as to retain the remaining bounding boxes as candidate bounding boxes, wherein the preset confidence threshold is determined when training the YOLOv5 network; then based on the confidence and category probability of the candidate bounding box, the candidate bounding box is classified to obtain the occluded target; and based on the offset and coordinates of the candidate bounding box, the position of the candidate bounding box is adjusted to obtain the accurate position of the occluded target in the feature map.
[0069] In the above manner, the dynamic occlusion detector can effectively identify and locate the occluded target.
[0070] In step S1023 of some embodiments, the confidence and category probability of the candidate bounding box may be dot-multiplied to obtain the category probability score of the candidate bounding box; and the candidate bounding box may be classified according to the category probability score to obtain the occluded target.
[0071] The confidence and category probability of the candidate bounding box can be dot-multiplied to obtain the probability score of the candidate bounding box in each category, and then the candidate bounding box with the highest probability score is retained through non-maximum suppression. The category of the candidate bounding box with the highest probability score is the occluded target.
[0072] Through the above method, the accuracy and reliability of occluded target detection can be ensured.
[0073] In step S103 of some embodiments, the occluded target may be segmented on the feature map by a dynamic occlusion segmentor to obtain feature information of the occluded target; the occluded target may be masked according to the position and feature information to obtain a first mask feature map.
[0074] Exemplarily, the dynamic occlusion segmenter can be a trained Mask R-CNN network. The dynamic occlusion segmenter can accurately, meticulously, and continuously predict the mask of the target on the feature map based on the edge, texture, contour and other information of the target in the feature map, and output the category or instance label of the mask based on the pixel features and positions in the mask to determine the boundary and shape of the target. The dynamic occlusion segmenter uses a two-stage, multi-task, feature pyramid, region of interest and other methods to improve the segmentation speed, accuracy, robustness and adaptability. It can perform segmentation on input feature maps of any size and scale, and can automatically adjust the size and scale of the output mask to adapt to targets of different scales and shapes. The dynamic occlusion segmenter can simultaneously segment multiple categories or instances on a feature map, and can distinguish the overlapping or occlusion relationship between different categories or instances, thereby effectively handling the dynamic occlusion problem.
[0075] Based on this, the feature map and the position of the occluded target can be input into a dynamic occlusion segmenter, so that the occluded target can be segmented on the feature map by the dynamic occlusion segmenter to obtain feature information of the occluded target, wherein the feature information of the occluded target includes information such as the edge, texture, and contour of the occluded target; then, according to the position of the occluded target and the feature information of the occluded target, the occluded target is masked to generate an initial mask of the occluded target, thereby obtaining a first mask feature map.
[0076] In this way, by performing mask processing on the occluded target, the obtained initial mask can be used to effectively deal with the occlusion phenomenon in the dynamic scene.
[0077] In step S104 of some embodiments, the context information of the occluded target can be extracted from the first mask feature map by a dynamic occlusion restorer; the initial mask is repaired according to the feature information and the context information to obtain a second mask feature map.
[0078] Exemplarily, the dynamic occlusion restorer can be a trained GAN network. The dynamic occlusion restorer can restore the shape and details of the target on the feature map according to the features and contextual information of the target in the feature map, thereby generating a complete mask of the target, and output the category or instance label of the complete mask according to the pixel features and positions in the complete mask, thereby determining the complete boundary and shape of the target. The generator of the dynamic occlusion restorer is used to generate a complete mask from the feature map, and the discriminator of the dynamic occlusion restorer is used to judge its authenticity and quality from the complete mask. There is an adversarial learning process between the generator and the discriminator. The generator tries to generate more and more realistic and high-quality complete masks, and the discriminator tries to distinguish between real and generated complete masks more and more accurately. Through such an adversarial learning process, the dynamic occlusion restorer can improve the recovery speed, accuracy, robustness and fidelity, thereby optimizing and repairing the mask on the input feature map of any size and scale, and can also automatically adjust the size and scale of the output complete mask to adapt to targets of different scales and shapes. The dynamic occlusion restorer can restore multiple categories or instances on a feature map at the same time, and can repair the overlapping or occlusion relationship between different categories or instances, thereby effectively handling the dynamic occlusion problem. In other words, the dynamic occlusion restorer can restore the shape and details of the occluded target on the feature map and output the complete mask of the target.
[0079] Based on this, the first mask feature map and the feature information of the occluded target can be input into the dynamic occlusion restorer to first extract the context information of the occluded target, and then the initial mask of the occluded target is repaired according to the feature information and context information of the occluded target to optimize the shape and details of the initial mask on the feature map, and obtain the mask of the occluded target and the second mask feature map.
[0080] In this way, by repairing the initial mask, the impact of background noise on subsequent tracking can be greatly reduced. Even when the occluded target is completely occluded, the mask repair can continue to track the state of the occluded target, thereby reducing the risk of tracking loss.
[0081] In step S105 of some embodiments, a dynamic occlusion tracker may be used to perform state analysis on the occluded target based on the mask on the second mask feature map to obtain state information of the occluded target; and the motion trajectory is generated based on the state information.
[0082] Exemplarily, the dynamic occlusion tracker can be a trained DeepSort network. The dynamic occlusion tracker can track the motion trajectory of the target on the feature map and output the unique identification and motion information of the target. The dynamic occlusion tracker can stably, continuously and quickly track the motion trajectory of the target on the feature map according to the target position, direction, speed and other information in the feature map, and output the unique identification and motion information of the target according to the characteristics and position of the target in the motion trajectory, so as to determine the motion state and behavior of the target. The dynamic occlusion tracker is mainly composed of a deep convolutional neural network and a Kalman filter. It can track on input feature maps of any size and scale, and can automatically adjust the size and scale of the output motion trajectory to adapt to targets of different scales and shapes. At the same time, multiple categories or instances can be tracked simultaneously on a feature map, and the overlapping or occlusion relationship between different categories or instances can be handled, thereby effectively handling the dynamic occlusion problem.
[0083] Based on this, a dynamic occlusion tracker can be used to perform state analysis on the occluded target based on the mask on the second mask feature map to obtain the state information of the occluded target, and then generate a motion trajectory based on the state information of the occluded target to achieve efficient and accurate tracking of occluded targets in dynamic scenes.
[0084] See also Figure 4 , Figure 4 It is a schematic block diagram of an occluded target tracking device provided in an embodiment of the present application. The occluded target tracking device can be configured in a computer device to execute the aforementioned occluded target tracking method.
[0085] like Figure 4 As shown, the occluded target tracking device 400 includes: a feature map acquisition module 401, an occluded target detection module 402, an occluded target mask module 403, a mask repair module 404 and an occluded target tracking module 405.
[0086] A feature map acquisition module 401 is used to acquire a feature map of video frame data;
[0087] An occluded target detection module 402 is used to perform occluded target detection processing on the feature map to obtain an occluded target and a position of the occluded target in the feature map;
[0088] An occluded object masking module 403 is used to perform masking processing on the occluded object on the feature map based on the position to obtain a first mask feature map, wherein the first mask feature map includes an initial mask of the occluded object;
[0089] A mask repair module 404 is used to repair the initial mask on the first mask feature map to obtain a second mask feature map, wherein the second mask feature map includes the mask of the occluded object;
[0090] The occluded target tracking module 405 is used to track the motion trajectory of the occluded target based on the mask on the second mask feature map.
[0091] In one embodiment, the occluded object detection module 402 is further configured to:
[0092] Performing occlusion target detection processing on the feature map through a dynamic occlusion detector to obtain a bounding box and a confidence, offset, coordinates, and category probability of the bounding box;
[0093] According to the confidence level, selecting a candidate bounding box from the bounding box;
[0094] Classify the candidate bounding box according to the confidence and category probability of the candidate bounding box to obtain the occluded target;
[0095] The position of the candidate bounding box is adjusted according to the offset and the coordinates of the candidate bounding box to obtain the position.
[0096] In one embodiment, the occluded object detection module 402 is further configured to:
[0097] Performing a dot product process on the confidence and category probability of the candidate bounding box to obtain a category probability score of the candidate bounding box;
[0098] The candidate bounding box is classified according to the category probability score to obtain the occluded target.
[0099] In one embodiment, the occlusion target mask module 403 is further used to:
[0100] Segmenting the occluded target on the feature map through a dynamic occlusion segmentor to obtain feature information of the occluded target;
[0101] The occluded target is subjected to mask processing according to the position and the feature information to obtain the first mask feature map.
[0102] In one embodiment, the mask repair module 404 is further configured to:
[0103] Extracting context information of the occluded object from the first mask feature map by a dynamic occlusion restorer;
[0104] The initial mask is repaired according to the feature information and the context information to obtain a second mask feature map.
[0105] In one embodiment, the occluded target tracking module 405 is further used to:
[0106] By means of a dynamic occlusion tracker, a state analysis is performed on the occluded target based on the mask on the second mask feature map to obtain state information of the occluded target;
[0107] The motion trajectory is generated according to the state information.
[0108] In one embodiment, the dynamic occlusion segmenter includes a trained YOLOv5 network, the dynamic occlusion segmenter includes a trained Mask R-CNN network, the dynamic occlusion restorer includes a trained GAN network, and the dynamic occlusion tracker includes a trained DeepSort network.
[0109] Among them, each module in the above-mentioned occluded target tracking device 400 corresponds to each step in the above-mentioned occluded target tracking method embodiment, and its functions and implementation processes are no longer repeated here one by one.
[0110] The occluded target tracking device 400 can execute the occluded target tracking method provided in the embodiment of the present application, and therefore can achieve the beneficial effects that can be achieved by the occluded target tracking method provided in the embodiment of the present application. Please refer to the previous embodiment for details and will not be repeated here.
[0111] The method and apparatus of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.
[0112] Exemplarily, the above method and apparatus may be implemented in the form of a computer program. The computer program may be implemented in the form of a computer program. Figure 5 Runs on the computer device shown.
[0113] See also Figure 5 , Figure 5 It is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application.
[0114] See also Figure 5 The computer device includes a processor and a memory connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.
[0115] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.
[0116] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any occluded target tracking method.
[0117] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0118] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:
[0119] Get the feature map of video frame data;
[0120] Performing occluded target detection processing on the feature map to obtain the occluded target and the position of the occluded target in the feature map;
[0121] Based on the position, masking the occluded object on the feature map to obtain a first mask feature map, wherein the first mask feature map includes an initial mask of the occluded object;
[0122] Performing a repair process on the initial mask on the first mask feature map to obtain a second mask feature map, wherein the second mask feature map includes a mask of the occluded object;
[0123] The motion trajectory of the occluded object is tracked based on the mask on the second mask feature map.
[0124] In one embodiment, when the processor performs occluded target detection processing on the feature map to obtain the occluded target and the position of the occluded target in the feature map, it is used to implement:
[0125] Performing occlusion target detection processing on the feature map through a dynamic occlusion detector to obtain a bounding box and a confidence, offset, coordinates, and category probability of the bounding box;
[0126] According to the confidence level, selecting a candidate bounding box from the bounding box;
[0127] Classify the candidate bounding box according to the confidence and category probability of the candidate bounding box to obtain the occluded target;
[0128] The position of the candidate bounding box is adjusted according to the offset and the coordinates of the candidate bounding box to obtain the position.
[0129] In one embodiment, when the processor performs classification processing on the candidate bounding box according to the confidence and category probability of the candidate bounding box to obtain the occluded target, it is configured to implement:
[0130] Performing a dot product process on the confidence and category probability of the candidate bounding box to obtain a category probability score of the candidate bounding box;
[0131] The candidate bounding box is classified according to the category probability score to obtain the occluded target.
[0132] In one embodiment, when the processor performs mask processing on the occluded target on the feature map based on the position to obtain the first mask feature map, it is used to implement:
[0133] By means of a dynamic occlusion segmenter, the occluded target is segmented on the feature map to obtain feature information of the occluded target;
[0134] The occluded target is subjected to mask processing according to the position and the feature information to obtain the first mask feature map.
[0135] In one embodiment, when the processor implements the repairing process on the initial mask on the first mask feature map to obtain the second mask feature map, it is further used to implement:
[0136] Extracting context information of the occluded object from the first mask feature map by a dynamic occlusion restorer;
[0137] The initial mask is repaired according to the feature information and the context information to obtain a second mask feature map.
[0138] In one embodiment, when the processor implements the tracking of the motion trajectory of the occluded target based on the mask on the second mask feature map, it is configured to implement:
[0139] By means of a dynamic occlusion tracker, a state analysis is performed on the occluded target based on the mask on the second mask feature map to obtain state information of the occluded target;
[0140] The motion trajectory is generated according to the state information.
[0141] In one embodiment, the dynamic occlusion segmenter includes a trained YOLOv5 network, the dynamic occlusion segmenter includes a trained Mask R-CNN network, the dynamic occlusion restorer includes a trained GAN network, and the dynamic occlusion tracker includes a trained DeepSort network.
[0142] The computer device can execute the occluded target tracking method provided in the embodiment of the present application, and therefore can achieve the beneficial effects that can be achieved by the occluded target tracking method provided in the embodiment of the present application. Please refer to the previous embodiment for details and will not be repeated here.
[0143] An embodiment of the present application also provides a computer-readable storage medium.
[0144] A computer program is stored on the computer-readable storage medium of the present application, and when the computer program is executed by a processor, the steps of the above-mentioned occluded target tracking method are implemented.
[0145] The computer-readable storage medium may be an internal storage unit of the obstructed target tracking device or computer device described in the aforementioned embodiment, such as a hard disk or memory of the obstructed target tracking device or computer device. The computer-readable storage medium may also be an external storage device of the obstructed target tracking device or computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital card (Secure Digital Card, SD Card), a flash card (Flash Card), etc., equipped on the obstructed target tracking device or computer device.
[0146] Since the computer program stored in the computer-readable storage medium can execute any of the occluded target tracking methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any of the occluded target tracking methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.
[0147] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.
[0148] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.
[0149] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0150] The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and these modifications or substitutions should be included in the protection scope of the present application.
Claims
1. A method for tracking an occluded target, characterized in that: The method comprises: Get the feature map of video frame data; Performing occluded target detection processing on the feature map to obtain the occluded target and the position of the occluded target in the feature map; Based on the position, masking the occluded object on the feature map to obtain a first mask feature map, wherein the first mask feature map includes an initial mask of the occluded object; Performing a repair process on the initial mask on the first mask feature map to obtain a second mask feature map, wherein the second mask feature map includes a mask of the occluded object; The motion trajectory of the occluded object is tracked based on the mask on the second mask feature map.
2. The method according to claim 1, characterized in that The performing occluded target detection processing on the feature map to obtain the occluded target and the position of the occluded target in the feature map includes: Performing occlusion target detection processing on the feature map through a dynamic occlusion detector to obtain a bounding box and a confidence, offset, coordinates, and category probability of the bounding box; According to the confidence level, selecting a candidate bounding box from the bounding box; Classify the candidate bounding box according to the confidence and category probability of the candidate bounding box to obtain the occluded target; The position of the candidate bounding box is adjusted according to the offset and the coordinates of the candidate bounding box to obtain the position.
3. The method according to claim 2, characterized in that The classifying the candidate bounding box according to the confidence and category probability of the candidate bounding box to obtain the occluded target includes: Performing a dot product process on the confidence and category probability of the candidate bounding box to obtain a category probability score of the candidate bounding box; The candidate bounding box is classified according to the category probability score to obtain the occluded target.
4. The method according to claim 1, characterized in that: The step of performing mask processing on the occluded target on the feature map based on the position to obtain a first mask feature map includes: By means of a dynamic occlusion segmenter, the occluded target is segmented on the feature map to obtain feature information of the occluded target; The occluded target is subjected to mask processing according to the position and the feature information to obtain the first mask feature map.
5. The method according to claim 4, characterized in that The repairing process is performed on the initial mask on the first mask feature map to obtain a second mask feature map, comprising: Extracting context information of the occluded object from the first mask feature map by a dynamic occlusion restorer; The initial mask is repaired according to the feature information and the context information to obtain a second mask feature map.
6. The method according to claim 1, characterized in that Tracking the motion trajectory of the occluded target based on the mask on the second mask feature map includes: By means of a dynamic occlusion tracker, a state analysis is performed on the occluded target based on the mask on the second mask feature map to obtain state information of the occluded target; The motion trajectory is generated according to the state information.
7. The method according to any one of claims 1 to 6, characterized in that: The dynamic occlusion segmenter includes a trained YOLOv5 network, the dynamic occlusion segmenter includes a trained Mask R-CNN network, the dynamic occlusion restorer includes a trained GAN network, and the dynamic occlusion tracker includes a trained DeepSort network.
8. An occluded target tracking device, characterized in that: The device comprises: A feature map acquisition module, used to acquire a feature map of video frame data; An occluded target detection module is used to perform occluded target detection processing on the feature map to obtain the occluded target and the position of the occluded target in the feature map; An occluded object mask module, configured to perform mask processing on the occluded object on the feature map based on the position to obtain a first mask feature map, wherein the first mask feature map includes an initial mask of the occluded object; A mask repair module, configured to repair the initial mask on the first mask feature map to obtain a second mask feature map, wherein the second mask feature map includes a mask of the occluded object; An occluded target tracking module is used to track the motion trajectory of the occluded target based on the mask on the second mask feature map.
9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the occluded target tracking method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the occluded target tracking method according to any one of claims 1 to 7 is implemented.