Dynamic target detection method based on deep learning

Abstract

The invention proposes a dynamic target detection method based on deep learning, and the method comprises the steps: extracting a dynamic target through unsupervised background modeling and morphological processing, combining a deep feature network with a lightweight visibility network, and outputting a target structured state; constructing a trajectory velocity modeling network through a differential equation according to the bounding box and the semantic features, generating a target motion trajectory, and performing numerical integration on the differential equation to obtain a predicted spatial position; fusing the semantic feature vector with the predicted spatial position to generate future appearance features of the target; when a frame image is input, constructing a candidate region based on the predicted spatial position, and calculating a matching confidence coefficient between the candidate region and a future appearance feature of the target to confirm a target position and a target feature; and if the matching confidence coefficient is lower than a preset threshold value, the previous frame state is left, and updating is not carried out, so that the robustness in a shielding or fuzzy state is improved.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning, and particularly relates to a dynamic object detection method based on deep learning. Background Art

[0002] With the rapid development of computer vision and artificial intelligence technologies, object detection methods based on deep learning have been widely applied to multiple dynamic vision scenarios such as intelligent monitoring, autonomous driving, drone vision, and intelligent transportation. In these scenarios, objects are usually in a continuous motion state, and are accompanied by complex factors such as illumination changes, occlusion, deformation, scale changes, and background interference, which pose great challenges to the detection system. The current mainstream object detection technologies are mainly divided into two categories: one is the static frame detection models represented by YOLO, SSD, Faster R-CNN, etc., which extract spatial features in images through convolutional neural networks and output bounding boxes and class labels; the other is the video object detection methods represented by Track-RCNN, FGFA (Flow-guided Feature Aggregation), etc., which attempt to introduce temporal information to enhance the multi-frame fusion ability, thereby improving the detection accuracy and continuity of dynamic objects.

[0003] Although a large number of methods have tried to improve the detection effect in dynamic scenarios by means of temporal context, multi-frame feature fusion, motion information modeling, etc. in recent years, there are still the following several core technical bottlenecks that have not been fundamentally solved. First, most current methods rely on the feature stitching of front and rear frames with a fixed window or optical flow estimation to model the object motion state, but such strategies are extremely prone to failure in the face of complex scenarios such as severe occlusion, fast movement, and background mutation, because they lack the ability to continuously model the motion trend of the object and cannot effectively predict the potential future positions of the object. Second, existing methods generally rely on the visual appearance information of the current frame of the object for recognition and classification. When the object appears blurred, in low light, occluded, or only partially visible, it is difficult for detectors based on local image features to stably output the object position and identity, resulting in problems such as false detection, missed detection, and even object tracking breakage. Third, most detectors do not introduce the modeling of the deep semantic association between the object motion behavior and appearance features in the design, resulting in a decline in the discriminative ability and insufficient robustness of the detector when the object undergoes severe pose changes or semantic deformations.

[0004] Therefore, the existing dynamic object detection technologies face two urgent problems: one is how to model the continuous movement of objects and predict their possible trajectories to achieve the temporal continuity and occlusion recovery ability of the detection process; the other is how to enhance the visual feature expression of objects so that they can still be recognized and tracked under non-ideal conditions, improving the stability and fault tolerance of the entire detection system. The superposition of these two problems directly restricts the application effect and deployment value of the deep object detection system in real dynamic scenarios. Summary of the Invention

[0005] The object of the present invention is to propose a dynamic object detection method based on deep learning, which realizes continuous, stable, and high-precision detection of fast-moving or partially occluded objects by introducing a collaborative framework of a motion trend modeling mechanism and a visual feature enhancement strategy.

[0006] To achieve the above object, the present invention provides a dynamic object detection method based on deep learning, and the method includes the following steps: Obtain an object image, extract dynamic objects through unsupervised background modeling and morphological processing, and combine a deep feature network and a lightweight visibility network to output an object structured state; the object structured state includes: an object bounding box, a semantic feature vector, and a visibility score; Construct a trajectory velocity modeling network according to the bounding box and semantic features through a differential equation to generate an object motion trajectory, introduce an acceleration regularization term to smooth the trajectory change, and finally obtain a predicted spatial position by numerically integrating the differential equation; Obtain a semantic feature vector, fuse the semantic feature vector with the predicted spatial position, generate a future appearance feature of the object through a trajectory-guided feature generation module, and introduce a structure alignment loss and a local attribute diversity preservation term to improve the feature quality; When a frame image is input, construct a candidate region based on the predicted spatial position, calculate the matching confidence between the candidate region and the future appearance feature of the object to confirm the object position and object feature; if the matching confidence is lower than a preset threshold, keep the previous frame state without updating to improve the robustness under occlusion or blur conditions.

[0007] Further, the extracting dynamic objects through unsupervised background modeling and morphological processing, and combining a deep feature network and a lightweight visibility network to output an object structured state specifically includes: Model the image through unsupervised background, perform background subtraction on each frame of image to generate a foreground mask; Perform morphological processing on the foreground mask to enhance the integrity of the object boundary, and obtain an optimized foreground mask from the foreground mask; Extract the connected regions from the optimized foreground mask, and record the regions of the connected regions as the target regions; the bounding box of the target region is the target bounding box; Crop the target region from the target image and send it into the depth feature network to extract semantic features as the semantic feature vector; Extract the structure vector from each target region in the optimized foreground mask; splice the structure vector with the depth feature map of the semantic feature vector and input it into the lightweight visibility network to output the visibility score; Combine the target bounding box, semantic feature vector and visibility score, and output the state of each target.

[0008] Further, the structure vector includes four dimensions: normalized area, aspect ratio, boundary compactness, and isolated target identification.

[0009] Further, the morphological processing includes erosion and dilation operations performed in sequence, which are used to optimize the integrity of the target boundary.

[0010] Further, the unsupervised background modeling uses Gaussian modeling or moving average method to generate the initial foreground mask through pixel-level difference threshold segmentation.

[0011] Further, the differential equation is expressed as: ; where, , is the predicted spatial position, indicating the position state of the target at time ; is the trajectory velocity modeling network with parameters ; is the observation weight factor, expressed as: ; where the formula means restricting the visibility score between the lower limit and 1 to prevent the trajectory from freezing when completely occluded; where the acceleration regularization term constrains the second-order difference of adjacent elements in the time series to suppress trajectory jumps and conform to the physical inertia logic.

[0012] Further, the future appearance features of the target are generated through the trajectory-guided feature generation module, specifically expressed as: ; where, represents the future appearance features of the generated target i; represents the semantic feature vector; is the predicted spatial position; Denote the spatial position encoding module; Denote the fusion model, which is composed of several layers of attention networks and feature transformation layers; Denote the learnable parameters of the TGFS module; Furthermore, the trajectory-guided feature generation module introduces a local attribute diversity preservation term, which is the structural alignment loss of adjacent dimensions of the future appearance feature and is used to enforce the difference in the distribution of feature sub-dimensions.

[0013] Furthermore, the spatial position encoding module is an MLP or SinusoidalEmbedding.

[0014] Furthermore, the generation of the candidate region is centered on the predicted spatial position and is constructed by a multi-scale sliding window or a region proposal network; Among them, the matching confidence is obtained by performing a cosine similarity score on the semantic feature vector extracted from the candidate region in the current frame image and the future appearance feature of the target.

[0015] The beneficial technical effects of the present invention are at least as follows: To solve the dual bottlenecks of existing dynamic target detection technologies in motion modeling and visual expression, the present invention proposes a deep learning target detection method for dynamic scenes. By introducing a collaborative framework of a motion trend modeling mechanism and a visual feature enhancement strategy, continuous, stable, and high-precision detection of fast-moving or partially occluded targets is achieved. The core innovation points of the present invention are mainly reflected in the following two aspects: First, a modeling mechanism for the target state of consecutive frames is proposed, which can learn the motion law of the target in the time dimension and assist the detection module in making forward-looking position inferences when the target is lost or occluded, thereby improving the system's continuous perception ability of dynamic targets; Second, the present invention designs a feature completion mechanism for enhancing target representation, which can still generate high-quality target features when the target appears blurred, low-resolution, or has appearance changes, thereby ensuring the stable recognition of the target identity by the detector. This design of collaborative optimization from the two dimensions of "motion trend prediction" and "visual completion enhancement" can effectively make up for the discontinuity and instability of existing technologies in the dynamic detection process, significantly improve the detection accuracy and robustness in complex actual scenarios, and has strong engineering practical value and promotion prospects. Description of the Drawings

[0016] The present invention is further described with the accompanying drawings, but the embodiments in the drawings do not constitute any limitation to the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the following drawings without creative efforts.

[0017] Figure 1Flowchart of the dynamic object detection method based on deep learning according to the present invention. Detailed implementation manners

[0018] The embodiments of the present invention will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

[0019] As Figure 1 shown, the dynamic object detection method based on deep learning provided by the embodiment of the present invention includes: S1. Obtain a target image, extract a dynamic object through unsupervised background modeling and morphological processing, and combine a deep feature network and a lightweight visibility network to output a target structured state; the target structured state includes: a target bounding box, a semantic feature vector, and a visibility score.

[0020] Specifically, a hybrid structured detection module (HSDM) is proposed in this step, which combines unsupervised background modeling, morphological foreground optimization, and deep feature extraction to achieve robust detection of dynamic objects in complex scenes. The specific steps of the solution are as follows: Construct a background model by Gaussian modeling or moving average method , and perform background subtraction on each frame of image to generate a foreground mask :

[0021] Where: : background model; T: pixel difference threshold; : initial foreground binary mask.

[0022] Further, perform erosion and dilation operations on to obtain an optimized foreground mask and enhance the integrity of the target boundary:

[0023] Further, extract connected regions in , record their bounding boxes . Crop this region from the original image and send it into a deep feature network (such as ResNet) to extract semantic features: ​

[0024] Furthermore, extract the following structural distribution features from each target region to form a structure vector : :

[0025] where: : the height and width of the image; the first dimension: the normalized area ( is the number of pixels in the region); the second dimension: the aspect ratio; the third dimension: the boundary compactness index, reflecting the degree of structural fragmentation; the fourth dimension: whether it is an isolated target (1 means not connected to other targets); each dimension has a clear physical meaning and constitutes a low-dimensional structural distribution descriptor.

[0026] Furthermore, concatenate the structural features with the depth feature map and input them into the lightweight visibility network to output the visibility score :

[0027] where, : the high-level features of the target semantic region output by the backbone network; : the concatenation operation; : two fully connected networks; : the Sigmoid function, outputting a normalized score; : indicating whether the target is visible, and the higher the occlusion / blur degree, the lower it is. This score will be used as a trajectory speed adjustment factor in step 2 to enhance the dynamic modeling ability of the system under occlusion; at the same time, it is also used as a weighting factor for the temporal consistency regularization term in this step to measure the cross-frame semantic stability.

[0028] Furthermore, to enhance the cross-frame feature stability, introduce a regularization term to encourage the semantic consistency between the features of the current frame and the matching targets in the previous frame:

[0029] This regularization term, as an additional loss in the training stage, helps to improve the tracking accuracy and feature smoothness.

[0030] Finally, the state of each target i is characterized as:

[0031] where, : the target bounding box; : the semantic feature vector; : the visibility score.

[0032] Through the above mechanism, this step realizes the joint modeling of the target spatial position, semantic appearance, and observation quality, making the detection results not only applicable to the target localization in this frame but also having the continuity, traceability, and discriminability for backward transmission, supporting the subsequent motion prediction and visual completion modules.

[0033] S2. Construct a trajectory velocity modeling network through a differential equation based on the bounding box and semantic features to generate the target motion trajectory. Introduce an acceleration regularization term to smooth the trajectory change, and finally obtain the predicted spatial position by numerically integrating the differential equation.

[0034] Specifically, this step focuses on the target structured state generated in step 1 to establish a time-continuous trajectory modeling system for predicting the potential position of the target in the next frame. The present invention proposes a "Gated Differential Trajectory Prediction Model (GODE-Tracker)", and its core innovation is: using the visibility score output in step 1 to control the speed weight of trajectory update, so that the model automatically slows down or suppresses trajectory perturbation when the target is occluded or blurred, enhancing the prediction stability.

[0035] First, construct the differential equation of the trajectory change of target i:

[0036] Where: : The position state of the target at time (the upper-left coordinate and width and height); : The trajectory velocity modeling network customized in this step, with parameters ; : The observation weight factor, expressed as:

[0037] means restricting the visibility score in step 1 to the lower limit (such as 0.1) to 1 to prevent the trajectory from freezing when completely occluded.

[0038] Furthermore, to suppress trajectory jumps and conform to the physical inertia logic, an acceleration regularization term is added:

[0039] Where: , : The positions of the target in the previous two frames, sourced from historical states; : The acceleration smoothing factor is used to adjust the penalty of the model for drastic trajectory changes. This item can be added to the loss function during training to encourage the predicted trajectory to maintain smoothness over time.

[0040] Finally, by numerically integrating the above differential system (such as the fourth-order Runge-Kutta), the predicted position of each target at the t+1 frame can be obtained: : The predicted spatial position, which serves as the spatial guiding input for the next-step feature generation module of the next step.

[0041] S3. Obtain the semantic feature vector, fuse the semantic feature vector with the predicted spatial position, generate the future appearance feature of the target through the trajectory-guided feature generation module, and introduce the structure alignment loss and the local attribute diversity preservation term to improve the feature quality.

[0042] Specifically, this step aims to solve the problem of missing target appearance information caused by factors such as target occlusion, blur, and rapid perspective change in dynamic video scenarios. To improve the system's ability to recognize and re-detect targets in invisible or blurred states, the present invention designs a trajectory-guided feature generation module (Trajectory-Guided Feature Synthesizer, abbreviated as TGFS) that fuses historical features and trajectory prediction information to generate the expected features of the target in the next frame . This feature serves as the "identity template" for the subsequent recognition module, especially suitable for situations where the target is temporarily unobservable visually.

[0043] The input of TGFS depends on the outputs of the previous two steps, forming a clear variable transfer path: The semantic feature vector output in step 1 ; The predicted spatial position output in step 2 .

[0044] Among them, the goal of the TGFS module is to construct a function that fuses with and outputs the predicted feature at time t+1 , and this feature needs to satisfy simultaneously: being consistent with the historical semantics, being related to the predicted spatial position, and having robust distinguishability.

[0045] Furthermore, the present invention defines the feature generation process as follows:

[0046] Where: : The future appearance feature of the generated target i; : The historical appearance feature extracted in step 1; : The target position predicted in step 2; : The spatial position encoding module (such as MLP or Sinusoidal Embedding); : The fusion model, which consists of several layers of attention networks and feature transformation layers; : The learnable parameters representing the TGFS module.

[0047] It can be understood that the present invention introduces spatial position encoding and fuses it with semantic features, enabling the generated features to have "spatial consistency", that is, not generating incorrect features that deviate from the true target position.

[0048] Furthermore, to enhance the model's ability to control the semantic quality of the generated features during the training phase, the present invention introduces a structure alignment loss and innovatively adds a local attribute diversity preservation term , which is used to encourage the differences between feature sub-dimensions and improve the discriminative ability during the re-identification process:

[0049] Among them, represents the j-th dimension of the future appearance feature; is the weight coefficient of the diversity loss (recommended to be 0.05); the inner summation encourages the distributional differences between different dimensions and prevents all dimensions from converging to similar values.

[0050] Among them, in the case of occlusion or blur, the lack of visual information may cause the "collapse" of feature dimensions, and the model outputs homogeneous features with low discriminability. This regularization term forces TGFS to retain individuality in sub-dimensions, which helps the subsequent matching module to perform effective identification.

[0051] S4. When a frame of image is input, construct a candidate region based on the predicted spatial position, calculate the matching confidence between the candidate region and the future appearance feature of the target, so as to confirm the target position and target feature; if the matching confidence is lower than the preset threshold, keep the previous frame state without updating, so as to improve the robustness in the occlusion or blur state.

[0052] Specifically, this step is the last execution link of the method of this patent, and its goal is: in the current frame image , utilize the target position predicted in step 2 and the future appearance feature of the target generated in step 3 to confirm the spatial position of the target and identify its identity, and complete a dynamic target detection output under "enhanced guidance".

[0053] Furthermore, with the predicted spatial position as the center, in the image Construct a set of local candidate regions For each region, extract the corresponding features Use the following method to perform similarity matching with the future appearance features of the target as follows:

[0054] where is the cosine similarity score, representing the matching confidence between the j-th candidate region and the target template; is the future appearance feature of the target generated in step 3; is the feature extracted from the candidate region in the current frame image. Find the most matching region and output the detection result of the target in the (t + 1)-th frame: Final target position: ; Final target feature: .

[0055] If the highest matching degree is lower than the threshold (such as 0.6), the system will mark the target as "status uncertain", retain the previous frame status, and not update it to improve the robustness of the system in the occlusion or blur state.

[0056] Finally, this step takes the spatial position and semantic features of each target at time (t + 1) as the structured detection output of the patent system, which is used for output result recording or further feedback to the trajectory modeling module to form a closed loop.

[0057] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0058] In several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Additionally, the couplings or direct couplings or communication connections shown or discussed with each other can be through some interfaces, and the indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0059] When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs.

[0060] Although the embodiments of the present invention have been shown and described, those skilled in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the claims and their equivalents.

Claims

1. A dynamic object detection method based on deep learning, characterized in that, The method includes: Obtain a target image, extract dynamic targets through unsupervised background modeling and morphological processing, and combine a deep feature network and a lightweight visibility network to output the structured state of the target; the structured state of the target includes: a target bounding box, a semantic feature vector, and a visibility score; Construct a trajectory velocity modeling network according to the bounding box and semantic features through a differential equation, generate a target motion trajectory, introduce an acceleration regularization term to smooth the trajectory change, and finally obtain the predicted spatial position by numerically integrating the differential equation; Obtain the semantic feature vector, fuse the semantic feature vector with the predicted spatial position, generate the future appearance feature of the target through a trajectory-guided feature generation module, and introduce a structure alignment loss and a local attribute diversity preservation term to improve the feature quality; When a frame image is input, construct a candidate region based on the predicted spatial position, calculate the matching confidence between the candidate region and the future appearance feature of the target to confirm the target position and target feature; if the matching confidence is lower than the preset threshold, retain the previous frame state without updating to improve the robustness in the occlusion or blur state.

2. The dynamic target detection method based on deep learning according to claim 1, wherein The extraction of dynamic targets through unsupervised background modeling and morphological processing, and the combination of a deep feature network and a lightweight visibility network to output the structured state of the target specifically includes: Model the image through unsupervised background, perform background subtraction on each frame of the image to generate a foreground mask; Perform morphological processing on the foreground mask to enhance the integrity of the target boundary, and obtain an optimized foreground mask from the foreground mask; Extract connected regions in the optimized foreground mask, and record the regions of the connected regions as target regions; the bounding box of the target region is the target bounding box; Crop the target region from the target image and send it into the deep feature network to extract semantic features as the semantic feature vector; Extract structure vectors from each target region in the optimized foreground mask; splice the structure vectors and the depth feature map of the semantic feature vector and input them into the lightweight visibility network to output the visibility score; Combine the target bounding box, semantic feature vector, and visibility score, and output the state of each target.

3. The dynamic target detection method based on deep learning according to claim 1, characterized in that The structure vector includes four dimensions: normalized area, aspect ratio, boundary compactness, and isolated target identification.

4. The dynamic target detection method based on deep learning according to claim 2, characterized in that The morphological processing includes erosion and dilation operations performed in sequence to optimize the integrity of the target boundary.

5. The dynamic target detection method based on deep learning according to claim 1, wherein The unsupervised background modeling uses Gaussian modeling or moving average method to generate an initial foreground mask through pixel-level difference threshold segmentation.

6. The dynamic target detection method based on deep learning according to claim 1, characterized in that, The differential equation is expressed as: ； Among them, , is the predicted spatial position, representing the position state of the target at time ; is the trajectory velocity modeling network, with parameters ; is the observation weight factor, expressed as: ； where the formula represents restricting the visibility score to a lower limit between 0 and 1 to prevent trajectory freezing when there is complete occlusion; Among them, the acceleration regularization term is constrained by the second-order difference of adjacent elements in the time series to suppress trajectory jumps and conform to the physical inertia logic.

7. The dynamic target detection method based on deep learning according to claim 1, wherein The generation of the future appearance feature of the target through the trajectory-guided feature generation module is specifically expressed as: ； Among them, represents the future appearance feature of the generated target i; represents the semantic feature vector; is the predicted spatial position; represents the spatial position encoding module; represents the fusion model, which consists of several layers of attention networks and feature transformation layers; represents the learnable parameters of the TGFS module.

8. The dynamic object detection method based on deep learning according to claim 7, wherein The trajectory-guided feature generation module introduces a local attribute diversity preservation term, and the local attribute diversity preservation term is the structure alignment loss of adjacent dimensions of the future appearance feature, which is used to enforce the difference in the distribution of feature sub-dimensions.

9. The dynamic target detection method based on deep learning according to claim 7, characterized in that, The spatial position encoding module is MLP or SinusoidalEmbedding.

10. The dynamic target detection method based on deep learning according to claim 1, characterized in that The generation of the candidate region is centered on the predicted spatial position and is constructed by a multi-scale sliding window or a region proposal network; Among them, the matching confidence is obtained by performing a cosine similarity score on the semantic feature vector extracted from the candidate region in the current frame image and the future appearance feature of the target.

Images