Multi-target visual identification method and system in dynamic scene

Abstract

The invention belongs to the technical field of visual recognition, and provides a multi-target visual recognition method and system in a dynamic scene, and the method comprises the steps: collecting a multi-target recognition image in real time in the dynamic scene, arranging the multi-target recognition image into a multi-target time sequence image sequence, constructing a background model of the multi-target recognition image, and carrying out the recognition of the multi-target time sequence image sequence; combining with dynamic background interference compensation based on optical flow field calculation, generating a moving target mask matrix, generating a target candidate frame, intercepting an image in the target candidate frame, marking the image as a moving target image, constructing and training a multi-target visual recognition model introduced into a channel space double-attention module, and recognizing the moving target image; and generating an information list of the moving targets in the moving target image according to an identification result, taking the moving targets as nodes, constructing a space-time diagram model containing the nodes and edges, realizing time sequence moving target association, judging whether fracture nodes appear or not, and if yes, constructing a conditional diffusion model to generate virtual samples as the fracture nodes to be matched with connected nodes.

Description

Technical Field

[0001] The present invention belongs to the field of visual recognition technology, and in particular to a multi-target visual recognition method and system in a dynamic scene. Background Art

[0002] In many fields such as modern intelligent monitoring, autonomous driving, and drone inspection, multi-target visual recognition technology in dynamic scenes is crucial. With the growing demand for intelligence and automation in various industries, accurately and efficiently identifying multiple targets in dynamic scenes has become a key issue that needs to be solved urgently.

[0003] In dynamic scenes, there are many complex factors that make multi-target visual recognition face severe challenges. Existing multi-target visual recognition methods have exposed many shortcomings when dealing with these complex situations. Traditional background difference-based methods often have relatively simple background model construction and are difficult to accurately adapt to the complex changes in the background in dynamic scenes. As a result, moving targets are easily misjudged as background during the background update process, or the background cannot be updated in time to accurately detect the target, resulting in target loss or misdetection. For example, in some traffic monitoring videos, when there are dynamic interferences such as shaking leaves in the background, existing multi-target visual recognition methods often misjudge them as moving targets.

[0004] In terms of target detection, the performance of traditional methods is significantly reduced when dealing with complex scenarios such as occlusion, uneven lighting, and dynamic background changes. For the occlusion problem, once the target is partially obscured, detection algorithms based on feature matching or region segmentation are easily unable to accurately identify the target due to missing features or incomplete regions. In the case of uneven lighting, such as in the transition area between indoors and outdoors, the rapid changes in light intensity and color will make it difficult to extract the target's features, resulting in reduced detection accuracy. When the background changes dynamically, such as when the camera moves or there is a large area of ​​background object movement in the scene, traditional detection algorithms are likely to misjudge background changes as target movement, resulting in a large number of false detections.

[0005] In the target recognition link, existing methods have shortcomings in feature extraction and classification. They have poor adaptability to the diversity and changes of targets in complex scenes, and it is difficult to extract effective features that are sufficient to distinguish different targets. There is a lack of effective enhancement of key target features and suppression mechanisms for background interference. In complex dynamic scenes, its recognition accuracy and robustness are still unsatisfactory.

[0006] In response to the above problems, the present invention proposes a multi-target visual recognition method and system in dynamic scenes. Summary of the Invention

[0007] In order to make up for the deficiencies of the prior art, at least one technical problem raised in the background technology is solved.

[0008] The technical solution adopted by the present invention to solve the technical problem is: a multi-target visual recognition method in a dynamic scene, comprising: Collect multi-target recognition images in real time in dynamic scenes and organize them into multi-target time-series image sequences; Based on the multi-target time-series image sequence, a background model of the multi-target recognition image is constructed, and the moving target mask matrix is ​​generated by combining the dynamic background interference compensation based on the optical flow field calculation; Generate target candidate frames based on the moving target mask matrix, capture the image within the target candidate frame, mark it as a moving target image, build and train a multi-target visual recognition model that introduces a channel-space dual-attention module, recognize the moving target image, and generate a list of moving target information in the moving target image based on the recognition results; The moving targets identified in the multi-target time-series image sequence are used as nodes. Based on the information list of the moving targets, a spatiotemporal graph model containing nodes and edges is constructed to realize the association of time-series moving targets and determine whether there are broken nodes. If so, a conditional diffusion model is constructed to generate virtual samples to match the broken nodes with connected nodes. The moving target mask matrix is ​​generated as follows: Acquire a multi-target time-series image sequence, and use an improved Gaussian mixture model to construct a background model for the multi-target recognition image. The background model includes a preset number of Gaussian distributions. Perform preliminary target detection on the current multi-target recognition image based on the background model. If the pixel value of the image pixel coordinate in the multi-target recognition image does not match all Gaussian distributions in the background model, the image pixel coordinate is judged to be a moving target candidate point. Otherwise, it is judged to be a background pixel. All image pixel coordinates in the multi-target recognition image are traversed, and a preliminary foreground mask is generated based on the judgment result. The preliminary foreground mask is then compensated for dynamic background interference to obtain a moving target mask matrix. The background model is constructed as follows: For each image pixel coordinate in the multi-target recognition image, the preset K Gaussian distributions are used to fit the pixel value changes of the image pixel coordinates in the multi-target time series image sequence, and the Gaussian distribution parameters are calculated; The multi-target time-series image sequence is divided into an initialization subsequence and an analysis subsequence. The initialization subsequence is used to initialize the Gaussian distribution parameters. In the analysis subsequence, for the pixel value of the image pixel coordinate in the multi-target recognition image, the matching status of the pixel value and each Gaussian distribution is judged. If the pixel value matches any Gaussian distribution, the Gaussian distribution parameters are updated. If it does not match the Gaussian distribution, only the weights in the Gaussian distribution parameters are updated. Traverse and analyze all multi-target recognition images in the subsequence, sort the Gaussian distributions based on Gaussian distribution parameters, and select a preset number of Gaussian distributions as background models based on the sorting; The method of performing dynamic background interference compensation on the preliminary foreground mask is: The optical flow field of the current multi-target recognition image and the adjacent multi-target recognition images is calculated to obtain the optical flow field of the multi-target recognition image. The affine transformation matrix is ​​estimated from the optical flow field using the random sampling consistency algorithm. The obtained affine transformation matrix is ​​used to perform inverse transformation on the background model to obtain the compensated background image. The current multi-target recognition image is differentiated from the compensated background image, and the difference value of each image pixel coordinate is calculated. For the moving target candidate point in the preliminary foreground mask, if the difference value of the image pixel coordinate corresponding to the moving target candidate point is less than the preset difference threshold, the image pixel coordinate is corrected to the background pixel to achieve dynamic background interference compensation for the preliminary foreground mask; The method for obtaining the information list of the moving targets is as follows: The information list of moving targets includes the target category, feature vector, bounding box center coordinates and corresponding multi-target recognition image timestamp of the moving target; The moving target image is identified using a multi-target visual recognition model to obtain the target category and feature vector of the moving target. The image pixel coordinates of the midpoint of the target candidate frame corresponding to the moving target image in the multi-target recognition image are obtained and marked as the center coordinates of the bounding box of the moving target. The specific method of obtaining the feature vector is as follows: The multi-target visual recognition model includes an image processing sub-model and a visual recognition sub-model. The moving target image is input into the image processing sub-model, which outputs a recognizable image. The recognizable image is then input into the visual recognition sub-model. The visual recognition sub-model includes a basic network and a channel-space dual attention module. The channel-space dual attention module includes a channel attention sub-module and a spatial attention sub-module. After the recognizable image is processed by the basic network, a basic feature map is obtained. The channel attention submodule processes the basic feature map, calculates the channel attention weight, and performs weighted processing on the basic feature map to obtain a channel weighted feature map. The spatial attention submodule processes the basic feature map, calculates the spatial attention weight, and performs weighted processing on the channel weighted feature map to obtain an enhanced feature map. The enhanced feature map is subjected to global average pooling and compressed to obtain a feature vector. The space-time graph model is constructed as follows: The moving targets identified in the multi-target time-series image sequence are used as nodes to construct a spatiotemporal graph model. The node attributes of each node contain a list of information about the corresponding moving targets. The list of moving targets includes the target category, feature vector, bounding box center coordinates, and the timestamp of the corresponding multi-target recognition image. Edges are established between adjacent nodes in the time sequence. The attributes of each edge include the edge weight. If the edge weight is less than the preset weight standard, the edge is cleared. The edge weight is calculated as follows: The nodes at both ends of the edge are marked as the front node and the back node according to the time sequence. The Kalman filter algorithm is used to predict the moving target corresponding to the front node. The center coordinates of the predicted bounding box at the next timestamp are obtained. The center coordinates of the predicted bounding box and the center coordinates of the bounding box of the moving target corresponding to the back node are processed to calculate the predicted similarity of the edge. Obtain the feature vectors of the moving targets corresponding to the nodes at both ends of the edge and calculate the cosine similarity between the feature vectors. The calculated cosine similarity is the feature similarity of the edge. Combined with the feature similarity of the edge, data fusion is performed to obtain the edge weight. The virtual sample is obtained as follows: Collect a large number of moving target sample images with occlusion or motion blur and the corresponding normal sample images, organize them to obtain a diffusion training set, build a conditional training model and use the diffusion training set for training; The feature vector of the moving target corresponding to the fracture node is obtained, and the center coordinates of the predicted bounding box corresponding to the moving target at the current timestamp are obtained. The feature vector and the center coordinates of the predicted bounding box are used as conditional inputs to the trained conditional diffusion model for conditional diffusion to generate virtual samples.

[0009] A multi-target visual recognition system in dynamic scenes, including the following modules: Acquisition module: collects multi-target recognition images in real time in dynamic scenes and organizes them into multi-target time-series image sequences; Background separation module: Based on the multi-target time-series image sequence, it builds a background model for multi-target recognition images, combines dynamic background interference compensation based on optical flow field calculation, and generates a moving target mask matrix; Target recognition module: Generates target candidate frames based on the moving target mask matrix, captures the image within the target candidate frame, marks it as a moving target image, builds and trains a multi-target visual recognition model that introduces a channel-space dual-attention module, recognizes the moving target image, and generates a list of moving target information in the moving target image based on the recognition results; Association recognition module: Using the moving targets identified in the multi-target time-series image sequence as nodes, based on the information list of the moving targets, a spatiotemporal graph model containing nodes and edges is constructed to realize the association of time-series moving targets and determine whether broken nodes appear. If so, a conditional diffusion model is constructed to generate virtual samples to match the broken nodes with connected nodes.

[0010] The beneficial effects of the present invention are as follows: 1. The present invention collects multi-target recognition images in real time and organizes them into time series, combines them with optical flow field calculation to compensate for dynamic background interference, and can accurately generate a moving target mask matrix, effectively eliminate background interference, and improve target detection accuracy. The mask matrix is ​​then used to generate target candidate frames, laying the foundation for subsequent accurate recognition of moving targets, making the recognition process more targeted, and greatly improving the efficiency and accuracy of multi-target recognition in complex dynamic scenes.

[0011] 2. The present invention constructs and trains a multi-target visual recognition model that introduces a channel-space dual-attention module, which can enhance the model's ability to capture key features, further optimize the recognition effect, and generate an accurate list of moving target information. At the same time, it constructs a spatiotemporal graph model to realize the association of temporal moving targets, which can clearly present the target motion trajectory. When a broken node appears, a virtual sample is generated through the conditional diffusion model to match the connected nodes, ensuring the integrity of the target association and providing reliable guarantee for the continuous and stable recognition of multiple targets in dynamic scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The present invention will be further described below with reference to the accompanying drawings.

[0013] Figure 1 This is a flowchart of the steps of a multi-target visual recognition method in a dynamic scene according to an embodiment of the present invention; Figure 2 This is a module architecture diagram of a multi-target visual recognition system in a dynamic scene described in an embodiment of the present invention. DETAILED DESCRIPTION

[0014] In order to make the technical means, creative features, objectives and effects achieved by the present invention easier to understand, the present invention is further described below in conjunction with specific implementation methods.

[0015] Example 1 See also Figure 1 As shown, a multi-target visual recognition method in a dynamic scene according to an embodiment of the present invention includes the following steps: S1: Real-time acquisition of multi-target recognition images in dynamic scenes, and organization of them into multi-target time-series image sequences; An industrial-grade high-speed camera was selected and its core parameters, including resolution, pixel size, frame rate, and exposure time, were set. Three-dimensional calibration of the camera was performed using a checkerboard calibration method. Several calibration plate images in different poses were collected. The pixel coordinates of each pixel in the calibration plate images were obtained. The data was processed based on the corresponding world coordinate system coordinates. The intrinsic parameter matrix of the industrial-grade high-speed camera was calculated. The distortion coefficient was then obtained by minimizing the reprojection error based on the intrinsic parameter matrix. It should be noted that the intrinsic parameter matrix and distortion coefficient are inherent parameters of the camera and belong to the intrinsic properties of the camera. The intrinsic parameter matrix describes the characteristics of the camera optical system, including the focal length and the coordinates of the image principal point. The distortion coefficient reflects the optical distortion characteristics of the camera lens, including the radial distortion coefficient and the tangential distortion coefficient, which are caused by hardware factors such as the manufacturing process of the lens. In dynamic scenes, industrial-grade high-speed cameras are used to capture multi-target recognition images in real time. The captured raw image data is transmitted to an industrial computer via a Gigabit Ethernet interface. A frame synchronization trigger mechanism is used to ensure the stability of the time interval between adjacent frames. The collected raw image data is preprocessed in real time. Specifically, the distortion coefficient is used to correct the distortion of the raw image data, and the distorted image pixel coordinates in the raw image data are corrected to the ideal undistorted image pixel coordinates. Bilateral filtering is used to remove salt and pepper noise in the raw image data. Adaptive histogram equalization is used to enhance the grayscale difference between multiple targets and the background, avoiding the loss of multiple targets caused by uneven lighting. Mark the raw image data that has been pre-processed in real time as a multi-target recognition image, set a tracking and recognition period with the current timestamp as the end point, and arrange the multi-target recognition images within the tracking and recognition period in ascending order based on the acquisition timestamp to form a multi-target time series image sequence S; ; Among them, T represents the total number of timestamps in the tracking and identification period. Since the current timestamp is the end of the tracking and identification period, T also represents the current timestamp, and t represents the timestamp in the tracking and identification period. , corresponding to the actual time , Indicates the time interval between adjacent frames, represents the multi-target recognition image corresponding to the timestamp t within the tracking and recognition period, Indicates the multi-target recognition image corresponding to the current timestamp; It should be noted that the purpose of this step is to complete image acquisition and preprocessing, providing a high-quality multi-target time-series image sequence for subsequent recognition. The intrinsic parameter matrix and distortion coefficient are obtained through camera calibration, image distortion is corrected, and image quality is improved through preprocessing such as filtering and enhancement. Finally, a multi-target time-series image sequence sorted by time is formed, eliminating image distortion caused by lens hardware defects, reducing the impact of noise and uneven lighting, ensuring that the target information in the image is accurate and complete, and laying a reliable data foundation for subsequent target detection and recognition. S2: Based on the multi-target time-series image sequence, a background model of the multi-target recognition image is constructed, and the moving target mask matrix is ​​generated by combining the dynamic background interference compensation based on the optical flow field calculation; Based on the multi-target time-series image sequence, the improved Gaussian mixture model (GMM) is used to construct the background model of the multi-target recognition image; Specifically, for multi-target recognition images For each image pixel coordinate (x, y) in , K=5 Gaussian distributions are used to fit the pixel value changes of the image pixel coordinates in the multi-target time series image sequence. The Gaussian distribution parameters are: ; in, is the weight of the k-th Gaussian distribution, is the mean of the k-th Gaussian distribution at the image pixel coordinate (x, y), is the variance of the kth Gaussian distribution at the image pixel coordinate (x, y); Taking the starting point of the multi-target time series image sequence as the starting point, an initialization subsequence is intercepted in the multi-target time series image sequence, and the part of the multi-target time series image sequence that does not include the initialization subsequence is marked as the analysis subsequence. The multi-target recognition image in the initialization subsequence is used to initialize the Gaussian distribution parameters. In the analysis subsequence, for the multi-target recognition image The pixel value of the image pixel coordinate (x, y) , calculate the pixel value The degree of matching with each Gaussian distribution is if: ; The pixel value is considered to match the kth Gaussian distribution, where Represents the standard deviation of the k-th Gaussian distribution at the image pixel coordinate (x, y); If the pixel value matches the kth Gaussian distribution, the Gaussian distribution parameters are updated: ; ; ; in, is the learning rate, is the weight update coefficient; If the pixel value matches the kth Gaussian distribution, update the weight to: ; Traverse and analyze all multi-target recognition images in the subsequence, and distribute all Gaussian distributions according to Sort in descending order, and select the first m Gaussian distributions whose weights are greater than the preset weight standard as the background model; Multi-target recognition image corresponding to the current timestamp based on background model Perform preliminary target detection. If multiple target recognition images are The pixel value of the image pixel coordinate (x, y) If it does not match all Gaussian distributions in the background model, the image pixel coordinates are judged to be the candidate point of the moving target, and the Otherwise, the image pixel coordinates are judged to be background pixels, and the ; Traversing multi-target recognition images All image pixel coordinates in , get the preliminary foreground mask; It should be noted that the preliminary foreground mask may contain dynamic background interference, so it is necessary to eliminate the dynamic background interference; The pyramid Lucas-Kanade optical flow algorithm is used to identify multiple targets corresponding to the current timestamp. and adjacent multi-target recognition images Perform optical flow calculation to obtain the optical flow field of the multi-target recognition image, estimate the affine transformation matrix M from the optical flow field through the random sampling consistency algorithm, and use the obtained affine transformation matrix M to perform inverse transformation on the background model to obtain the compensated background image ; Multi-target recognition images With the compensated background image Perform differential calculations to determine the difference between the pixel coordinates of each image. ; ; Based on the calculated difference value, the preliminary foreground mask is dynamically compensated for background interference. If the difference value of the image pixel coordinates is less than the preset difference threshold, the image pixel coordinates are corrected to the background pixel. ; Traverse all image pixel coordinates to perform dynamic background interference compensation on the preliminary foreground mask to obtain the moving target mask matrix ; ; in, Indicates the resolution of the multi-target recognition image; It should be noted that the purpose of this step is to build a background model and separate the moving target from the dynamic background. The improved Gaussian mixture model is used to fit the pixel value changes to generate a preliminary foreground mask, which is then combined with the optical flow field and affine transformation matrix for motion compensation to obtain an accurate moving target mask matrix, effectively distinguishing the moving target from the dynamic background, reducing the interference of the dynamic background on target detection, and improving the accuracy and robustness of moving target detection. K=5 Gaussian distributions are used to build the background model, and the model is processed in stages through initialization subsequences and analysis subsequences to better adapt to background changes. The optical flow field calculation is combined with the affine transformation matrix estimation for motion compensation, which specifically eliminates dynamic background interference and improves the accuracy of the foreground mask. S3: Generate target candidate frames based on the moving target mask matrix, capture the image within the target candidate frame, mark it as a moving target image, build and train a multi-target visual recognition model that introduces a channel-space dual-attention module, recognize the moving target image, and generate a list of moving target information in the moving target image based on the recognition results; Based on the obtained moving target mask matrix, an improved Selective Search algorithm is used to generate several target candidate boxes. The target candidate boxes represent the circumscribed rectangle of each moving target in the multi-target recognition image. The image pixel coordinates of the midpoint of the target candidate box in the multi-target recognition image are obtained and marked as the center coordinates of the bounding box of the moving target. In the multi-target recognition image, the image within the target candidate box is intercepted and marked as the moving target image. A channel-space dual attention module is introduced to build and train a multi-target visual recognition model to identify moving target images. The multi-target visual recognition model includes an image processing sub-model and a visual recognition sub-model. Specifically, the image processing sub-model includes an input layer, a preprocessing layer, and an output layer. The input layer receives an image. In the preprocessing layer, the received image is size-normalized using bicubic interpolation, and data enhancement is performed, including random horizontal flipping, brightness adjustment, and contrast adjustment. The size-normalized and data-augmented image is normalized to normalize the pixel values ​​of the three RGB channels of the image to the range of [-1, 1], and a recognizable image is output. The input of the visual recognition sub-model is a recognizable image, and the output is a feature vector and target category. ResNet-50 is selected as the base network. The base network contains 5 convolutional blocks. Residual unit connections are used to solve the gradient vanishing problem during deep network training. The recognizable image is convolved through the base network to obtain the basic feature map. A channel-spatial dual attention module is added after the last convolutional block of the base network. The channel-spatial dual attention module includes a channel attention submodule and a spatial attention submodule. The channel attention submodule performs a global average pooling operation on the basic feature map to obtain a channel description vector, which is then input into a neural network consisting of two fully connected layers. The first fully connected layer uses the ReLU activation function, and the output dimension is C / r, where C represents the number of channels and r represents the compression ratio. The second fully connected layer restores the dimension to C and uses the sigmoid activation function to obtain the channel attention weight. The basic feature map is weighted based on the channel attention weight to obtain a channel weighted feature map. The spatial attention submodule performs maximum pooling and average pooling on the basic feature map along the channel dimension to obtain two feature maps. The two feature maps are spliced ​​according to the channel dimension and input into the convolution layer for convolution operation. The number of channels after convolution is 1. After the sigmoid activation function, the spatial attention weight is obtained. The channel-weighted feature map is weighted based on the spatial attention weight to obtain the enhanced feature map. It should be noted that the role of the enhanced feature map is to enhance the key features of the target and suppress background interference; Perform global average pooling on the enhanced feature map and compress the enhanced feature map into a feature vector; A large number of multi-target sample images of labeled target categories are collected in actual dynamic scenes, covering scenes with different weather, different lighting, different time periods and different target states. The multi-target sample set is sorted and input into the image processing sub-model to obtain a recognizable image set of labeled target categories. The recognizable image set of labeled target categories is divided into a training set and a validation set according to the ratio of 8:2. The training set is used to train the visual recognition sub-model. The total number of training rounds is set, and the cross-entropy loss function is used as the loss function. The optimizer adopts stochastic gradient descent. The validation set is used to evaluate the performance of the visual recognition sub-model after each training round. When the accuracy of the validation set no longer improves between consecutive rounds, or the number of training rounds reaches the preset total number of training rounds, the training of the visual recognition sub-model is judged to be complete. The moving target image is input into the multi-target visual recognition model, and the moving target image is output as a recognizable image through the visual recognition sub-model, and the recognizable image is output as a feature vector and target category of the moving target image through the visual recognition sub-model; For each moving target, a list of information including target category, feature vector, bounding box center coordinates and corresponding multi-target recognition image timestamp is generated; It should be noted that the purpose of this step is to generate target candidate frames and complete target recognition, output a list containing target categories, feature vectors and other information, extract target candidate frames based on the moving target mask matrix, and use a multi-target visual recognition model with a channel-space dual-attention module to identify the target image, obtain feature vectors and target categories, accurately locate moving targets and identify their categories. The extracted feature vectors can effectively characterize the target appearance features, provide a reliable basis for subsequent cross-frame target association, improve the accuracy of target recognition and feature representation capabilities, introduce a channel-space dual-attention module, strengthen the key features of the target and suppress background interference, enhance the discriminability of the feature map, and adopt a pre-processing method that combines bicubic interpolation, multi-dimensional data enhancement and normalization to improve the model's adaptability to different scenarios. S4: Using the moving targets identified in the multi-target time-series image sequence as nodes, a spatiotemporal graph model consisting of nodes and edges is constructed to achieve temporal moving target association and determine whether a broken node appears. If so, a conditional diffusion model is constructed to generate virtual samples to match the broken nodes with connected nodes. The moving targets identified in the multi-target time-series image sequence are used as nodes to construct a spatiotemporal graph model. The node attributes of each node contain a list of information about the corresponding moving targets. Edges are established between adjacent nodes in the time sequence. The attributes of each edge include an edge weight. The edge weight is used to measure the possibility that the nodes at both ends of the edge belong to the same moving target. If the edge weight is greater than or equal to the preset weight standard, the nodes at both ends of the edge are judged to belong to the same moving target. Otherwise, the nodes at both ends of the edge are judged not to belong to the same moving target and the edge is cleared. The edge weight is calculated by weighted fusion of the edge prediction similarity and the edge feature similarity; The prediction similarity of the edge is calculated as follows: For the nodes at both ends of the edge, the node with the earlier time sequence is marked as the front node, and the node with the later time sequence is marked as the back node. The Kalman filter algorithm is used to predict the moving target corresponding to the front node, and the center coordinates of the predicted bounding box at the next timestamp are predicted. The Euclidean distance d between the center coordinates of the predicted bounding box and the center coordinates of the bounding box of the moving target corresponding to the back node is calculated. The predicted similarity of the edge is calculated based on the Euclidean distance d. ; ; Represents the preset scale parameter. The smaller the Euclidean distance, the greater the predicted similarity. The edge feature similarity is obtained by obtaining the feature vectors of the moving targets corresponding to the nodes at both ends of the edge and calculating the cosine similarity between the feature vectors. The calculated cosine similarity is the edge feature similarity. Based on the spatiotemporal graph model, the temporal motion target association is realized, the nodes connected by edges are classified into a unified node group, and the node groups are sorted according to the time sequence to obtain the temporal node sequence; If there is no edge connecting the node in the current timestamp to any node in the previous timestamp, mark the node as a node to be analyzed; If there is no edge connecting a node in the previous timestamp with any node in the current timestamp, mark the node as a broken node, obtain the time series node sequence of the broken node, and use the Kalman filter algorithm to predict the moving target corresponding to the time series node sequence to obtain the center coordinates of the predicted bounding box of the moving target corresponding to the current timestamp; The conditional diffusion model is constructed and trained using the feature vector of the moving target and the center coordinates of the bounding box as conditions; Specifically, a large number of sample images of moving targets with occlusion or motion blur are collected in actual dynamic scenes, and normal sample images of the corresponding moving targets without occlusion or motion blur are collected to obtain a diffusion training set. The diffusion training set is used to train the constructed conditional training model, so that the conditional diffusion model learns the noise parameters when the moving target is occluded and motion blurred; Based on any broken node, a conditional diffusion model is used to generate virtual samples to match the connected nodes of the broken node; Specifically, the feature vector of the moving target corresponding to the fracture node is obtained, and the center coordinates of the predicted bounding box corresponding to the moving target at the current timestamp are obtained. The feature vector and the center coordinates of the predicted bounding box are used as conditional inputs into the trained conditional diffusion model for conditional diffusion to generate several virtual samples. The virtual samples include virtual feature vectors and virtual bounding box center coordinates. For any virtual sample, the virtual feature vector and the virtual bounding box center coordinates are combined into a virtual similarity analysis vector. Get all the nodes to be analyzed within the current timestamp. For any node to be analyzed, obtain the feature vector and bounding box center coordinates of the motion target corresponding to the node to be analyzed. Combine the feature vector and bounding box center coordinates into an actual similarity analysis vector. Calculate the cosine similarity between the actual similarity analysis vector and any virtual similarity analysis vector. If the cosine similarity is greater than the preset similarity threshold, determine that the node to be analyzed and the broken node belong to the same motion target, and establish an edge between the node to be analyzed and the broken node to connect them. Traverse all nodes to be analyzed. If no node to be analyzed belongs to the same moving target as the broken node, the moving target is determined to be lost. The time sequence node sequence of the broken node is marked as the target moving trajectory of the corresponding moving target and stored. For the node to be analyzed, traverse all broken nodes. If there is a node to be analyzed and any broken node does not belong to the same moving target, mark the node to be analyzed as the starting node, and determine that the moving target corresponding to the node is the new moving target; It should be noted that the purpose of this step is to construct a spatiotemporal graph model to realize the association of temporal motion targets and the repair of trajectory breaks. A spatiotemporal graph is constructed with motion targets as nodes, and target association is realized through edge weight calculation. For trajectory breaks, the conditional diffusion model is used to generate virtual samples for matching, improve the target motion trajectory, realize the association of the same target between different frames, solve the trajectory breakage problem caused by occlusion, motion blur, etc., ensure the continuity of the target and the integrity of the trajectory, and provide complete trajectory data for subsequent tasks such as target behavior analysis. The edge weight is calculated by weighted fusion of motion prediction similarity and feature similarity, comprehensively considering the target motion continuity and appearance consistency, improving the association accuracy, and using the conditional diffusion model to generate virtual samples for trajectory break repair. Combined with dual matching of virtual and real samples, the adaptability to complex scenes is enhanced. The technical solution of an embodiment of the present invention is: real-time acquisition of multi-target recognition images in dynamic scenes, organizing them into a multi-target time-series image sequence, constructing a background model of the multi-target recognition images based on the multi-target time-series image sequence, combining dynamic background interference compensation based on optical flow field calculation, generating a moving target mask matrix, generating a target candidate frame according to the moving target mask matrix, intercepting the image within the target candidate frame, marking it as a moving target image, constructing and training a multi-target visual recognition model that introduces a channel-space dual attention module, recognizing the moving target image, generating an information list of the moving targets in the moving target image according to the recognition results, using the moving targets identified in the multi-target time-series image sequence as nodes, constructing a spatiotemporal graph model containing nodes and edges, realizing the association of time-series moving targets and judging whether broken nodes appear. If so, constructing a conditional diffusion model to generate virtual samples to match the broken nodes with connected nodes.

[0016] Example 2 like Figure 2 As shown, a multi-target visual recognition system in a dynamic scene according to an embodiment of the present invention includes the following modules: Acquisition module: collects multi-target recognition images in real time in dynamic scenes and organizes them into multi-target time-series image sequences; Background separation module: Based on the multi-target time-series image sequence, it builds a background model for multi-target recognition images, combines dynamic background interference compensation based on optical flow field calculation, and generates a moving target mask matrix; Target recognition module: Generates target candidate frames based on the moving target mask matrix, captures the image within the target candidate frame, marks it as a moving target image, builds and trains a multi-target visual recognition model that introduces a channel-space dual-attention module, recognizes the moving target image, and generates a list of moving target information in the moving target image based on the recognition results; Association recognition module: Using the moving targets identified in the multi-target time-series image sequence as nodes, based on the information list of the moving targets, a spatiotemporal graph model containing nodes and edges is constructed to realize the association of time-series moving targets and determine whether broken nodes appear. If so, a conditional diffusion model is constructed to generate virtual samples to match the broken nodes with connected nodes.

[0017] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the foregoing embodiments. The foregoing embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-target visual recognition method in a dynamic scene, characterized by: include: Collect multi-target recognition images in real time in dynamic scenes and organize them into multi-target time-series image sequences; Based on the multi-target time-series image sequence, a background model of the multi-target recognition image is constructed, and the moving target mask matrix is ​​generated by combining the dynamic background interference compensation based on the optical flow field calculation; Generate target candidate frames based on the moving target mask matrix, capture the image within the target candidate frame, mark it as a moving target image, build and train a multi-target visual recognition model that introduces a channel-space dual-attention module, recognize the moving target image, and generate a list of moving target information in the moving target image based on the recognition results; Taking the moving targets identified in the multi-target time-series image sequence as nodes, a spatiotemporal graph model containing nodes and edges is constructed based on the information list of the moving targets to realize the association of time-series moving targets and determine whether there are broken nodes. If so, a conditional diffusion model is constructed to generate virtual samples to match the connected nodes for the broken nodes.

2. The multi-target visual recognition method in a dynamic scene according to claim 1, characterized in that: The moving target mask matrix is ​​generated as follows: Acquire a multi-target time-series image sequence, and use an improved Gaussian mixture model to construct a background model for the multi-target recognition image. The background model includes a preset number of Gaussian distributions. Based on the background model, preliminary target detection is performed on the current multi-target recognition image. If the pixel value of the image pixel coordinate in the multi-target recognition image does not match all Gaussian distributions in the background model, the image pixel coordinate is judged to be a moving target candidate point, otherwise it is judged to be a background pixel. All image pixel coordinates in the multi-target recognition image are traversed, and a preliminary foreground mask is generated based on the judgment result. The preliminary foreground mask is then compensated for dynamic background interference to obtain a moving target mask matrix.

3. The multi-target visual recognition method in a dynamic scene according to claim 2, characterized in that: The background model is constructed as follows: For each image pixel coordinate in the multi-target recognition image, the preset K Gaussian distributions are used to fit the pixel value changes of the image pixel coordinates in the multi-target time series image sequence, and the Gaussian distribution parameters are calculated; The multi-target time-series image sequence is divided into an initialization subsequence and an analysis subsequence. The initialization subsequence is used to initialize the Gaussian distribution parameters. In the analysis subsequence, for the pixel value of the image pixel coordinate in the multi-target recognition image, the matching status of the pixel value and each Gaussian distribution is judged. If the pixel value matches any Gaussian distribution, the Gaussian distribution parameters are updated. If it does not match the Gaussian distribution, only the weights in the Gaussian distribution parameters are updated. All multi-target recognition images in the subsequence are traversed and analyzed, Gaussian distributions are sorted based on Gaussian distribution parameters, and a preset number of Gaussian distributions are selected as background models based on the sorting.

4. The multi-target visual recognition method in a dynamic scene according to claim 2, characterized in that: The method of performing dynamic background interference compensation on the preliminary foreground mask is: The optical flow field of the current multi-target recognition image and the adjacent multi-target recognition images is calculated to obtain the optical flow field of the multi-target recognition image. The affine transformation matrix is ​​estimated from the optical flow field using the random sampling consistency algorithm. The obtained affine transformation matrix is ​​used to perform inverse transformation on the background model to obtain the compensated background image. The current multi-target recognition image is differentiated from the compensated background image, and the differential value of each image pixel coordinate is calculated. For the moving target candidate point in the preliminary foreground mask, if the differential value of the image pixel coordinate corresponding to the moving target candidate point is less than the preset differential threshold, the image pixel coordinate is corrected to the background pixel to achieve dynamic background interference compensation for the preliminary foreground mask.

5. The multi-target visual recognition method in a dynamic scene according to claim 1, characterized in that: The method for obtaining the information list of the moving target is: The information list of moving targets includes the target category, feature vector, bounding box center coordinates and corresponding multi-target recognition image timestamp of the moving target; The multi-target visual recognition model is used to identify the moving target image, and the target category and feature vector of the moving target are obtained. The image pixel coordinates of the midpoint of the target candidate box corresponding to the moving target image in the multi-target recognition image are obtained and marked as the center coordinates of the bounding box of the moving target.

6. The multi-target visual recognition method in a dynamic scene according to claim 5, characterized in that: The specific method of obtaining the feature vector is: The multi-target visual recognition model includes an image processing sub-model and a visual recognition sub-model. The moving target image is input into the image processing sub-model, which outputs a recognizable image. The recognizable image is then input into the visual recognition sub-model. The visual recognition sub-model includes a basic network and a channel-space dual attention module. The channel-space dual attention module includes a channel attention sub-module and a spatial attention sub-module. After the recognizable image is processed by the basic network, a basic feature map is obtained. The channel attention submodule processes the basic feature map, calculates the channel attention weight, and performs weighted processing on the basic feature map to obtain a channel weighted feature map. The spatial attention submodule processes the basic feature map, calculates the spatial attention weight, and performs weighted processing on the channel weighted feature map to obtain an enhanced feature map. The enhanced feature map is globally average pooled and compressed to obtain a feature vector.

7. The multi-target visual recognition method in a dynamic scene according to claim 1, characterized in that: The spatiotemporal graph model is constructed as follows: A spatiotemporal graph model is constructed using the moving targets identified in a multi-target time-series image sequence as nodes. The node attributes of each node contain an information list of the corresponding moving targets. The information list of the moving targets includes the target category, feature vector, bounding box center coordinates and the corresponding multi-target recognition image timestamp. Edges are established between adjacent nodes in the time sequence. The attributes of each edge include the edge weight. If the edge weight is less than the preset weight standard, the edge will be cleared.

8. The multi-target visual recognition method in a dynamic scene according to claim 7, characterized in that: The edge weight is calculated as follows: The nodes at both ends of the edge are marked as the front node and the back node according to the time sequence. The Kalman filter algorithm is used to predict the moving target corresponding to the front node. The center coordinates of the predicted bounding box at the next timestamp are obtained. The center coordinates of the predicted bounding box and the center coordinates of the bounding box of the moving target corresponding to the back node are processed to calculate the predicted similarity of the edge. The feature vectors of the moving targets corresponding to the nodes at both ends of the edge are obtained, and the cosine similarity between the feature vectors is calculated. The calculated cosine similarity is the feature similarity of the edge. The edge weight is obtained by data fusion combined with the feature similarity of the edge.

9. The multi-target visual recognition method in a dynamic scene according to claim 8, characterized in that: The virtual sample is obtained as follows: Collect a large number of moving target sample images with occlusion or motion blur and the corresponding normal sample images, organize them to obtain a diffusion training set, build a conditional training model and use the diffusion training set for training; The feature vector of the moving target corresponding to the fracture node is obtained, and the center coordinates of the predicted bounding box corresponding to the moving target at the current timestamp are obtained. The feature vector and the center coordinates of the predicted bounding box are used as conditional inputs to the trained conditional diffusion model for conditional diffusion to generate virtual samples.

10. A multi-target visual recognition system in a dynamic scene, characterized by: Includes the following modules: Acquisition module: collects multi-target recognition images in real time in dynamic scenes and organizes them into multi-target time-series image sequences; Background separation module: Based on the multi-target time-series image sequence, it builds a background model for multi-target recognition images, combines dynamic background interference compensation based on optical flow field calculation, and generates a moving target mask matrix; Target recognition module: Generates target candidate frames based on the moving target mask matrix, captures the image within the target candidate frame, marks it as a moving target image, builds and trains a multi-target visual recognition model that introduces a channel-space dual-attention module, recognizes the moving target image, and generates a list of moving target information in the moving target image based on the recognition results; Association recognition module: Using the moving targets identified in the multi-target time-series image sequence as nodes, based on the information list of the moving targets, a spatiotemporal graph model containing nodes and edges is constructed to realize the association of time-series moving targets and determine whether broken nodes appear. If so, a conditional diffusion model is constructed to generate virtual samples to match the broken nodes with connected nodes.

Images