Target tracking method, system, device and storage medium based on pattern recognition

Abstract

This application provides a target tracking method, system, device, and storage medium based on pattern recognition. The method determines multiple motion postures of a target during motion through all edge contour information, performs spatial pattern recognition on each motion posture, and obtains edge shape features for each motion posture. A spectral convolution operation is performed on the tracking spot of the target image between frames in the current image stream to obtain the tracking interference amount at each inter-frame transition of the current image stream. The spectral convolution information of the target's edge contour in the current image stream is then determined through each tracking interference amount and all edge contour information. The edge shape features are edge-coupled with the spectral convolution information to obtain the target's edge coupling features. The target's tracking frame is determined based on the edge coupling features, and the target is tracked based on the tracking frame. The above scheme can improve the credibility of target tracking results.

Description

Technical Field

[0001] The present application relates to the field of image analysis technology, and more specifically, to a target tracking method, system, device and storage medium based on pattern recognition. Background Art

[0002] With the development of computers, target tracking is widely used in video surveillance, autonomous driving, behavior analysis and other fields. The core task of target tracking is to accurately identify and track specified targets in video or image sequences. Traditional target tracking methods mainly rely on the target's color, shape, texture and other features, but these methods are prone to failure in complex backgrounds, occlusions and lighting changes. In order to improve the accuracy and robustness of target tracking, target tracking methods based on deep learning have been proposed in recent years.

[0003] Existing target tracking methods usually rely on a single recognition mode (such as spatial recognition or spectral recognition), which only considers one aspect of the target's characteristics. It is easily interfered with in complex environments and ignores other information that may affect target tracking. The use of a single feature will lead to insufficient robustness of the tracking results and make it difficult to cope with problems such as target deformation and occlusion. The spatial recognition mode may not be effective when dealing with changes in target shape, and the spectral recognition mode is prone to failure when the lighting changes. Combining the spatial recognition mode and the spectral recognition mode to edge couple the target tracking image can ensure both sensitivity to shape features and sensitivity to spectral features during target tracking, thereby improving the credibility of the target tracking results. Therefore, how to combine the spatial recognition mode and the spectral recognition mode to achieve edge coupling of the target tracking image and thus improve the credibility of the target tracking results is a difficult problem faced by the industry. Summary of the Invention

[0004] The present application provides a target tracking method, system, device and storage medium based on pattern recognition, which can combine spatial recognition mode and spectral recognition mode to achieve edge coupling of target tracking images, thereby improving the credibility of target tracking results.

[0005] In a first aspect, the present application provides a target tracking method based on pattern recognition, comprising:

[0006] Collect the tracking image stream of the target at each movement, and then determine the edge contour information of the target in each tracking image stream;

[0007] Determine multiple motion postures of the target during motion through all edge contour information, perform spatial pattern recognition on each motion posture according to the spatial transformation characteristics of the target, and obtain the edge shape characteristics of the target in each motion posture;

[0008] Monitor the current image stream of the target and perform a spectral convolution operation on the tracking spot of the target image between frames in the current image stream based on the inter-frame spectral information of the tracking image stream to obtain the tracking interference amount at each inter-frame transition of the current image stream. Then, determine the spectral convolution information of the target edge contour in the current image stream through each tracking interference amount and all edge contour information;

[0009] Edge coupling is performed on the edge shape feature and the spectral convolution information to obtain an edge coupling feature of the target in the current image stream;

[0010] A tracking frame of the target in the current image stream is determined according to the edge coupling feature, and the target is tracked based on the tracking frame.

[0011] In some embodiments, determining edge contour information of the target in each tracking image stream specifically includes:

[0012] For each tracking image stream, obtaining position locking information within a tracking frame in each tracking image in the tracking image stream;

[0013] Determine, according to each position locking information, a posture feature of the tracking frame in each tracking image in the tracking image stream;

[0014] The edge contour information of the target in the tracking image stream is determined through all the posture features, and then the edge contour information of the target in each tracking image stream is obtained.

[0015] In some embodiments, determining multiple motion postures of a moving target using all edge contour information specifically includes:

[0016] Determine the contour structure features in each edge contour information;

[0017] Map each contour structure feature into a motion contour based on a convolutional neural network;

[0018] determining profile similarity between respective motion profiles;

[0019] All motion contours are merged into multiple motion poses based on the similarity of each contour.

[0020] In some embodiments, performing spatial pattern recognition on each motion posture based on the spatial transformation characteristics of the target to obtain edge shape characteristics of the target in each motion posture specifically includes:

[0021] For each motion posture, the morphological transformation domain of the motion posture is extracted from the spatial transformation features of the target;

[0022] Performing edge detection on the morphological transformation domain to obtain edge information of the motion posture;

[0023] Based on the spatial recognition algorithm, all motion contours in the motion posture are aligned to obtain the posture shape information of the motion posture;

[0024] The edge shape features of the target in the motion posture are determined by the edge information and the posture shape information, and then the edge shape features of the target in each motion posture are obtained.

[0025] In some embodiments, performing a spectral convolution operation on the tracking spot of the target image between frames in the current image stream based on the inter-frame spectral information of the tracking image stream to obtain the tracking interference amount at each inter-frame transition of the current image stream specifically includes:

[0026] For each inter-frame transition of the current image stream, the spectrum variation during the inter-frame transition is obtained from the inter-frame spectrum information of the tracking image stream;

[0027] Obtaining an adjacent target image during inter-frame conversion from an inter-frame target image of a current image stream;

[0028] performing a convolution operation on the tracking light spot of the adjacent target image based on the spectral variation to obtain a convolution light spot;

[0029] The tracking interference amount of the current image stream during the inter-frame transition is determined by the convolution light spot, and then the tracking interference amount of the current image stream during each inter-frame transition is obtained.

[0030] In some embodiments, determining the spectral convolution information of the edge contour of the target in the current image stream by using each tracking interference amount and all edge contour information specifically includes:

[0031] Perform convolution coupling on all edge contour information to obtain the convolution coupling domain of the edge contour in the target;

[0032] Performing interference correction on the inter-frame conversion process of each convolution coupling sub-block in the convolution coupling domain by using each tracking interference amount to obtain a plurality of spectral convolution values;

[0033] The spectral convolution information of the edge contour of the target in the current image stream is determined according to all the spectral convolution values.

[0034] In some embodiments, edge coupling the edge shape feature with the spectral convolution information to obtain the edge coupling feature of the target in the current image stream specifically includes:

[0035] determining an edge difference domain between the edge shape feature and the spectral convolution information;

[0036] performing coupling association on each edge block in the edge difference domain to obtain a plurality of coupling association values;

[0037] The edge coupling characteristics of the target in the current image stream are determined through all coupling correlation values.

[0038] In a second aspect, the present application provides a target tracking system based on pattern recognition, comprising:

[0039] An acquisition module is used to acquire the tracking image stream of the target each time it moves, and then determine the edge contour information of the target in each tracking image stream;

[0040] A processing module is used to determine multiple motion postures of the target during motion based on all edge contour information, perform spatial pattern recognition on each motion posture according to the spatial transformation characteristics of the target, and obtain edge shape characteristics of the target in each motion posture;

[0041] The processing module is further configured to monitor the current image stream of the target, perform a spectral convolution operation on the tracking spot of the target image between frames in the current image stream based on the inter-frame spectral information of the tracking image stream, obtain a tracking interference amount at each inter-frame transition of the current image stream, and then determine the spectral convolution information of the edge contour of the target in the current image stream through each tracking interference amount and all edge contour information;

[0042] The processing module is further configured to perform edge coupling on the edge shape feature and the spectral convolution information to obtain an edge coupling feature of the target in the current image stream;

[0043] An execution module is configured to determine a tracking frame of a target in a current image stream according to the edge coupling feature, and then track the target based on the tracking frame.

[0044] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device executes the above-mentioned pattern recognition-based target tracking method.

[0045] In a fourth aspect, the present application provides a computer-readable storage medium, in which instructions or codes are stored. When the instructions or codes are run on a computer, the computer implements the above-mentioned pattern recognition-based target tracking method when executed.

[0046] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0047] The present application provides a target tracking method, system, device and storage medium based on pattern recognition, in which a tracking image stream of the target is collected each time the target moves, and then the edge contour information of the target in each tracking image stream is determined; multiple motion postures of the target during movement are determined through all the edge contour information, and spatial pattern recognition is performed on each motion posture according to the spatial conversion characteristics of the target to obtain the edge shape characteristics of the target under each motion posture; the current image stream of the target is monitored, and a spectral convolution operation is performed on the tracking spot of the target image between frames in the current image stream based on the inter-frame spectral information of the tracking image stream to obtain the tracking interference amount at each inter-frame conversion of the current image stream, and then the spectral convolution information of the edge contour of the target in the current image stream is determined through each tracking interference amount and all the edge contour information; the edge shape feature is edge-coupled with the spectral convolution information to obtain the edge coupling feature of the target in the current image stream; the tracking frame of the target in the current image stream is determined according to the edge coupling feature, and then the target is tracked based on the tracking frame.

[0048] It can be seen that in this application, the tracking frame of the target in the current image stream can be determined according to the edge coupling feature, and then the target can be tracked based on the tracking frame; first, spatial pattern recognition can more accurately understand the spatial transformation of the target and extract the edge shape features under each motion posture, which can facilitate the subsequent prediction of the target's motion posture and shape, and reduce errors and omissions in target tracking; secondly, the spectral features between each frame in the current image stream can be evaluated through the spectral convolution operation, thereby extracting the tracking interference amount, and combining the tracking interference amount with the edge contour information to accurately calculate the spectral convolution features of the target, thereby improving the accuracy of accurate tracking of the target edge, facilitating the identification and differentiation of targets in complex environments, and reducing tracking errors caused by spectral interference; finally, the edge coupling feature combines the spatial recognition mode and the spectral recognition mode, comprehensively considering the shape features and spectral features of the target, which can improve the comprehensiveness and accuracy of target tracking, reduce the limitations of a single mode, enable the tracking system to better handle changes and interference in complex environments, and improve the credibility of the target tracking results. In summary, based on the above scheme, the spatial recognition mode and the spectral recognition mode can be combined to achieve edge coupling of the target tracking image, thereby improving the credibility of the target tracking results. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0050] Figure 1 is an exemplary flow chart of a target tracking method based on pattern recognition according to some embodiments of the present application;

[0051] Figure 2 is a schematic diagram of a process for determining a tracking interference amount according to some embodiments of the present application;

[0052] Figure 3 is a schematic diagram of a process for determining edge coupling characteristics according to some embodiments of the present application;

[0053] Figure 4 is a schematic diagram of exemplary hardware and / or software of a pattern recognition-based target tracking system according to some embodiments of the present application;

[0054] Figure 5 It is a structural diagram of a computer device for implementing a target tracking method based on pattern recognition according to some embodiments of the present application. DETAILED DESCRIPTION

[0055] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0056] The embodiments of the present application provide a target tracking method, system, device and storage medium based on pattern recognition. The core of the method is to determine multiple motion postures of the target during motion through all edge contour information, perform spatial pattern recognition on each motion posture, and obtain edge shape features under each motion posture; perform spectral convolution operation on the tracking spot of the target image between frames in the current image stream to obtain the tracking interference amount at each frame transition of the current image stream, and then determine the spectral convolution information of the edge contour of the target in the current image stream through each tracking interference amount and all edge contour information; edge-couple the edge shape features with the spectral convolution information to obtain the edge coupling features of the target in the current image stream; determine the tracking frame of the target in the current image stream based on the edge coupling features, and then track the target based on the tracking frame. Based on the above scheme, the credibility of the target tracking results can be improved.

[0057] In order to better understand the above technical solution, the following will be combined with the accompanying drawings and specific implementation methods to describe the above technical solution in detail. Figure 1 , which is an exemplary flow chart of a target tracking method based on pattern recognition according to some embodiments of the present application. The target tracking method based on pattern recognition mainly includes the following steps:

[0058] In step 101, a tracking image stream of the target is collected each time the target moves, and then edge contour information of the target in each tracking image stream is determined.

[0059] It should be noted that, in the present application, the target is a selected tracking target; the tracking image stream is a data stream composed of tracking images of each movement process of the target, wherein each frame of the tracking image contains the position locking information of the tracking frame on the target, the conversion area from the previous frame image to the current frame image and the spectral conversion information between the tracking images between adjacent frames. The tracking frame refers to the rectangular frame used to surround the target in target tracking, and the position locking information is the position of the tracking frame in each frame of the tracking image, that is, the position of the target in each frame of the tracking image. The spectral conversion information is the spectral conversion process information between the current frame tracking image and the previous frame tracking image, which can be collected by existing spectral instruments (for example: hyperspectral imaging equipment); in specific implementation, the tracking image stream of each movement of the target within a specified time period (the default is within the last year) is obtained in the specified target tracking system, wherein the specified target tracking system refers to the target tracking system applied in this scheme.

[0060] In some embodiments, determining edge contour information of a target in each tracking image stream may be achieved by using the following steps:

[0061] For each tracking image stream, obtaining position locking information within a tracking frame in each tracking image in the tracking image stream;

[0062] Determine, according to each position locking information, a posture feature of the tracking frame in each tracking image in the tracking image stream;

[0063] The edge contour information of the target in the tracking image stream is determined through all the posture features, and then the edge contour information of the target in each tracking image stream is obtained.

[0064] It should be noted that, in the present application, edge contour information is a collection of the target's motion shape information and motion direction information in its historical motion; in specific implementation, first, for each tracking image stream, the position lock information within the tracking frame in each tracking image in the tracking image stream is obtained; then, for each tracking image in the tracking image stream, the existing image processing algorithm (for example, the Canny edge detection algorithm) can be used to extract the contour features of the target in the position lock information of the tracking image, and the contour feature can be used as the posture feature of the tracking frame in the tracking image. The posture feature is a feature used to describe the shape information and direction information of the target in the tracking image. Finally, the collection of all posture features can be used as the edge contour information of the tracking image stream, and the edge contour information of the target in each tracking image stream can be obtained by the above method.

[0065] In step 102, multiple motion postures of the target are determined through all edge contour information, and spatial pattern recognition is performed on each motion posture according to the spatial transformation characteristics of the target to obtain the edge shape characteristics of the target in each motion posture.

[0066] In some embodiments, determining multiple motion postures of a moving target using all edge contour information may be achieved by using the following steps:

[0067] Determine the contour structure features in each edge contour information;

[0068] Map each contour structure feature into a motion contour based on a convolutional neural network;

[0069] determining profile similarity between respective motion profiles;

[0070] All motion contours are merged into multiple motion poses based on the similarity of each contour.

[0071] It should be noted that, in this application, motion posture is a feature that reflects the motion form of a target in different time periods; contour similarity indicates the degree of similarity of the contour information of the target during motion. The contour similarity calculation can accurately compare the morphological similarities under different motion contours, which facilitates the subsequent merging of all motion contours to obtain the motion posture; motion contour is used to describe the contour features of a target during motion; contour structural features are parameters that describe the shape and geometric features of a specified target.

[0072] In a specific implementation, first, an existing feature extraction algorithm (e.g., Hu moment algorithm) can be used to extract feature descriptors of each edge contour information as contour structural features in each edge contour information; second, for each contour structural feature, a convolutional neural network model is initialized, and a large amount of labeled contour data is obtained to train the convolutional neural network model. The contour structural feature can be input into the trained convolutional neural network model for contour mapping, and the contour mapping result is used as the motion contour corresponding to the contour structural feature; then, the Hausdorff distance between each motion contour can be calculated using an existing feature matching algorithm as the contour similarity between each motion contour; finally, each contour similarity can be used as a clustering parameter of an existing clustering algorithm (e.g., K-means clustering algorithm), and the clustering algorithm can be used to cluster and merge all motion contours to obtain multiple contour clustering clusters. All motion contours in each contour clustering cluster can be classified into the motion posture of the contour clustering cluster, and all motion postures can be obtained. Similar contours are automatically grouped by the clustering algorithm to improve the efficiency and accuracy of posture classification. The grouped motion postures can accurately reflect the motion pattern of the target in different time periods.

[0073] It should be noted that, in this application, the spatial transformation feature refers to the changing feature of the target's position and posture in space; in specific implementation, for each frame of tracking image, the transformation area in the tracking image is obtained as the morphological transformation domain. The morphological transformation domain of each frame of tracking image can be obtained in the above manner, and the set of all morphological transformation domains can be used as the spatial transformation feature of the target, wherein the morphological transformation domain represents the changing area when the target transforms between different motion postures.

[0074] In some embodiments, spatial pattern recognition is performed on each motion posture based on the spatial transformation characteristics of the target to obtain the edge shape characteristics of the target in each motion posture. The following steps can be used to achieve this:

[0075] For each motion posture, the morphological transformation domain of the motion posture is extracted from the spatial transformation features of the target;

[0076] Performing edge detection on the morphological transformation domain to obtain edge information of the motion posture;

[0077] Based on the spatial recognition algorithm, all motion contours in the motion posture are aligned to obtain the posture shape information of the motion posture;

[0078] The edge shape features of the target in the motion posture are determined by the edge information and the posture shape information, and then the edge shape features of the target in each motion posture are obtained.

[0079] It should be noted that, in this application, edge shape features represent the edge shape description information of the target in a specific motion posture; edge information is the feature information describing the edge of the target in a specific motion posture; posture shape information is the feature information describing the overall shape of the target in a specific motion posture. By integrating all motion contours, complete posture shape information can be obtained, thereby improving the accuracy of the description of each motion posture shape.

[0080] In the specific implementation, first, for each motion posture, the morphological transformation domain of the motion posture is extracted from the spatial transformation features of the target; secondly, a specific edge detection algorithm (for example, the Canny edge detection algorithm) can be used to perform edge detection on the morphological transformation domain, and the edge detection result can be used as the edge information of the motion posture; then, a spatial recognition algorithm (for example, Procrustes analysis) can be used to perform shape alignment on all motion contours in the motion posture, and the result of the shape alignment can be used as the posture shape information of the motion posture; finally, a feature fusion model is initialized, and the posture shape information is used as the basic fusion target of the feature fusion model, and the edge information is used as the edge correction parameter of the feature fusion model in the fusion. The feature fusion model is used to complete the feature fusion of the edge shape of the target in the motion posture, and the result after feature fusion can be used as the edge shape feature of the target in the motion posture. The edge shape features of the target in each motion posture can be obtained in the above manner.

[0081] In step 103, the current image stream of the target is monitored, and a spectral convolution operation is performed on the tracking spot of the target image between frames in the current image stream based on the inter-frame spectral information of the tracking image stream to obtain the tracking interference amount at each inter-frame transition of the current image stream, and then the spectral convolution information of the edge contour of the target in the current image stream is determined through each tracking interference amount and all edge contour information.

[0082] It should be noted that, in the present application, the current image stream is the data stream of each frame of tracking image of the target under current tracking in the specified target tracking system; in specific implementation, the tracking image stream of the target under current tracking can be monitored in the specified target tracking system as the current image stream.

[0083] In some embodiments, based on the inter-frame spectral information of the tracking image stream, a spectral convolution operation is performed on the tracking spot of the target image between frames in the current image stream to obtain the tracking interference amount at each inter-frame transition of the current image stream. Figure 2 As described above, the figure is a schematic diagram of a process for determining the tracking interference amount in some embodiments of the present application. In this embodiment, determining the tracking interference amount can be implemented by using the following steps:

[0084] In step 1031, for each inter-frame transition of the current image stream, the spectrum variation during the inter-frame transition is obtained from the inter-frame spectrum information of the tracking image stream;

[0085] In step 1032, an adjacent target image during inter-frame conversion is obtained from the inter-frame target image of the current image stream;

[0086] In step 1033, a convolution operation is performed on the tracking light spot of the adjacent target image based on the spectral variation to obtain a convolved light spot;

[0087] In step 1034, the tracking interference amount of the current image stream during the inter-frame transition is determined by the convolution spot, thereby obtaining the tracking interference amount of the current image stream during each inter-frame transition.

[0088] It should be noted that, in this application, the tracking interference amount represents the degree of interference to the current image stream during inter-frame conversion; the spectral change amount is a quantitative indicator used to measure the change in the target spectral characteristics between adjacent frames in the current image stream; the inter-frame spectral information is the light reflection or absorption characteristics of the target at a specific wavelength, and the spectral conversion information between adjacent tracking images can be used as inter-frame spectral information; the convolution spot refers to the interference spot used to describe the convolution image conversion between adjacent frames.

[0089] In the specific implementation, first, for each inter-frame conversion of the current image stream, the spectral conversion information at the time of inter-frame conversion is obtained from the inter-frame spectral information of the tracking image stream, and the difference between the spectra before and after the conversion in the spectral conversion information is used as the spectral change amount at the time of inter-frame conversion. Secondly, the adjacent target image at the time of inter-frame conversion is obtained in the inter-frame target image of the current image stream; then, a convolution model based on a neural network is initialized, the spectral change amount is used as the convolution kernel of the convolution model, and the tracking spot of the adjacent target image is used as the convolution target of the convolution model. The convolution operation is completed using the convolution model, and the result of the convolution operation can be used as the convolution spot; finally, the number of pixels that differ between the convolution spot and the tracking spot can be counted as the tracking interference amount of the current image stream at the time of inter-frame conversion. The tracking interference amount at each inter-frame conversion of the current image stream can be obtained in the above manner.

[0090] In some embodiments, determining the spectral convolution information of the edge contour of the target in the current image stream by using each tracking interference amount and all edge contour information can be achieved by using the following steps:

[0091] Perform convolution coupling on all edge contour information to obtain the convolution coupling domain of the edge contour in the target;

[0092] Performing interference correction on the inter-frame conversion process of each convolution coupling sub-block in the convolution coupling domain by using each tracking interference amount to obtain a plurality of spectral convolution values;

[0093] The spectral convolution information of the edge contour of the target in the current image stream is determined according to all the spectral convolution values.

[0094] It should be noted that in this application, the spectral convolution information represents the comprehensive spectral feature information of the target edge contour. The use of this spectral convolution information can improve the accuracy and robustness of target recognition, thereby forming a complete spectral convolution information map, and further improving the subsequent tracking accuracy and tracking robustness of the target; the spectral convolution value is a parameter that reflects the spectral characteristics of the target; the convolution coupling domain is a data domain composed of the edge spectral information of multiple frames of images after convolution coupling processing.

[0095] In the specific implementation, first, for each edge contour information, the edge contour information can be converted into a two-dimensional contour matrix, and a part of the area (the default is 3x3 matrix) is randomly selected in the two-dimensional contour matrix as the local contour feature, and a convolution kernel (the default is 3x3 matrix) is selected. The convolution kernel is slid n times on the two-dimensional contour matrix, and the sub-block of the convolution kernel in the two-dimensional contour matrix after each sliding can be used as the convolution sub-block, and the convolution value after each sliding is calculated as the convolution value of each convolution sub-block corresponding to the edge contour information. The convolution value of each convolution sub-block corresponding to each edge contour information can be obtained in the above manner. For each convolution sub-block, the value after accumulating the convolution values ​​of the convolution sub-blocks corresponding to each edge contour information is used as the convolution coupling value of the convolution sub-block, and the accumulated value can be used as the convolution coupling value of the convolution sub-block. The convolution sub-block containing the convolution coupling value after addition is used as the convolution coupling sub-block corresponding to the convolution sub-block. The convolution coupling sub-block corresponding to each convolution sub-block can be obtained in the above manner, and each convolution coupling sub-block can be arranged according to its position in the corresponding two-dimensional contour matrix as the convolution coupling domain of the edge contour in the target; then, for each convolution coupling sub-block in the convolution coupling domain, the product of the mean value of all tracking interference amounts and the convolution value of the convolution coupling sub-block can be used as the denoised convolution value of the convolution coupling sub-block. The denoised convolution value of each convolution coupling sub-block in the convolution coupling domain can be obtained in the above manner, and all denoised convolution values ​​can be deduplicated and used as multiple spectral convolution values; finally, the set of all spectral convolution values ​​can be used as the spectral convolution information of the edge contour of the target in the current image stream.

[0096] In step 104, edge coupling is performed on the edge shape feature and the spectral convolution information to obtain edge coupling features of the target in the current image stream.

[0097] In some embodiments, the edge shape feature is edge-coupled with the spectral convolution information to obtain the edge coupling feature of the target in the current image stream, referring to Figure 3 As described above, this figure is a schematic diagram of the process of determining edge coupling characteristics in some embodiments of the present application. In this embodiment, determining edge coupling characteristics can be achieved by using the following steps:

[0098] In step 1041, an edge difference domain between the edge shape feature and the spectral convolution information is determined;

[0099] In step 1042, coupling association is performed on each edge block in the edge difference domain to obtain a plurality of coupling association values;

[0100] In step 1043 , the edge coupling characteristics of the target in the current image stream are determined through all coupling correlation values.

[0101] It should be noted that, in this application, the edge coupling feature is a feature that reflects the complete edge information of the target; the edge difference domain refers to the difference area between the edge shape feature and the spectral convolution information; the coupling correlation value represents the numerical value of the degree of correlation between edge blocks.

[0102] In specific implementation, first, each convolution coupling sub-block can be regarded as an edge block, and the Euclidean distance between the edge coupling feature and each spectral convolution value in the spectral convolution information can be calculated as the edge difference value of the corresponding edge block, and the set of all edge difference values ​​can be regarded as the edge difference domain; then, for each edge block, the edge difference value corresponding to the edge block in the edge difference domain and the average of the edge difference values ​​corresponding to all adjacent edge blocks can be used as the coupling value of the edge block. In the above manner, the coupling value of each edge block in the edge difference domain can be obtained, and all coupling values ​​can be removed and used as the coupling association value; finally, the set of all coupling association values ​​can be used as the edge coupling feature.

[0103] In step 105, a tracking frame of the target in the current image stream is determined according to the edge coupling feature, and the target is tracked based on the tracking frame.

[0104] In some embodiments, determining the tracking frame of the target in the current image stream according to the edge coupling feature can be implemented by the following steps:

[0105] Initialize a tracking frame based on the first frame target image of the target;

[0106] Determine edge information of the target in the tracking frame by using the edge coupling feature;

[0107] A tracking frame of the target in the current image stream is determined according to the edge information and the tracking frame.

[0108] In the specific implementation, first, the first frame image in the current image stream is selected, the position and size of the target are automatically marked, and a rectangular frame is initialized with the position and size as the initial tracking frame; then, the existing edge detection algorithm (for example, Canny edge detection) can be used to combine the edge coupling features in the tracking frame to obtain the accurate target boundary as the edge information of the target in the tracking frame; finally, in the tracking frame of the current frame, the edge information is used as the edge contour of the target, and the position and size of the tracking frame are adjusted so that it accurately surrounds the edge contour of the target. The above steps are repeated to process each frame of the target image in the current image stream to form a tracking frame composed of continuous edge contours.

[0109] In some embodiments, tracking the target based on the tracking frame is to use the tracking frame as a tracking result of the target in the current image stream.

[0110] In the present application, the tracking frame of the target in the current image stream can be determined according to the edge coupling feature, and then the target can be tracked based on the tracking frame; first, spatial pattern recognition can more accurately understand the spatial transformation of the target and extract the edge shape features under each motion posture, which can facilitate the subsequent prediction of the target's motion posture and shape, reducing errors and omissions in target tracking; second, through the spectral convolution operation, the spectral features between each frame in the current image stream can be evaluated, thereby extracting the tracking interference amount. Combining the tracking interference amount with the edge contour information can accurately calculate the spectral convolution features of the target, thereby improving the accurate tracking reliability of the target edge, facilitating the identification and differentiation of targets in complex environments, and reducing tracking errors caused by spectral interference; finally, the edge coupling feature combines the spatial recognition mode and the spectral recognition mode, comprehensively considering the shape and spectral features of the target, which can improve the comprehensiveness and accuracy of target tracking, reduce the limitations of a single mode, enable the tracking system to better handle changes and interference in complex environments, and improve the credibility of the target tracking results; in summary, based on the above scheme, the spatial recognition mode and the spectral recognition mode can be combined to achieve edge coupling of the target tracking image, thereby improving the credibility of the target tracking results, which is a difficult problem faced by the industry.

[0111] In addition, in another aspect of the present application, in some embodiments, the present application provides a target tracking system based on pattern recognition, referring to Figure 4 , which is a schematic diagram of exemplary hardware and / or software of a pattern recognition-based target tracking system according to some embodiments of the present application, including: an acquisition module 201, a processing module 202, and an execution module 203, which are described as follows:

[0112] The acquisition module 201 in this application is mainly used to acquire the tracking image stream of the target during each movement, and then determine the edge contour information of the target in each tracking image stream;

[0113] Processing module 202, in this application, is used to determine multiple motion postures of the target during motion based on all edge contour information, perform spatial pattern recognition on each motion posture based on the spatial transformation characteristics of the target, and obtain edge shape characteristics of the target in each motion posture;

[0114] It should be noted that the processing module 202 is also used to monitor the current image stream of the target, perform a spectral convolution operation on the tracking spot of the target image between frames in the current image stream based on the inter-frame spectral information of the tracking image stream, obtain the tracking interference amount at each inter-frame transition of the current image stream, and then determine the spectral convolution information of the edge contour of the target in the current image stream through each tracking interference amount and all edge contour information;

[0115] In addition, the processing module 202 is further configured to perform edge coupling on the edge shape feature and the spectral convolution information to obtain an edge coupling feature of the target in the current image stream;

[0116] The execution module 203 in this application is mainly used to determine the tracking frame of the target in the current image stream according to the edge coupling feature, and then track the target based on the tracking frame.

[0117] The above describes in detail examples of target tracking methods, systems, devices and storage media based on pattern recognition provided by the embodiments of the present application. It can be understood that, in order to implement the above functions, the corresponding devices include hardware structures and / or software modules that perform the corresponding functions. Those skilled in the art should easily appreciate that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0118] In some embodiments, the present application also provides a computer device, which includes a memory and a processor, the memory being used to store a computer program, and the processor being used to call and run the computer program from the memory, so that the computer device executes the above-mentioned pattern recognition-based target tracking method.

[0119] In some embodiments, reference Figure 5The dotted line in the figure indicates that the unit or module is optional. The figure is a schematic diagram of the structure of a computer device for implementing a target tracking method based on pattern recognition according to an embodiment of the present application. The target tracking method based on pattern recognition described in the above embodiment can be Figure 5 The computer device shown in the figure is implemented, and the computer device includes at least one processor 301, a memory 302 and at least one communication unit 305. The computer device can be a terminal device, a server or a chip.

[0120] The processor 301 may be a general-purpose processor or a dedicated processor. For example, the processor 301 may be a central processing unit (CPU), which may be used to control the computer device, execute software programs, and process data from the software programs. The computer device may also include a communication unit 305 for inputting (receiving) and outputting (transmitting) signals.

[0121] For example, the computer device may be a chip, the communication unit 305 may be an input and / or output circuit of the chip, or the communication unit 305 may be a communication interface of the chip, and the chip may be a component of a terminal device, a network device, or other device.

[0122] For another example, the computer device may be a terminal device or a server, and the communication unit 305 may be a transceiver of the terminal device or the server, or the communication unit 305 may be a transceiver circuit of the terminal device or the server.

[0123] The computer device may include one or more memories 302, on which a program 304 is stored. The program 304 can be executed by the processor 301 to generate instructions 303, so that the processor 301 executes the method described in the above method embodiment according to the instructions 303. Optionally, data (such as a target audit model) can also be stored in the memory 302. Optionally, the processor 301 can also read data stored in the memory 302. The data can be stored at the same storage address as the program 304, or at a different storage address from the program 304.

[0124] The processor 301 and the memory 302 may be provided separately or integrated together, for example, integrated on a system on chip (SOC) of a terminal device.

[0125] It should be understood that each step of the above method embodiment can be completed by a hardware-based logic circuit or software-based instructions in the processor 301. The processor 301 can be a CPU, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0126] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0127] For example, in some embodiments, the present application also provides a computer-readable storage medium, in which instructions or codes are stored. When the instructions or codes are run on a computer, the computer implements the above-mentioned pattern recognition-based target tracking method when executing.

[0128] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0129] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A target tracking method based on pattern recognition, characterized in that: The steps include: Collect the tracking image stream of the target at each movement, and then determine the edge contour information of the target in each tracking image stream; Determine multiple motion postures of the target during motion through all edge contour information, perform spatial pattern recognition on each motion posture according to the spatial transformation characteristics of the target, and obtain the edge shape characteristics of the target in each motion posture; Monitor the current image stream of the target and perform a spectral convolution operation on the tracking spot of the target image between frames in the current image stream based on the inter-frame spectral information of the tracking image stream to obtain the tracking interference amount at each inter-frame transition of the current image stream. Then, determine the spectral convolution information of the target edge contour in the current image stream through each tracking interference amount and all edge contour information; Edge coupling is performed on the edge shape feature and the spectral convolution information to obtain an edge coupling feature of the target in the current image stream; determining a tracking frame of the target in the current image stream according to the edge coupling feature, and then tracking the target based on the tracking frame; The spectral convolution operation is performed on the tracking spot of the target image between frames in the current image stream based on the inter-frame spectral information of the tracking image stream to obtain the tracking interference amount at each inter-frame transition of the current image stream. Specifically, the following is performed: For each inter-frame transition of the current image stream, the spectrum variation during the inter-frame transition is obtained from the inter-frame spectrum information of the tracking image stream; Obtaining an adjacent target image during inter-frame conversion from an inter-frame target image of a current image stream; performing a convolution operation on the tracking light spot of the adjacent target image based on the spectral variation to obtain a convolution light spot; Determining the tracking interference amount of the current image stream during inter-frame transitions by using the convolution light spot, and then obtaining the tracking interference amount of the current image stream during each inter-frame transition; The step of edge coupling the edge shape feature with the spectral convolution information to obtain the edge coupling feature of the target in the current image stream specifically includes: determining an edge difference domain between the edge shape feature and the spectral convolution information; performing coupling association on each edge block in the edge difference domain to obtain a plurality of coupling association values; The edge coupling characteristics of the target in the current image stream are determined through all coupling correlation values.

2. The method according to claim 1, wherein Determining the edge contour information of the target in each tracking image stream specifically includes: For each tracking image stream, obtaining position locking information within a tracking frame in each tracking image in the tracking image stream; Determine, according to each position locking information, a posture feature of the tracking frame in each tracking image in the tracking image stream; The edge contour information of the target in the tracking image stream is determined through all the posture features, and then the edge contour information of the target in each tracking image stream is obtained.

3. The method according to claim 1, wherein The multiple motion postures of the target during motion are determined by all edge contour information, including: Determine the contour structure features in each edge contour information; Map each contour structure feature into a motion contour based on a convolutional neural network; determining profile similarity between respective motion profiles; All motion contours are merged into multiple motion poses based on the similarity of each contour.

4. The method according to claim 1, wherein Based on the spatial transformation characteristics of the target, spatial pattern recognition is performed on each motion posture to obtain the edge shape characteristics of the target in each motion posture, including: For each motion posture, the morphological transformation domain of the motion posture is extracted from the spatial transformation features of the target; Performing edge detection on the morphological transformation domain to obtain edge information of the motion posture; Based on the spatial recognition algorithm, all motion contours in the motion posture are aligned to obtain the posture shape information of the motion posture; The edge shape features of the target in the motion posture are determined by the edge information and the posture shape information, and then the edge shape features of the target in each motion posture are obtained.

5. The method according to claim 1, wherein The spectral convolution information of the target edge contour in the current image stream is determined by using each tracking interference amount and all edge contour information, specifically including: Perform convolution coupling on all edge contour information to obtain the convolution coupling domain of the edge contour in the target; Performing interference correction on the inter-frame conversion process of each convolution coupling sub-block in the convolution coupling domain by using each tracking interference amount to obtain a plurality of spectral convolution values; The spectral convolution information of the edge contour of the target in the current image stream is determined according to all the spectral convolution values.

6. A target tracking system based on pattern recognition, which uses the method according to any one of claims 1 to 5 to perform target tracking, characterized in that: The system includes: An acquisition module is used to acquire the tracking image stream of the target each time it moves, and then determine the edge contour information of the target in each tracking image stream; A processing module is used to determine multiple motion postures of the target during motion based on all edge contour information, perform spatial pattern recognition on each motion posture according to the spatial transformation characteristics of the target, and obtain edge shape characteristics of the target in each motion posture; The processing module is further configured to monitor the current image stream of the target, perform a spectral convolution operation on the tracking spot of the target image between frames in the current image stream based on the inter-frame spectral information of the tracking image stream, obtain a tracking interference amount at each inter-frame transition of the current image stream, and then determine the spectral convolution information of the edge contour of the target in the current image stream through each tracking interference amount and all edge contour information; The processing module is further configured to perform edge coupling on the edge shape feature and the spectral convolution information to obtain an edge coupling feature of the target in the current image stream; An execution module is configured to determine a tracking frame of a target in a current image stream according to the edge coupling feature, and then track the target based on the tracking frame.

7. A computer device, characterized in that: The computer device includes a memory and a processor, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device executes the target tracking method based on pattern recognition according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions or codes, and when the instructions or codes are executed on a computer, the computer implements the target tracking method based on pattern recognition according to any one of claims 1 to 5.

Images