Target tracking method, system and device based on pattern recognition and storage medium

Abstract

The invention provides a target tracking method, system and device based on pattern recognition and a storage medium, and the method comprises the steps: determining a plurality of motion postures of a target during motion through all edge contour information, carrying out the spatial pattern recognition of each motion posture, and obtaining the edge shape features under each motion posture; performing spectrum convolution operation on the tracking light spot of the inter-frame target image in the current image flow to obtain the tracking interference variable of the current image flow during each inter-frame conversion, and further determining the spectrum convolution information of the edge contour of the target in the current image flow through each tracking interference variable and all the edge contour information; performing edge coupling on the edge shape features and the spectrum convolution information to obtain edge coupling features of the target; and determining a tracking frame of the target according to the edge coupling features, and tracking the target based on the tracking frame. Based on the scheme, the credibility of the target tracking result can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of image analysis, and more specifically, to a target tracking method, system, device and storage medium based on pattern recognition. Background Art

[0002] With the development of computers, target tracking is widely used in video surveillance, autonomous driving, behavior analysis and other fields. The core task of target tracking is to accurately identify and track specified targets in video or image sequences. Traditional target tracking methods mainly rely on the target's color, shape, texture and other features, but these methods are prone to failure in complex backgrounds, occlusions, and lighting changes. In order to improve the accuracy and robustness of target tracking, a target tracking method based on deep learning has been proposed in recent years.

[0003] Existing target tracking methods usually rely on a single recognition mode (such as spatial recognition or spectral recognition), only consider one aspect of the target's characteristics, are easily disturbed in complex environments, and ignore other information that may affect target tracking. The use of a single feature will lead to insufficient robustness of the tracking results and make it difficult to deal with problems such as target deformation and occlusion. The spatial recognition mode may not be effective when dealing with changes in target shape, and the spectral recognition mode is prone to failure when the illumination changes. Combining the spatial recognition mode with the spectral recognition mode to achieve edge coupling of the target tracking image can ensure both sensitivity to shape features and sensitivity to spectral features during target tracking, thereby improving the credibility of the target tracking results. Therefore, how to combine the spatial recognition mode with the spectral recognition mode to achieve edge coupling of the target tracking image, thereby improving the credibility of the target tracking results, is a difficult problem faced by the industry. Summary of the invention

[0004] The present application provides a target tracking method, system, device and storage medium based on pattern recognition, which can combine spatial recognition pattern and spectral recognition pattern to realize edge coupling of target tracking image, thereby improving the credibility of target tracking results.

[0005] In a first aspect, the present application provides a target tracking method based on pattern recognition, comprising: Collect the tracking image stream of the target at each movement, and then determine the edge contour information of the target in each tracking image stream; Determine multiple motion postures of the target during motion through all edge contour information, perform spatial pattern recognition on each motion posture according to the spatial transformation characteristics of the target, and obtain the edge shape characteristics of the target in each motion posture; The current image stream of the monitoring target is performed with a spectral convolution operation on the tracking spot of the target image between frames in the current image stream based on the inter-frame spectral information of the tracking image stream, so as to obtain the tracking interference amount at each inter-frame conversion of the current image stream, and then determine the spectral convolution information of the edge contour of the target in the current image stream through each tracking interference amount and all edge contour information; Edge coupling is performed on the edge shape feature and the spectral convolution information to obtain an edge coupling feature of the target in the current image stream; A tracking frame of the target in the current image stream is determined according to the edge coupling feature, and then the target is tracked based on the tracking frame.

[0006] In some embodiments, determining edge contour information of a target in each tracking image stream specifically includes: For each tracking image stream, obtaining position locking information within a tracking frame in each tracking image in the tracking image stream; Determine the posture feature of the tracking frame in each tracking image in the tracking image stream according to each position locking information; The edge contour information of the target in the tracking image stream is determined through all the posture features, and then the edge contour information of the target in each tracking image stream is obtained.

[0007] In some embodiments, determining multiple motion postures of a target during motion through all edge contour information specifically includes: Determine the contour structure features in each edge contour information; Map each contour structure feature into a motion contour based on a convolutional neural network; determining profile similarities between respective motion profiles; All motion contours are merged into multiple motion gestures based on the similarity of each contour.

[0008] In some embodiments, spatial pattern recognition is performed on each motion posture according to the spatial transformation characteristics of the target to obtain edge shape characteristics of the target in each motion posture, specifically including: For each motion posture, the morphological transformation domain of the motion posture is extracted from the spatial transformation features of the target; Performing edge detection on the morphological transformation domain to obtain edge information of the motion posture; Based on the spatial recognition algorithm, all motion contours in the motion posture are aligned to obtain the posture shape information of the motion posture; The edge shape features of the target in the motion posture are determined by the edge information and the posture shape information, and then the edge shape features of the target in each motion posture are obtained.

[0009] In some embodiments, performing a spectral convolution operation on the tracking spot of the target image between frames in the current image stream based on the inter-frame spectral information of the tracking image stream to obtain the tracking interference amount at each inter-frame conversion of the current image stream specifically includes: For each inter-frame transition of the current image stream, the spectrum variation during the inter-frame transition is obtained from the inter-frame spectrum information of the tracking image stream; Acquire an adjacent target image during inter-frame conversion from an inter-frame target image of a current image stream; Performing a convolution operation on the tracking light spot of the adjacent target image based on the spectral variation to obtain a convolution light spot; The tracking interference amount of the current image stream during the frame transition is determined by the convolution spot, and then the tracking interference amount of the current image stream during each frame transition is obtained.

[0010] In some embodiments, determining the spectral convolution information of the edge contour of the target in the current image stream through each tracking interference amount and all edge contour information specifically includes: Perform convolution coupling on all edge contour information to obtain the convolution coupling domain of the edge contour in the target; Performing interference correction on the inter-frame conversion process of each convolution coupling sub-block in the convolution coupling domain by tracking each interference amount, so as to obtain a plurality of spectral convolution values; The spectral convolution information of the edge contour of the target in the current image stream is determined according to all the spectral convolution values.

[0011] In some embodiments, edge coupling the edge shape feature with the spectral convolution information to obtain the edge coupling feature of the target in the current image stream specifically includes: Determining an edge difference domain between the edge shape feature and the spectral convolution information; Performing coupling association on each edge block in the edge difference domain to obtain a plurality of coupling association values; The edge coupling characteristics of the target in the current image stream are determined through all coupling correlation values.

[0012] In a second aspect, the present application provides a target tracking system based on pattern recognition, comprising: An acquisition module is used to acquire the tracking image stream of the target each time it moves, and then determine the edge contour information of the target in each tracking image stream; A processing module is used to determine multiple motion postures of the target during motion through all edge contour information, and to perform spatial pattern recognition on each motion posture according to the spatial transformation characteristics of the target to obtain edge shape characteristics of the target in each motion posture; The processing module is also used to monitor the current image stream of the target, perform a spectral convolution operation on the tracking spot of the target image between frames in the current image stream based on the inter-frame spectral information of the tracking image stream, obtain the tracking interference amount at each inter-frame conversion of the current image stream, and then determine the spectral convolution information of the edge contour of the target in the current image stream through each tracking interference amount and all edge contour information; The processing module is further used to perform edge coupling on the edge shape feature and the spectral convolution information to obtain edge coupling features of the target in the current image stream; An execution module is used to determine a tracking frame of a target in a current image stream according to the edge coupling feature, and then track the target based on the tracking frame.

[0013] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device executes the above-mentioned pattern recognition-based target tracking method.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, in which instructions or codes are stored. When the instructions or codes are run on a computer, the computer implements the above-mentioned pattern recognition-based target tracking method when executed.

[0015] The technical solution provided by the embodiments disclosed in this application has the following beneficial effects: The present application provides a target tracking method, system, device and storage medium based on pattern recognition. The tracking image stream of the target is collected each time the target moves, and then the edge contour information of the target in each tracking image stream is determined; multiple motion postures of the target during motion are determined through all the edge contour information, and spatial pattern recognition is performed on each motion posture according to the spatial conversion characteristics of the target to obtain the edge shape characteristics of the target in each motion posture; the current image stream of the target is monitored, and a spectral convolution operation is performed on the tracking spot of the target image between frames in the current image stream based on the inter-frame spectral information of the tracking image stream to obtain the tracking interference amount at each inter-frame conversion of the current image stream, and then the spectral convolution information of the edge contour of the target in the current image stream is determined through each tracking interference amount and all the edge contour information; the edge shape feature is edge-coupled with the spectral convolution information to obtain the edge coupling feature of the target in the current image stream; the tracking frame of the target in the current image stream is determined according to the edge coupling feature, and then the target is tracked based on the tracking frame.

[0016] It can be seen that in this application, the tracking frame of the target in the current image stream can be determined according to the edge coupling feature, and then the target can be tracked based on the tracking frame; first, spatial pattern recognition can more accurately understand the spatial transformation of the target, and extract the edge shape features under each motion posture, which can facilitate the subsequent prediction of the motion posture and shape of the target, and reduce errors and omissions in target tracking; secondly, the spectral features between each frame in the current image stream can be evaluated through the spectral convolution operation, so as to extract the tracking interference amount, and the spectral convolution features of the target can be accurately calculated by combining the tracking interference amount with the edge contour information, thereby improving the accuracy of tracking the edge of the target. The credibility is improved, which is convenient for identifying and distinguishing targets in complex environments, and reducing tracking errors caused by spectral interference; finally, the edge coupling feature combines the spatial recognition mode and the spectral recognition mode, comprehensively considers the shape characteristics and spectral characteristics of the target, can improve the comprehensiveness and accuracy of target tracking, reduce the limitations of a single mode, and enable the tracking system to better handle changes and interference in complex environments, and improve the credibility of target tracking results. In summary, based on the above scheme, the edge coupling of the target tracking image can be achieved by combining the spatial recognition mode and the spectral recognition mode, thereby improving the credibility of the target tracking results. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0018] Figure 1 is an exemplary flow chart of a target tracking method based on pattern recognition according to some embodiments of the present application; Figure 2 is a schematic diagram of a process for determining a tracking interference amount according to some embodiments of the present application; Figure 3 is a schematic diagram of a process for determining edge coupling characteristics according to some embodiments of the present application; Figure 4 is a schematic diagram of exemplary hardware and / or software of a target tracking system based on pattern recognition according to some embodiments of the present application; Figure 5 It is a structural diagram of a computer device for implementing a target tracking method based on pattern recognition according to some embodiments of the present application. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0020] The embodiment of the present application provides a target tracking method, system, device and storage medium based on pattern recognition, the core of which is to determine multiple motion postures of the target during motion through all edge contour information, perform spatial pattern recognition on each motion posture, and obtain edge shape features under each motion posture; perform spectral convolution operation on the tracking spot of the target image between frames in the current image stream, obtain the tracking interference amount at each frame transition of the current image stream, and then determine the spectral convolution information of the edge contour of the target in the current image stream through each tracking interference amount and all edge contour information; edge-couple the edge shape feature with the spectral convolution information to obtain the edge coupling feature of the target in the current image stream; determine the tracking frame of the target in the current image stream according to the edge coupling feature, and then track the target based on the tracking frame. Based on the above scheme, the credibility of the target tracking result can be improved.

[0021] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods. Figure 1 , which is an exemplary flow chart of a target tracking method based on pattern recognition according to some embodiments of the present application, and the target tracking method based on pattern recognition mainly includes the following steps: In step 101, a tracking image stream of the target is collected each time it moves, and then edge contour information of the target in each tracking image stream is determined.

[0022] It should be noted that, in the present application, the target is a selected tracking target; the tracking image stream is a data stream composed of tracking images of each movement process of the target, wherein each frame of the tracking image contains the position locking information of the tracking frame on the target, the conversion area from the previous frame image to the current frame image and the spectral conversion information between the tracking images between adjacent frames, the tracking frame refers to the rectangular frame used to surround the target in target tracking, the position locking information is the position of the tracking frame in each frame of the tracking image, that is, the position of the target in each frame of the tracking image, the spectral conversion information is the spectral conversion process information between the current frame tracking image and the previous frame tracking image, which can be collected by existing spectral instruments (for example: hyperspectral imaging equipment); in specific implementation, the tracking image stream of each movement of the target within a specified time period (the default is within the last year) is obtained in the specified target tracking system, wherein the specified target tracking system refers to the target tracking system applied in this scheme.

[0023] In some embodiments, determining edge contour information of a target in each tracking image stream may be implemented by the following steps: For each tracking image stream, obtaining position locking information within a tracking frame in each tracking image in the tracking image stream; Determine the posture feature of the tracking frame in each tracking image in the tracking image stream according to each position locking information; The edge contour information of the target in the tracking image stream is determined through all the posture features, and then the edge contour information of the target in each tracking image stream is obtained.

[0024] It should be noted that, in the present application, the edge contour information is a collection of the target's motion shape information and motion direction information in the historical motion; in specific implementation, first, for each tracking image stream, the position lock information in the tracking frame in each tracking image in the tracking image stream is obtained; then, for each tracking image in the tracking image stream, the existing image processing algorithm (for example: Canny edge detection algorithm) can be used to extract the contour features of the target in the position lock information of the tracking image, and the contour feature can be used as the posture feature of the tracking frame in the tracking image. The posture feature is a feature used to describe the shape information and direction information of the target in the tracking image; finally, the collection of all posture features can be used as the edge contour information of the tracking image stream, and the edge contour information of the target in each tracking image stream can be obtained by the above method.

[0025] In step 102, multiple motion postures of the target are determined through all edge contour information, and spatial pattern recognition is performed on each motion posture according to the spatial transformation characteristics of the target to obtain edge shape characteristics of the target in each motion posture.

[0026] In some embodiments, determining multiple motion postures of a target during motion by using all edge contour information may be implemented by the following steps: Determine the contour structure features in each edge contour information; Map each contour structure feature into a motion contour based on a convolutional neural network; determining profile similarities between respective motion profiles; All motion contours are merged into multiple motion gestures based on the similarity of each contour.

[0027] It should be noted that, in the present application, motion posture refers to the characteristics of the motion form of the target in different time periods; contour similarity indicates the similarity of the contour information of the target in motion. The contour similarity calculation can accurately compare the morphological similarities under different motion contours, which facilitates the subsequent merging of all motion contours to obtain the motion posture; the motion contour is used to describe the contour characteristics of the target in motion; the contour structure characteristics are parameters that describe the shape and geometric characteristics of a specified target.

[0028] In the specific implementation, firstly, the feature descriptors of each edge contour information can be extracted using the existing feature extraction algorithm (for example, Hu moment algorithm) as the contour structure features in each edge contour information; secondly, for each contour structure feature, a convolutional neural network model is initialized, and a large amount of annotated contour data is obtained to train the convolutional neural network model, and the contour structure feature can be input into the trained convolutional neural network model for contour mapping, and the result of contour mapping is used as the motion contour corresponding to the contour structure feature; then, the Hausdorff distance between each motion contour can be calculated using the existing feature matching algorithm as the contour similarity between each motion contour; finally, each contour similarity can be used as the clustering parameter of the existing clustering algorithm (for example, K-means clustering algorithm), and the clustering algorithm can be used to cluster and merge all motion contours to obtain multiple contour clustering clusters, and all motion contours in each contour clustering cluster can be classified as the motion posture of the contour clustering cluster, and all motion postures can be obtained. Similar contours are automatically grouped by the clustering algorithm to improve the efficiency and accuracy of posture classification. The grouped motion postures can accurately reflect the motion mode of the target in different time periods.

[0029] It should be noted that, in the present application, the spatial transformation feature refers to the changing characteristics of the position and posture of the target in space; in the specific implementation, for each frame of tracking image, the transformation area in the tracking image is obtained as the morphological transformation domain, and the morphological transformation domain of each frame of tracking image can be obtained by the above method. The set of all morphological transformation domains can be used as the spatial transformation feature of the target, wherein the morphological transformation domain represents the changing area when the target converts between different motion postures.

[0030] In some embodiments, spatial pattern recognition is performed on each motion posture according to the spatial transformation characteristics of the target, and the edge shape characteristics of the target in each motion posture can be obtained by the following steps: For each motion posture, the morphological transformation domain of the motion posture is extracted from the spatial transformation features of the target; Performing edge detection on the morphological transformation domain to obtain edge information of the motion posture; Based on the spatial recognition algorithm, all motion contours in the motion posture are aligned to obtain the posture shape information of the motion posture; The edge shape features of the target in the motion posture are determined by the edge information and the posture shape information, and then the edge shape features of the target in each motion posture are obtained.

[0031] It should be noted that, in the present application, edge shape features represent edge shape description information of a target in a specific motion posture; edge information is feature information describing the edge of a target in a specific motion posture; posture shape information is feature information describing the overall shape of a target in a specific motion posture. By integrating all motion contours, complete posture shape information can be obtained, thereby improving the accuracy of the shape description of each motion posture.

[0032] In the specific implementation, first, for each motion posture, the morphological transformation domain of the motion posture is extracted from the spatial transformation features of the target; secondly, a specific edge monitoring algorithm (for example, Canny edge detection algorithm) can be used to perform edge detection on the morphological transformation domain, and the edge detection result can be used as the edge information of the motion posture; then, a spatial recognition algorithm (for example, Procrustes analysis) can be used to perform shape alignment on all motion contours in the motion posture, and the result of the shape alignment can be used as the posture shape information of the motion posture; finally, a feature fusion model is initialized, the posture shape information is used as the basic fusion target of the feature fusion model, the edge information is used as the edge correction parameter of the feature fusion model in the fusion, and the feature fusion model is used to complete the feature fusion of the edge shape of the target in the motion posture, and the result after the feature fusion can be used as the edge shape feature of the target in the motion posture. The edge shape features of the target in each motion posture can be obtained in the above manner.

[0033] In step 103, the current image stream of the monitored target is subjected to a spectral convolution operation on the tracking spot of the target image between frames in the current image stream based on the inter-frame spectral information of the tracking image stream, so as to obtain the tracking interference amount at each inter-frame transition of the current image stream, and then determine the spectral convolution information of the edge contour of the target in the current image stream through each tracking interference amount and all the edge contour information.

[0034] It should be noted that in the present application, the current image stream is the data stream of each frame of tracking image of the target under current tracking in the specified target tracking system; in specific implementation, the tracking image stream of the target under current tracking can be monitored in the specified target tracking system as the current image stream.

[0035] In some embodiments, based on the inter-frame spectral information of the tracking image stream, a spectral convolution operation is performed on the tracking spot of the inter-frame target image in the current image stream to obtain the tracking interference amount at each inter-frame transition of the current image stream, referring to Figure 2 The figure is a schematic diagram of a process for determining a tracking interference amount in some embodiments of the present application. In this embodiment, determining the tracking interference amount can be implemented by using the following steps: In step 1031, for each inter-frame transition of the current image stream, the spectrum variation during the inter-frame transition is obtained from the inter-frame spectrum information of the tracking image stream; In step 1032, an adjacent target image during inter-frame conversion is obtained from the inter-frame target image of the current image stream; In step 1033, a convolution operation is performed on the tracking light spot of the adjacent target image based on the spectral variation to obtain a convolution light spot; In step 1034, the tracking interference amount of the current image stream during the frame transition is determined by the convolution spot, and then the tracking interference amount of the current image stream during each frame transition is obtained.

[0036] It should be noted that, in the present application, the tracking interference amount represents the degree of interference to the current image stream during the inter-frame conversion; the spectral change amount is a quantitative indicator used to measure the change in the target spectral characteristics between adjacent frames in the current image stream; the inter-frame spectral information is the light reflection or absorption characteristics of the target at a specific wavelength, and the spectral conversion information between adjacent tracking images can be used as the inter-frame spectral information; the convolution spot refers to the interference spot used to describe the convolution image conversion between adjacent frames.

[0037] In the specific implementation, first, for each inter-frame conversion of the current image stream, the spectral conversion information at the time of inter-frame conversion is obtained from the inter-frame spectral information of the tracking image stream, and the difference between the spectra before and after the conversion in the spectral conversion information is used as the spectral change amount at the time of inter-frame conversion; secondly, the adjacent target image at the time of inter-frame conversion is obtained in the inter-frame target image of the current image stream; then, a convolution model based on a neural network is initialized, the spectral change amount is used as the convolution kernel of the convolution model, and the tracking spot of the adjacent target image is used as the convolution target of the convolution model, and the convolution operation is completed using the convolution model, and the result of the convolution operation can be used as the convolution spot; finally, the number of pixels that differ between the convolution spot and the tracking spot can be counted as the tracking interference amount of the current image stream at the time of inter-frame conversion. The tracking interference amount at each inter-frame conversion of the current image stream can be obtained in the above manner.

[0038] In some embodiments, determining the spectral convolution information of the edge contour of the target in the current image stream through each tracking interference amount and all edge contour information can be implemented by the following steps: Perform convolution coupling on all edge contour information to obtain the convolution coupling domain of the edge contour in the target; Performing interference correction on the inter-frame conversion process of each convolution coupling sub-block in the convolution coupling domain by tracking each interference amount, so as to obtain a plurality of spectral convolution values; The spectral convolution information of the edge contour of the target in the current image stream is determined according to all the spectral convolution values.

[0039] It should be noted that in the present application, the spectral convolution information represents the comprehensive spectral feature information of the target edge contour. The use of this spectral convolution information can improve the accuracy and robustness of target recognition, thereby forming a complete spectral convolution information map, which can further improve the subsequent tracking accuracy and tracking robustness of the target; the spectral convolution value is a parameter that reflects the spectral characteristics of the target; the convolution coupling domain is a data domain composed of the edge spectral information of multiple frames of images after convolution coupling processing.

[0040] In the specific implementation, first, for each edge contour information, the edge contour information can be converted into a two-dimensional contour matrix, and a part of the area (the default is a 3x3 matrix) is randomly selected in the two-dimensional contour matrix as the local contour feature, and a convolution kernel (the default is a 3x3 matrix) is selected. The convolution kernel slides on the two-dimensional contour matrix n times, and the sub-block of the convolution kernel in the two-dimensional contour matrix after each sliding can be used as a convolution sub-block, and the convolution value after each sliding is calculated as the convolution value of each convolution sub-block corresponding to the edge contour information. The convolution value of each convolution sub-block corresponding to each edge contour information can be obtained in the above manner. For each convolution sub-block, the convolution value of the convolution sub-block corresponding to each edge contour information is accumulated as the convolution coupling value of the convolution sub-block, and the accumulated value can be used as the convolution coupling value of the convolution sub-block. The convolution sub-block containing the convolution coupling value after addition is used as the convolution coupling sub-block corresponding to the convolution sub-block. The convolution coupling sub-block corresponding to each convolution sub-block can be obtained by the above method, and each convolution coupling sub-block can be arranged according to the position in the corresponding two-dimensional contour matrix as the convolution coupling domain of the edge contour in the target; then, for each convolution coupling sub-block in the convolution coupling domain, the product of the mean value of all tracking interference amounts and the convolution value of the convolution coupling sub-block can be used as the denoising convolution value of the convolution coupling sub-block. The denoising convolution value of each convolution coupling sub-block in the convolution coupling domain can be obtained by the above method, and all denoising convolution values ​​can be removed and used as multiple spectral convolution values; finally, the set of all spectral convolution values ​​can be used as the spectral convolution information of the edge contour of the target in the current image stream.

[0041] In step 104, the edge shape feature is edge-coupled with the spectral convolution information to obtain an edge coupling feature of the target in the current image stream.

[0042] In some embodiments, the edge shape feature is edge-coupled with the spectral convolution information to obtain the edge coupling feature of the target in the current image stream, referring to Figure 3 As described above, the figure is a schematic diagram of a process for determining edge coupling features in some embodiments of the present application. In this embodiment, determining edge coupling features can be implemented by the following steps: In step 1041, an edge difference domain between the edge shape feature and the spectral convolution information is determined; In step 1042, coupling association is performed on each edge block in the edge difference domain to obtain a plurality of coupling association values; In step 1043, the edge coupling characteristics of the target in the current image stream are determined through all coupling association values.

[0043] It should be noted that, in the present application, the edge coupling feature is a feature that reflects the complete edge information of the target; the edge difference domain refers to the difference area between the edge shape feature and the spectral convolution information; the coupling correlation value represents the numerical value of the degree of correlation between edge blocks.

[0044] In specific implementation, first, each convolution coupling sub-block can be regarded as an edge block, and the Euclidean distance between the edge coupling feature and each spectral convolution value in the spectral convolution information can be calculated as the edge difference value of the corresponding edge block, and the set of all edge difference values ​​can be regarded as the edge difference domain; then, for each edge block, the edge difference value corresponding to the edge block in the edge difference domain and the average of the edge difference values ​​corresponding to all adjacent edge blocks can be used as the coupling value of the edge block. The coupling value of each edge block in the edge difference domain can be obtained in the above manner, and all coupling values ​​can be removed and used as coupling association values; finally, the set of all coupling association values ​​can be used as the edge coupling feature.

[0045] In step 105, a tracking frame of the target in the current image stream is determined according to the edge coupling feature, and then the target is tracked based on the tracking frame.

[0046] In some embodiments, determining the tracking frame of the target in the current image stream according to the edge coupling feature may be implemented by the following steps: Initialize a tracking frame based on the first frame target image of the target; Determine edge information of the target in the tracking frame by using the edge coupling feature; A tracking frame of the target in the current image stream is determined according to the edge information and the tracking frame.

[0047] In the specific implementation, first, the first frame image in the current image stream is selected, the position and size of the target are automatically marked, and a rectangular frame is initialized with the position and size as the initial tracking frame; then, the existing edge detection algorithm (for example: Canny edge detection) can be used to combine the edge coupling features in the tracking frame to obtain the accurate target boundary as the edge information of the target in the tracking frame; finally, in the tracking frame of the current frame, the edge information is used as the edge contour of the target, and the position and size of the tracking frame are adjusted to accurately surround the edge contour of the target, and the above steps are repeated to process each frame of the target image in the current image stream to form a tracking frame composed of continuous edge contours.

[0048] In some embodiments, tracking the target based on the tracking frame is to use the tracking frame as a tracking result of the target in the current image stream.

[0049] In the present application, the tracking frame of the target in the current image stream can be determined according to the edge coupling feature, and then the target can be tracked based on the tracking frame; first, spatial pattern recognition can more accurately understand the spatial transformation of the target and extract the edge shape features under each motion posture, which can facilitate the subsequent prediction of the motion posture and shape of the target and reduce errors and omissions in target tracking; secondly, the spectral features between each frame in the current image stream can be evaluated through the spectral convolution operation, so as to extract the tracking interference amount, and the spectral convolution features of the target can be accurately calculated by combining the tracking interference amount with the edge contour information, thereby improving the reliability of accurate tracking of the target edge, facilitating the identification and differentiation of targets in complex environments, and reducing tracking errors caused by spectral interference; finally, the edge coupling feature combines the spatial recognition mode and the spectral recognition mode, comprehensively considers the shape features and spectral features of the target, can improve the comprehensiveness and accuracy of target tracking, reduce the limitations of a single mode, enable the tracking system to better handle changes and interference in complex environments, and improve the credibility of target tracking results; in summary, based on the above scheme, the edge coupling of the target tracking image can be achieved by combining the spatial recognition mode and the spectral recognition mode, thereby improving the credibility of the target tracking results is a difficult problem faced by the industry.

[0050] In addition, in another aspect of the present application, in some embodiments, the present application provides a target tracking system based on pattern recognition, referring to Figure 4 , which is a schematic diagram of exemplary hardware and / or software of a target tracking system based on pattern recognition according to some embodiments of the present application, including: an acquisition module 201, a processing module 202 and an execution module 203, which are described as follows: The acquisition module 201 in the present application is mainly used to acquire the tracking image stream of the target at each movement, and then determine the edge contour information of the target in each tracking image stream; Processing module 202, in the present application, is used to determine multiple motion postures of the target during motion through all edge contour information, perform spatial pattern recognition on each motion posture according to the spatial transformation characteristics of the target, and obtain edge shape characteristics of the target in each motion posture; It should be noted that the processing module 202 is also used to monitor the current image stream of the target, and perform a spectral convolution operation on the tracking spot of the target image between frames in the current image stream based on the inter-frame spectral information of the tracking image stream, so as to obtain the tracking interference amount at each inter-frame conversion of the current image stream, and then determine the spectral convolution information of the edge contour of the target in the current image stream through each tracking interference amount and all edge contour information; In addition, the processing module 202 is further used to perform edge coupling on the edge shape feature and the spectral convolution information to obtain edge coupling features of the target in the current image stream; The execution module 203 in the present application is mainly used to determine the tracking frame of the target in the current image stream according to the edge coupling feature, and then track the target based on the tracking frame.

[0051] The above describes in detail the examples of the target tracking method, system, device and storage medium based on pattern recognition provided by the embodiments of the present application. It can be understood that the corresponding device includes a hardware structure and / or software module corresponding to each function in order to realize the above functions. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0052] In some embodiments, the present application also provides a computer device, which includes a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device executes the above-mentioned pattern recognition-based target tracking method.

[0053] In some embodiments, reference Figure 5 , the dotted line in the figure indicates that the unit or the module is optional, and the figure is a schematic diagram of the structure of a computer device for implementing a target tracking method based on pattern recognition according to an embodiment of the present application. The target tracking method based on pattern recognition described in the above embodiment can be Figure 5The computer device shown in the figure is implemented, and the computer device includes at least one processor 301, a memory 302 and at least one communication unit 305. The computer device can be a terminal device, a server or a chip.

[0054] The processor 301 may be a general-purpose processor or a special-purpose processor. For example, the processor 301 may be a central processing unit (CPU), which may be used to control the computer device, execute software programs, and process data of the software programs. The computer device may also include a communication unit 305 to implement signal input (reception) and output (transmission).

[0055] For example, the computer device may be a chip, the communication unit 305 may be an input and / or output circuit of the chip, or the communication unit 305 may be a communication interface of the chip, and the chip may be a component of a terminal device, a network device, or other devices.

[0056] For another example, the computer device may be a terminal device or a server, and the communication unit 305 may be a transceiver of the terminal device or the server, or the communication unit 305 may be a transceiver circuit of the terminal device or the server.

[0057] The computer device may include one or more memories 302, on which a program 304 is stored. The program 304 can be executed by the processor 301 to generate instructions 303, so that the processor 301 performs the method described in the above method embodiment according to the instructions 303. Optionally, data (such as a target audit model) can also be stored in the memory 302. Optionally, the processor 301 can also read the data stored in the memory 302, and the data can be stored at the same storage address as the program 304, or the data can be stored at a different storage address from the program 304.

[0058] The processor 301 and the memory 302 may be provided separately or integrated together, for example, integrated on a system on chip (SOC) of the terminal device.

[0059] It should be understood that each step of the above method embodiment can be completed by a hardware-based logic circuit or software-based instructions in the processor 301. The processor 301 can be a CPU, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0060] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0061] For example, in some embodiments, the present application also provides a computer-readable storage medium, in which instructions or codes are stored. When the instructions or codes are run on a computer, the computer implements the above-mentioned pattern recognition-based target tracking method when executed.

[0062] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0063] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A target tracking method based on pattern recognition, characterized in that: The steps include: Collect the tracking image stream of the target at each movement, and then determine the edge contour information of the target in each tracking image stream; Determine multiple motion postures of the target during motion through all edge contour information, perform spatial pattern recognition on each motion posture according to the spatial transformation characteristics of the target, and obtain the edge shape characteristics of the target in each motion posture; The current image stream of the monitoring target is performed with a spectral convolution operation on the tracking spot of the target image between frames in the current image stream based on the inter-frame spectral information of the tracking image stream, so as to obtain the tracking interference amount at each inter-frame conversion of the current image stream, and then determine the spectral convolution information of the edge contour of the target in the current image stream through each tracking interference amount and all edge contour information; Edge coupling is performed on the edge shape feature and the spectral convolution information to obtain edge coupling features of the target in the current image stream; A tracking frame of the target in the current image stream is determined according to the edge coupling feature, and then the target is tracked based on the tracking frame.

2. The method according to claim 1, characterized in that Determining the edge contour information of the target in each tracking image stream specifically includes: For each tracking image stream, obtaining position locking information within a tracking frame in each tracking image in the tracking image stream; Determine the posture feature of the tracking frame in each tracking image in the tracking image stream according to each position locking information; The edge contour information of the target in the tracking image stream is determined through all the posture features, and then the edge contour information of the target in each tracking image stream is obtained.

3. The method according to claim 1, characterized in that The multiple motion postures of the target during motion are determined by all edge contour information, including: Determine the contour structure features in each edge contour information; Map each contour structure feature into a motion contour based on a convolutional neural network; determining profile similarities between respective motion profiles; All motion contours are merged into multiple motion gestures based on the similarity of each contour.

4. The method according to claim 1, characterized in that According to the spatial transformation characteristics of the target, the spatial pattern recognition is performed on each motion posture, and the edge shape characteristics of the target in each motion posture are obtained, which specifically include: For each motion posture, the morphological transformation domain of the motion posture is extracted from the spatial transformation features of the target; Performing edge detection on the morphological transformation domain to obtain edge information of the motion posture; Based on the spatial recognition algorithm, all motion contours in the motion posture are aligned to obtain the posture shape information of the motion posture; The edge shape features of the target in the motion posture are determined by the edge information and the posture shape information, and then the edge shape features of the target in each motion posture are obtained.

5. The method according to claim 1, characterized in that Based on the inter-frame spectral information of the tracking image stream, a spectral convolution operation is performed on the tracking spot of the inter-frame target image in the current image stream to obtain the tracking interference amount at each inter-frame conversion of the current image stream, which specifically includes: For each inter-frame transition of the current image stream, the spectrum variation during the inter-frame transition is obtained from the inter-frame spectrum information of the tracking image stream; Acquire an adjacent target image during inter-frame conversion from an inter-frame target image of a current image stream; Performing a convolution operation on the tracking light spot of the adjacent target image based on the spectral variation to obtain a convolution light spot; The tracking interference amount of the current image stream during the frame transition is determined by the convolution spot, and then the tracking interference amount of the current image stream during each frame transition is obtained.

6. The method according to claim 1, characterized in that The spectral convolution information of the edge contour of the target in the current image stream is determined by each tracking interference amount and all edge contour information, specifically including: Perform convolution coupling on all edge contour information to obtain the convolution coupling domain of the edge contour in the target; Performing interference correction on the inter-frame conversion process of each convolution coupling sub-block in the convolution coupling domain by tracking each interference amount, so as to obtain a plurality of spectral convolution values; The spectral convolution information of the edge contour of the target in the current image stream is determined according to all the spectral convolution values.

7. The method according to claim 1, characterized in that The edge shape feature is edge-coupled with the spectral convolution information to obtain the edge coupling feature of the target in the current image stream, specifically comprising: Determining an edge difference domain between the edge shape feature and the spectral convolution information; Performing coupling association on each edge block in the edge difference domain to obtain a plurality of coupling association values; The edge coupling characteristics of the target in the current image stream are determined through all coupling correlation values.

8. A target tracking system based on pattern recognition, characterized in that: include: An acquisition module is used to acquire the tracking image stream of the target each time it moves, and then determine the edge contour information of the target in each tracking image stream; A processing module is used to determine multiple motion postures of the target during motion through all edge contour information, and to perform spatial pattern recognition on each motion posture according to the spatial transformation characteristics of the target to obtain edge shape characteristics of the target in each motion posture; The processing module is also used to monitor the current image stream of the target, perform a spectral convolution operation on the tracking spot of the target image between frames in the current image stream based on the inter-frame spectral information of the tracking image stream, obtain the tracking interference amount at each inter-frame conversion of the current image stream, and then determine the spectral convolution information of the edge contour of the target in the current image stream through each tracking interference amount and all edge contour information; The processing module is further used to perform edge coupling on the edge shape feature and the spectral convolution information to obtain edge coupling features of the target in the current image stream; An execution module is used to determine a tracking frame of the target in the current image stream according to the edge coupling feature, and then track the target based on the tracking frame.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device executes the target tracking method based on pattern recognition as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions or codes, and when the instructions or codes are executed on a computer, the computer implements the target tracking method based on pattern recognition as described in any one of claims 1 to 7.

Images