Object identification detection and real-time tracking system

Abstract

The invention belongs to the technical field of object identification and tracking, and discloses an object identification detection and real-time tracking system. Comprising the following steps: acquiring a video image sequence, and preprocessing each frame of image in the video image sequence to obtain a plurality of frames of standard images; based on each frame of standard image, constructing an image directed graph model, and performing clustering analysis on the image directed graph model to obtain n potential target objects and corresponding feature descriptions; based on the n potential target objects and the corresponding feature descriptions, performing target object detection in the current frame standard image to obtain a target object; the target object obtained through detection is tracked, the accuracy and robustness of detection and tracking are optimized, and the target object can be reliably locked and continuously tracked.

Description

Technical Field

[0001] The present invention relates to the technical field of object recognition and tracking, and more specifically, to an object recognition detection and real-time tracking system. Background Art

[0002] The patent with application publication number CN107564037A discloses a multi-robot detection and tracking method based on local features, which mainly solves the problems of large moving object detection error and "out of control" in the intelligent monitoring technology in the prior art. The steps are as follows: S1, select appropriate local features; S2, image preprocessing; S3, feature extraction; S4, pattern recognition; S5, extract the center of mass; S6, Kalman filter algorithm for tracking and filtering; S7, calculate the world coordinates; use the selection of more ideal local features instead of the irregular robot as a whole as the detection object, greatly reducing the detection error and the error of center of mass extraction; use the method of mutual feedback between detection and tracking to reduce the detection range and improve the real-time and stability of the system; when the detection result of the current image frame is not ideal, make full use of the most recent valid detection result to narrow the detection range of the next frame, with a strong error correction mechanism to avoid the "out of control" phenomenon caused by too long detection time.

[0003] However, existing methods still usually rely on artificially designed features and are difficult to adapt to the diversity and changes of target objects in complex environments. Taking robot navigation as an example, when a robot moves in indoor and outdoor environments, it needs to detect and track various objects in real time, such as people, vehicles, obstacles, etc. The shapes, sizes, and colors of these target objects vary greatly, and artificially designed features are difficult to cover all situations. Secondly, when processing video image sequences, existing methods need to re-extract features and detect targets for each frame of the image, which requires huge computational complexity and poor real-time performance. Taking video surveillance as an example, when multiple cameras work at the same time, a large amount of video data needs to be processed in real time. If complex feature extraction and detection are performed for each frame, it is easy to cause system overload and fail to meet the needs of real-time response. Furthermore, the detection and tracking accuracy of existing methods needs to be improved. When encountering bad weather, congested road conditions, etc., there will be various noises and complex backgrounds in the video images. Existing methods are easily interfered with, resulting in failure in target object detection and tracking.

[0004] In view of this, the present invention proposes an object recognition detection and real-time tracking system to solve the above problems. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solutions: an object recognition detection and real-time tracking system, comprising: an image acquisition and processing module, a clustering analysis module and a detection and tracking module, wherein the modules are connected by wire and / or wireless means; The image acquisition and processing module includes an acquisition unit and a processing unit. The acquisition unit is used to acquire a video image sequence, and the processing unit is used to pre-process each frame of the video image sequence to obtain a plurality of frames of standard images. The cluster analysis module includes a model building unit and an analysis unit. The building unit builds an image directed graph model based on each frame of the standard image. The analysis unit is used to perform cluster analysis on the image directed graph model to obtain n potential target objects and corresponding feature descriptions. The detection and tracking module includes a detection unit and a tracking module. The detection unit performs target object detection in the current frame standard image based on n potential target objects and corresponding feature descriptions to obtain the target object; the tracking module is used to track the detected target object.

[0006] Furthermore, the pre-processing method includes: Convert the image to a grayscale image, traverse each pixel in the grayscale image, count the number of occurrences of each grayscale value, and use an array of length 256 to store the number of occurrences of each grayscale value; draw a histogram based on the array; the horizontal axis of the histogram represents the grayscale value, and the vertical axis of the histogram represents the number of occurrences of the corresponding grayscale value; The number of occurrences of grayscale values ​​in the histogram is taken as the histogram data, and the histogram data is normalized; the normalized histogram data is fitted to obtain the mean of its Gaussian distribution and standard deviation ; Mean based on Gaussian distribution and standard deviation , calculate the peak height of the Gaussian distribution and peak width ; Peak height ;in, is the initial scale parameter, For time, is the preset growth rate; is the adaptive shape parameter; Adaptive shape parameters ;in, and is a constant coefficient, Represents at time Histogram data observed at each moment; is the kernel function; Peak width ; Count the number of pixels with grayscale value 0 s_0 and the number of pixels with grayscale value 255 s_255 in the histogram, and calculate the ratio of s_0 to s_255 as a noise intensity quantification index; Perform Fourier transform on the normalized histogram data to obtain the frequency spectrum, find the peak in the frequency spectrum, and count the peak amplitude and peak period of the peak; Preset the Gaussian threshold ranges corresponding to the peak height and the peak width, respectively. If the calculated peak height and peak width are both within the corresponding Gaussian threshold ranges, it is determined that Gaussian noise exists. A preset intensity threshold range is set. If the noise intensity quantification index is within the threshold range, it is determined that salt and pepper noise exists. Preset the period threshold ranges corresponding to the peak amplitude and the peak period, respectively. If the statistically obtained peak amplitude and peak period are within the corresponding period threshold ranges, it is determined that periodic noise exists. If there is Gaussian noise, use Gaussian filter or mean filter to filter, if there is salt and pepper noise, use median filter or adaptive median filter to filter; if there is periodic noise, perform adaptive special filtering; after filtering is completed, obtain a denoised image; perform image enhancement on the denoised image to obtain a standard image.

[0007] Furthermore, the method of performing adaptive special filtering includes: Perform a two-dimensional discrete Fourier transform on the image to convert it from the spatial domain to the frequency domain to obtain the two-dimensional frequency spectrum of the image ,in, represents the frequency in the horizontal direction, Indicates the frequency in the vertical direction; initially defines an adaptive band-stop filter and defines the transfer function of the adaptive band-stop filter ; ;in, Indicates frequency point The Euclidean distance to the center of the adaptive bandstop filter, is the normalization constant, and is the order control constant coefficient; Initialize normalization constant and the order control constant coefficient; and by collecting a training data set, using cross validation or grid search methods to optimize the normalization constant on the training data set and the order control constant coefficient, find the optimal normalization constant and a combination of order control constant coefficients; The two-dimensional frequency spectrum of the image Perform a point multiplication operation with the transfer function to obtain the two-dimensional frequency spectrum after filtering ;right A two-dimensional inverse discrete Fourier transform is performed to convert it from the frequency domain back to the spatial domain, and adaptive special filtering is completed.

[0008] Furthermore, the method of performing image enhancement on the denoised image includes: Defining Mapping Values , and construct a correction lookup table, for each input pixel value in the denoised image , calculate the corresponding output pixel value ;in, , and is a nonlinear parameter; the input pixel value and output pixel value The corresponding images are stored in a correction lookup table; a mapping operation is performed based on the correction lookup table to obtain a standard image; The mapping operation is performed as follows: for each pixel in the image, the correction lookup table is used for the three channels of R, G, and B to convert its pixel value from the input pixel value Map to output pixel values; recombine the mapped pixel values ​​into a new image, which is the standard image.

[0009] Furthermore, the mapping value The definitions include: Preset Mapping Values The search range is discretized into a series of discrete mapping value points, recorded as mapping points; the evaluation index function is defined ; ;in, , and is the weight coefficient of each item, is the maximum pixel value in the image after image enhancement, is the minimum pixel value in the image after image enhancement, is the total number of gray levels of the image after image enhancement, is the gray level index, For the The probability of gray levels, Represents the horizontal gradient of the image after image enhancement. Represents the vertical gradient of the image after image enhancement; Within the search range, a grid containing all mapping points is constructed, and for each mapping point, the value of the corresponding evaluation index function is initialized; each mapping point in the grid is traversed; for the current mapping point, it is used as the mapping value, and a mapping operation is performed to obtain an enhanced image, and the value of the evaluation index function of the enhanced image is calculated, and the value of the corresponding evaluation index function in the grid is updated; After traversing all grid points, find the mapping point with the largest value of the evaluation index function in the grid as the optimal mapping point, and build a new grid near the optimal mapping point; repeatedly search for the mapping point with the largest value of the evaluation index function in the new grid until the accuracy of the grid reaches the preset accuracy threshold; The calculation formula of the grid accuracy is: ;in, The mapping value corresponding to the optimal mapping point is The accuracy of the grid, is the upper limit of the search range, is the lower limit of the search range; is the number of intervals into which the search range is equally divided; The mapping value corresponding to the optimal mapping point is The value of the evaluation index function when for Relative to the mapped value The gradient of for Relative to the mapped value The second-order derivative of and is the equilibrium parameter, and ; is the density control parameter, is the curvature control parameter.

[0010] Furthermore, the image directed graph model is constructed in a manner including: The standard image is divided into several pixel blocks, and for each pixel block, its feature descriptor is extracted; the feature descriptor is a histogram of oriented gradients, a local binary pattern or a scale-invariant feature transform; For each pixel block in the standard image, treat it as a node in the image directed graph model and calculate the similarity between each pair of pixel blocks; According to the spatial position of the pixel blocks in the standard image, a directed edge is connected between each pair of adjacent pixel blocks; the direction of the directed edge is from left to right or from top to bottom; Assign a weight to each directed edge, and the weight value is equal to the similarity between the corresponding two pixel blocks; preset a similarity threshold, for two non-adjacent pixel blocks; if their similarity is greater than or equal to the similarity threshold, also connect a directed edge between the two, thus completing the construction of the directed graph model of the image.

[0011] Furthermore, the method of dividing the standard image into a plurality of pixel blocks includes: Initialize a sliding window of a fixed size; start from the upper left corner of the standard image and slide the sliding window from left to right or from top to bottom on the image; For the area of ​​the standard image covered by the current sliding window, the texture complexity of the pixel values ​​inside it is calculated; the calculation method of texture complexity includes: For the area of ​​the standard image covered by the current sliding window, a gray-level co-occurrence matrix is ​​constructed, and the texture complexity is calculated based on the gray-level co-occurrence matrix. ; ;in, is the contrast of the area, is the entropy of the pixels in the region, is the gray level co-occurrence matrix Line The element value of the column, is the distance weighted kernel function; is the distance attenuation factor; is the pixel weighting function; Pixel weighting function ;in, For the The pixel corresponding to the row is The direction of the pixel corresponding to the column, is the preset main direction of interest; A uniform threshold is preset. If the texture complexity of the area is less than the uniform threshold, the current sliding window is regarded as a pixel block; If the texture complexity of the region is greater than or equal to the uniform threshold, the sliding window is segmented; The splitting is done as follows: The current sliding window is divided into four equal sub-windows in the horizontal and vertical directions. For each sub-window, it is repeatedly determined whether it needs to be split again until the texture complexity of all sub-windows is less than the uniform threshold. These sub-windows are regarded as pixel blocks; the sliding window is continued until the entire standard image is covered and several pixel blocks are obtained.

[0012] Furthermore, the cluster analysis method includes: Initialize the clustering parameters of cluster analysis, which include the number of clusters, the maximum number of local minimization q, the rate of change adjustment parameter b1, and the neighborhood adjustment parameter b2; Randomly select r nodes from the image directed graph model as the initial cluster centers; calculate the minimum distance from all nodes to the r cluster centers, and assign each node to the cluster where the nearest cluster center is located based on the calculated minimum distance; calculate the overall intra-cluster sum of squares of the current cluster and locally minimize the current cluster; The way to perform local minimization is to randomly select a node , remove it from its cluster, define the node Neighborhood ; Neighborhood Contains nodes Distance less than Nodes, where is the maximum distance between two nodes in the image directed graph model, in the neighborhood In the calculation, the node Redistribute the sum of squares in the population to the other r-1 clusters, choose the redistribution method that minimizes the sum of squares in the population, and repeat until q local minimizations are performed; in this process, dynamically adjust the size of b2 so that the neighborhood The size of is gradually reduced; if a cluster with a smaller overall sum of squares than the current cluster is found in the process of local minimization, the current cluster is replaced by this cluster; Repeat until the rate of change of the sum of squares within the population is less than b1. Finally, r clusters are obtained. Each cluster corresponds to a potential target object, and a total of n potential target objects are obtained. The value of r is equal to the value of n. For each cluster, calculate the weighted average feature descriptor of all nodes in it as the feature description of the corresponding potential target object; The calculation formula of the weighted average feature descriptor is: ;in, For the The weighted average feature descriptor of clusters, is the corresponding node in the cluster The weight of For Node feature descriptor.

[0013] Furthermore, the method of performing target object detection includes: Define a detection sliding window, scan the current frame standard image using the detection sliding window, and generate candidate target regions; for each candidate target region, extract its feature descriptor, record it as the candidate region feature descriptor, and match the candidate region feature descriptor with the obtained feature descriptions of the n potential target objects; the matching method is: Calculate the distance between the feature descriptor of the candidate region and the weighted average feature descriptor of each potential target object as the matching score between the corresponding potential target object and the candidate target region; The potential target object and the candidate target area with the highest matching score are marked, that is, the corresponding candidate target area is marked as the potential target object; A confidence threshold is preset, and candidate target regions whose matching scores with all potential target objects are less than the confidence threshold are judged as non-target objects and eliminated; for several candidate target regions marked as the same potential target object, post-processing is performed to obtain the corresponding target object; Post-processing methods include: Based on the matching scores of several candidate target regions and the same potential target object, the aspect ratio of the candidate target region and the area of ​​the candidate target region, an optimal candidate target region is screened out and marked as the potential target object; Screening methods include: Set the matching score threshold, remove candidate target areas whose matching scores are greater than the matching score threshold, set the upper and lower bounds of the aspect ratio [as_min, as_max], and for the remaining candidate areas, remove candidate target areas whose aspect ratios are not within the upper and lower bounds [as_min, as_max]; Set the upper and lower bounds of the area [ar_min, ar_max], and remove candidate target areas whose areas are not within the upper and lower bounds [ar_min, ar_max]; The candidate target area with the largest matching score among the remaining candidate target areas is taken as the most suitable candidate target area.

[0014] Furthermore, the method of tracking the detected target object includes: Generate N2 particles, each particle represents a state of the target object, and assign an initial weight to each particle, which is usually set to 1 / N2; for each particle, predict the state at the next moment according to the motion model, that is, predict the particle; extract the observation feature corresponding to each predicted particle, and calculate the observation likelihood of each predicted particle, that is, the similarity between the observation feature and the motion model; update the weight of each particle according to the observation likelihood; calculate the optimal state of the target according to the states of all particles and the corresponding weights, and take the mean of the sum of weights as the optimal state; and resample; the resampling method is residual resampling, systematic resampling or stratified sampling; after resampling, the weights of all particles are reset to 1 / N2; repeat the tracking of the detected target object.

[0015] The technical effects and advantages of an object recognition detection and real-time tracking system of the present invention are as follows: The present invention can automatically discover potential target objects and their feature descriptions by constructing an image directed graph model and utilizing cluster analysis, thus avoiding the defects of manually designed features and greatly improving the adaptability to the diversity and changes of target objects in complex scenes; secondly, after performing a cluster analysis on the first frame image of the video sequence, subsequent frames only need to be detected and tracked based on the obtained potential target object features, which greatly reduces the amount of calculation and significantly improves the real-time performance of the system, so that it can meet the strict requirements of real-time applications; furthermore, the input image is carefully preprocessed, including adaptive denoising, image enhancement and other links, which effectively improves the system's robustness to complex backgrounds and various noises and ensures the quality of input data; in addition, in the target detection and tracking stage, a variety of strategies are combined, such as sliding window scanning, candidate area screening, particle filtering, etc., to further optimize the accuracy and robustness of detection and tracking, so that the target object can be reliably locked and continuously tracked. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of an object recognition detection and real-time tracking system of the present invention; Figure 2 A schematic diagram of an object recognition detection and real-time tracking method of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Example 1 See also Figure 1 As shown, this embodiment provides an object recognition detection and real-time tracking system, including: an image acquisition and processing module, a cluster analysis module and a detection and tracking module, and each module is connected by wired and / or wireless means to achieve data transmission between modules; The image acquisition and processing module includes an acquisition unit and a processing unit. The acquisition unit is used to acquire a video image sequence, and the processing unit is used to pre-process each frame of the video image sequence to obtain a plurality of frames of standard images. The cluster analysis module includes a model building unit and an analysis unit. The building unit builds an image directed graph model based on each frame of the standard image. The analysis unit is used to perform cluster analysis on the image directed graph model to obtain n potential target objects and corresponding feature descriptions. The detection and tracking module includes a detection unit and a tracking module. The detection unit performs target object detection in the current frame standard image based on n potential target objects and corresponding feature descriptions to obtain the target object; the tracking module is used to track the detected target object.

[0019] Use a USB camera, webcam, etc. connected to a computer to collect video image frames in real time and obtain a video image sequence.

[0020] Further, the preprocessing method includes: Convert the image to a grayscale image, traverse each pixel in the grayscale image, count the number of occurrences of each grayscale value, and use an array of length 256 to store the number of occurrences of each grayscale value; draw a histogram based on the array; the horizontal axis of the histogram represents the grayscale value (0-255), and the vertical axis of the histogram represents the number of occurrences of the corresponding grayscale value.

[0021] The number of grayscale values ​​in the histogram is taken as the histogram data, and the histogram data is normalized so that the sum of the number of grayscale values ​​is 1; the normalized histogram data is fitted to obtain the mean of its Gaussian distribution. and standard deviation .

[0022] Mean based on Gaussian distribution and standard deviation , calculate the peak height of the Gaussian distribution and peak width ; Peak height ;in, is the initial scale parameter, For time, The preset growth rate makes the amplitude increase or decrease exponentially; is an adaptive shape parameter used to adjust the shape of the function.

[0023] Adaptive shape parameters ;in, and is a constant coefficient, is a scalar value representing the The data points observed at each moment (histogram data); is a kernel function, such as Gaussian kernel, Laplace kernel, etc.; Peak width .

[0024] The count s_0 of the pixel points with a grayscale value of 0 and the count s_255 of the pixel points with a grayscale value of 255 in the histogram are counted, and the ratio of s_0 to s_255 is calculated as a noise intensity quantification index.

[0025] Perform Fourier transform on the normalized histogram data to obtain the frequency spectrum, find the peak in the frequency spectrum, and count the peak amplitude and peak period of the peak.

[0026] The peak height and the peak width are preset to have corresponding Gaussian threshold ranges, and if the calculated peak height and peak width are both within the corresponding Gaussian threshold ranges, it is determined that Gaussian noise exists.

[0027] A preset intensity threshold range is set. If the noise intensity quantification index is within the threshold range, it is determined that salt and pepper noise exists.

[0028] The peak amplitude and the peak period respectively correspond to period threshold ranges, and if the statistically obtained peak amplitude and peak period are within the corresponding period threshold ranges, it is determined that periodic noise exists.

[0029] If there is Gaussian noise, use Gaussian filter or mean filter to filter, if there is salt and pepper noise, use median filter or adaptive median filter to filter; if there is periodic noise, perform adaptive special filtering; after filtering is completed, a denoised image is obtained.

[0030] The methods for performing adaptive special filtering include: Perform a two-dimensional discrete Fourier transform (2D DFT) on the image to convert it from the spatial domain to the frequency domain to obtain the two-dimensional frequency spectrum of the image ,in, represents the frequency in the horizontal direction, Indicates the frequency in the vertical direction.

[0031] Initially define an adaptive band-stop filter, determine the frequency range that needs to be filtered out based on periodic noise, and define the transfer function of the adaptive band-stop filter ; ;in, Indicates frequency point The Euclidean distance to the center of the adaptive bandstop filter, is a normalization constant that controls the bandwidth of the filter, and is the order control constant coefficient, and the steepness of the adaptive band-stop filter shows an exponential growth or exponential decay as the frequency increases.

[0032] Initialize normalization constant and the order control constant coefficient; and by collecting a training data set (an image data set containing various periodic noises, which can be images collected from actual scenes or artificially synthesized images), cross-validation or grid search methods are used to optimize the normalization constant on the training data set. The goal is to find the optimal normalization constant that maximizes the noise removal effect. and the combination of the order control constant coefficient.

[0033] The two-dimensional frequency spectrum of the image Perform a point multiplication operation with the transfer function to obtain the two-dimensional frequency spectrum after filtering ;right A two-dimensional inverse discrete Fourier transform (2D IDFT) is performed to convert it from the frequency domain back to the spatial domain, and the filtering is completed.

[0034] Furthermore, the denoised image is enhanced to obtain a standard image; specifically, the mapping value is defined , and construct a correction lookup table, for each input pixel value in the denoised image (usually 0 to 255), calculate the corresponding output pixel value ;in, , and is a nonlinear parameter used to control the degree of nonlinearity; the input pixel value and output pixel value The corresponding data are stored in a correction lookup table; a mapping operation is performed based on the correction lookup table to obtain a standard image.

[0035] The mapping operation is performed as follows: for each pixel in the image, the correction lookup table is used for the three channels of R, G, and B to convert its pixel value from the input pixel value Map to output pixel values; recombine the mapped pixel values ​​into a new image, which is the standard image.

[0036] Mapping Values The definitions include: Preset Mapping Values The search range is usually between [0.2, 5]. The search range is discretized into a series of discrete mapping value points, recorded as mapping points; define the evaluation index function ; ;in, , and is the weight coefficient of each item, is the maximum pixel value in the image after image enhancement, is the minimum pixel value in the image after image enhancement, is the total number of gray levels of the image after image enhancement, is the gray level index, For the The probability of gray levels, Represents the horizontal gradient of the image after image enhancement. Represents the vertical gradient of the image after image enhancement.

[0037] Within the search range, a grid containing all mapping points is constructed. For each mapping point, the value of the corresponding evaluation index function is initialized, such as setting it to 0 or other default values; each mapping point in the grid is traversed.

[0038] For the current mapping point, it is used as the mapping value, a mapping operation is performed to obtain an enhanced image, the value of the evaluation index function of the enhanced image is calculated, and the value of the corresponding evaluation index function in the grid is updated.

[0039] After traversing all grid points, find the mapping point with the largest value of the evaluation index function in the grid as the optimal mapping point, and build a new grid near the optimal mapping point; repeat the search for the mapping point with the largest value of the evaluation index function in the new grid until the accuracy of the grid reaches the preset accuracy threshold.

[0040] The calculation formula of the grid accuracy is: ;in, The mapping value corresponding to the optimal mapping point is The accuracy of the grid, is the upper limit of the search range, is the lower limit of the search range; is the number of intervals into which the search range is equally divided; The mapping value corresponding to the optimal mapping point is The value of the evaluation index function when for Relative to the mapped value When the gradient of the value of the evaluation index function is large, the density of grid points increases, resulting in a more refined search in the area; for Relative to the mapped value The second-order derivative of and is the equilibrium parameter, and ; is the density control parameter, are curvature control parameters, which are used to control the degree of change of mesh density.

[0041] Furthermore, the construction method of the image directed graph model includes: The standard image is divided into several pixel blocks, and for each pixel block, its feature descriptor is extracted; the feature descriptor is HOG (Histogram of Oriented Gradients), LBP (Local Binary Pattern) or SIFT (Scale Invariant Feature Transform).

[0042] For each pixel block in the standard image, it is treated as a node in the image directed graph model, and the similarity between each pair of pixel blocks (cosine similarity between feature descriptors) is calculated.

[0043] According to the spatial position of the pixel blocks in the standard image, a directed edge is connected between each pair of adjacent pixel blocks; the direction of the directed edge is from left to right or from top to bottom.

[0044] Assign a weight to each directed edge, and the weight value is equal to the similarity between the corresponding two pixel blocks; preset a similarity threshold, for two non-adjacent pixel blocks; if their similarity is greater than or equal to the similarity threshold, also connect a directed edge between the two, indicating that there is a similarity relationship between them, thus completing the construction of the directed graph model of the image.

[0045] Ways to divide a standard image into several pixel blocks include: Initialize a sliding window of a fixed size, such as 8x8 pixels; start from the upper left corner of the standard image, and slide the sliding window from left to right or from top to bottom on the image. The step size of each slide can be fixed (such as 4 pixels) or adaptive.

[0046] For the area of ​​the standard image covered by the current sliding window, the texture complexity of the pixel values ​​inside it is calculated; the calculation method of texture complexity includes: For the area of ​​the standard image covered by the current sliding window, a grayscale co-occurrence matrix is ​​constructed. Specifically, four main directions are selected, including 0° (horizontal direction), 45°, 90° (vertical direction) and 135°, and the distance is 1 pixel or other distances are set as required; the co-occurrence matrix of the grayscale values ​​of pixels in the area is calculated; any pixel is defined as a reference pixel, and for each pair of reference pixels and their neighboring pixels, the corresponding position of the combination of their grayscale values ​​in the grayscale co-occurrence matrix is ​​calculated, and the count value of the position is added by 1, and all element values ​​in the grayscale co-occurrence matrix are divided by the total number of pixel pairs, so that the element values ​​of the grayscale co-occurrence matrix are normalized to the range of [0, 1] to obtain the grayscale co-occurrence matrix.

[0047] Texture complexity is calculated based on the gray-level co-occurrence matrix ; ;in, is the contrast of the area, is the entropy of the pixels in the region, is the gray level co-occurrence matrix Line The element value of the column, It is a distance-weighted kernel function (using Gaussian kernel or other kernel functions) used to weight pixel pairs with different distances; is the distance attenuation factor, which controls the influence of distance on homogeneity; It is a pixel weighting function, which is used to assign different weights to different element values.

[0048] Pixel weighting function ;in, For the The pixel corresponding to the row is The direction of the pixel corresponding to the column, is the preset main direction of interest.

[0049] A uniform threshold is preset. If the texture complexity of the area is less than the uniform threshold, it means that the area is relatively uniform, and the current sliding window is regarded as a pixel block. If the texture complexity of the area is greater than or equal to the uniform threshold, it means that there are more details and edge information in the area, and the sliding window is segmented.

[0050] The splitting is done as follows: The current sliding window is divided into four equal sub-windows in the horizontal and vertical directions. For each sub-window, it is repeatedly determined whether it needs to be split again until the texture complexity of all sub-windows is less than the uniform threshold. These sub-windows are regarded as pixel blocks. All the obtained pixel blocks are recorded, including their position, size and other information. The sliding window is continued until the entire standard image is covered and several pixel blocks are obtained.

[0051] In the above process, the image will be divided into multiple pixel blocks of different sizes. In areas with simple textures, the pixel blocks will be larger, while in areas with complex textures, the pixel blocks will be smaller. This is conducive to balancing the trade-off between computational efficiency and detail retention.

[0052] Further, the cluster analysis methods include: Initialize the clustering parameters of cluster analysis, which include the number of clusters, the maximum number of local minimization q, the change rate adjustment parameter b1, and the neighborhood adjustment parameter b2.

[0053] Randomly select r nodes (pixel blocks) from the image directed graph model as the initial cluster centers; calculate the minimum distance from all nodes to these r cluster centers, and assign each node to the cluster where the nearest cluster center is located based on the calculated minimum distance.

[0054] Calculate the overall internal sum of squares of the current cluster (the sum of squares of the overall internal distances) and minimize it locally for the current cluster.

[0055] The way to perform local minimization is to randomly select a node , remove it from its cluster, define the node Neighborhood , neighborhood Contains nodes Distance less than Nodes, where is the maximum distance between two nodes in the image directed graph model, in the neighborhood In the calculation, the node Redistribute the sum of squares in the population to the other r-1 clusters, choose the redistribution method that minimizes the sum of squares in the population, and repeat until q local minimizations are performed; in this process, dynamically adjust the size of b2 so that the neighborhood The size of is gradually reduced (the degree of reduction is customized) to accelerate convergence; if a cluster with a smaller overall sum of squares than the current cluster is found during the local minimization process, the current cluster is replaced by this cluster.

[0056] Repeat until the rate of change of the sum of squares within the population is less than b1, and finally r clusters are obtained, each cluster corresponds to a potential target object, and a total of n potential target objects are obtained. The value of r is equal to the value of n; For each cluster, the weighted average feature descriptor of all nodes (pixel blocks) in it is calculated as the feature description of the corresponding potential target object.

[0057] The calculation formula of the weighted average feature descriptor is: ;in, For the The weighted average feature descriptor of clusters, is the corresponding node in the cluster The weight (similarity) of For Node feature descriptor.

[0058] Further, the target object detection method includes: A detection sliding window is defined, and the current frame standard image is scanned using the detection sliding window to generate a candidate target area; the size of the detection sliding window can be fixed or adaptive, and it usually slides from left to right and from top to bottom on the image with a certain step size.

[0059] For each candidate target region, extract its feature descriptor, record it as the candidate region feature descriptor, and match the candidate region feature descriptor with the obtained feature descriptions of the n potential target objects; the matching method is: The distance (such as Euclidean distance, cosine similarity, etc.) between the feature descriptor of the candidate region and the weighted average feature descriptor of each potential target object is calculated as the matching score between the corresponding potential target object and the candidate target region.

[0060] The potential target object and the candidate target region with the highest matching score are marked, that is, the corresponding candidate target region is marked as the potential target object.

[0061] A confidence threshold is preset, and candidate target areas whose matching scores with all potential target objects are less than the confidence threshold are judged as non-target objects and eliminated; for several candidate target areas marked as the same potential target object, post-processing is performed to obtain the corresponding target object.

[0062] Post-processing methods include: Based on the matching scores of several candidate target regions and the same potential target object, the aspect ratio of the candidate target region and the area of ​​the candidate target region, an optimal candidate target region is screened out and marked as the potential target object.

[0063] Screening methods include: Set the matching score threshold, eliminate candidate target areas whose matching scores are greater than the matching score threshold, set the upper and lower bounds of the aspect ratio [as_min, as_max], and for the remaining candidate areas, eliminate candidate target areas whose aspect ratios are not within the upper and lower bounds [as_min, as_max].

[0064] Set the upper and lower bounds of the area [ar_min, ar_max], and remove candidate target areas whose areas are not within the upper and lower bounds [ar_min, ar_max]; The candidate target area with the largest matching score among the remaining candidate target areas is taken as the most suitable candidate target area.

[0065] It can effectively eliminate a large number of interference areas that obviously do not meet the characteristics of the target object and retain the most likely target candidate areas, thereby improving the accuracy and robustness of detection.

[0066] Furthermore, the method of tracking the detected target object includes: Generate N2 particles (samples), each particle represents a state of the target object (position, scale, direction, etc.), usually use Gaussian distribution to randomly sample around the target, and assign an initial weight to each particle, usually set to 1 / N2; for each particle, according to the motion model (such as constant velocity model, random walk model, etc.), predict the state at the next moment, that is, predict the particle; extract the observation features corresponding to each predicted particle (such as color histogram, HOG, etc.), calculate the observation likelihood of each predicted particle, that is, the similarity between the observation feature and the motion model; according to the observation likelihood, update the weight of each particle (the weight of the previous iteration plus the observation likelihood and normalize it); according to the state of all particles and the corresponding weights, calculate the optimal state of the target, and take the mean of the sum of weights as the optimal state; since the weight of the particle will gradually degenerate, resampling is performed to eliminate low-weight particles; the resampling method is residual resampling, systematic resampling or stratified sampling; after resampling, the weights of all particles are reset to 1 / N2; repeatedly implement tracking of the detected target object.

[0067] In this embodiment, by constructing an image directed graph model and utilizing cluster analysis, potential target objects and their feature descriptions can be automatically discovered, thus avoiding the defects of manually designed features and greatly improving the ability to adapt to the diversity and changes of target objects in complex scenes; secondly, after performing a cluster analysis on the first frame of the video sequence, subsequent frames only need to be detected and tracked based on the obtained features of the potential target objects, which greatly reduces the amount of calculation and significantly improves the real-time performance of the system, enabling it to meet the strict requirements of real-time applications; furthermore, the input image is carefully preprocessed, including adaptive denoising, image enhancement and other links, which effectively improves the robustness of the system to complex backgrounds and various noises and ensures the quality of the input data; in addition, in the target detection and tracking stage, a variety of strategies are combined, such as sliding window scanning, candidate area screening, particle filtering, etc., to further optimize the accuracy and robustness of detection and tracking, so that the target object can be reliably locked and continuously tracked.

[0068] Example 2 See also Figure 2 As shown, the part not described in detail in this embodiment is described in Example 1, which provides an object recognition detection and real-time tracking method, including: S1, obtaining a video image sequence, preprocessing each frame of the video image sequence, and obtaining several frames of standard images; S2. Based on each frame of the standard image, an image directed graph model is constructed, and the image directed graph model is clustered and analyzed to obtain n potential target objects and corresponding feature descriptions; S3. Based on the n potential target objects and the corresponding feature descriptions, target object detection is performed in the current frame standard image to obtain the target object; and the detected target object is tracked.

[0069] Example 3 This embodiment discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the operation mode of the object recognition detection and real-time tracking method provided above is implemented.

[0070] Since the electronic device introduced in this embodiment is an electronic device used to implement an object recognition detection and real-time tracking method in the embodiment of the present application, based on the object recognition detection and real-time tracking method introduced in the embodiment of the present application, the technical personnel of the field can understand the specific implementation of the electronic device of the present embodiment and its various variations, so how the electronic device implements the method in the embodiment of the present application is not described in detail here. As long as the technical personnel of the field implement the electronic device used in the object recognition detection and real-time tracking method in the embodiment of the present application, it belongs to the scope of protection of this application.

[0071] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.

[0072] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technical users in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. An object recognition detection and real-time tracking system, characterized in that: include: The image acquisition and processing module, the cluster analysis module and the detection and tracking module are connected to each other by wired and / or wireless means; The image acquisition and processing module includes an acquisition unit and a processing unit. The acquisition unit is used to acquire a video image sequence, and the processing unit is used to pre-process each frame of the video image sequence to obtain a plurality of frames of standard images. The cluster analysis module includes a model building unit and an analysis unit. The building unit builds an image directed graph model based on each frame of the standard image. The analysis unit is used to perform cluster analysis on the image directed graph model to obtain n potential target objects and corresponding feature descriptions. The detection and tracking module includes a detection unit and a tracking module. The detection unit detects the target object in the current frame standard image based on n potential target objects and corresponding feature descriptions to obtain the target object; The tracking module is used to track the detected target object.

2. The object recognition detection and real-time tracking system according to claim 1, characterized in that: The method of pre-processing comprises: Convert the image to a grayscale image, traverse each pixel in the grayscale image, count the number of occurrences of each grayscale value, and use an array of length 256 to store the number of occurrences of each grayscale value; draw a histogram based on the array; the horizontal axis of the histogram represents the grayscale value, and the vertical axis of the histogram represents the number of occurrences of the corresponding grayscale value; The number of occurrences of grayscale values ​​in the histogram is taken as the histogram data, and the histogram data is normalized; the normalized histogram data is fitted to obtain the mean of its Gaussian distribution and standard deviation ; Mean based on Gaussian distribution and standard deviation , calculate the peak height of the Gaussian distribution and peak width ; Peak height ;in, is the initial scale parameter, For time, is the preset growth rate; is the adaptive shape parameter; Adaptive shape parameters ;in, and is a constant coefficient, Represents at time Histogram data observed at each moment; is the kernel function; Peak width ; Count the number of pixels with grayscale value 0 s_0 and the number of pixels with grayscale value 255 s_255 in the histogram, and calculate the ratio of s_0 to s_255 as a noise intensity quantification index; Perform Fourier transform on the normalized histogram data to obtain the frequency spectrum, find the peak in the frequency spectrum, and count the peak amplitude and peak period of the peak; Preset the Gaussian threshold ranges corresponding to the peak height and the peak width, respectively. If the calculated peak height and peak width are both within the corresponding Gaussian threshold ranges, it is determined that Gaussian noise exists. A preset intensity threshold range is set. If the noise intensity quantification index is within the threshold range, it is determined that salt and pepper noise exists. Preset the period threshold ranges corresponding to the peak amplitude and the peak period, respectively. If the statistically obtained peak amplitude and peak period are within the corresponding period threshold ranges, it is determined that periodic noise exists. If there is Gaussian noise, use Gaussian filter or mean filter to filter, if there is salt and pepper noise, use median filter or adaptive median filter to filter; if there is periodic noise, perform adaptive special filtering; after filtering is completed, obtain a denoised image; perform image enhancement on the denoised image to obtain a standard image.

3. The object recognition detection and real-time tracking system according to claim 2, characterized in that: The method for performing adaptive special filtering includes: Perform a two-dimensional discrete Fourier transform on the image to convert it from the spatial domain to the frequency domain to obtain the two-dimensional frequency spectrum of the image ,in, represents the frequency in the horizontal direction, Indicates the frequency in the vertical direction; initially defines an adaptive band-stop filter and defines the transfer function of the adaptive band-stop filter ; ;in, Indicates frequency point The Euclidean distance to the center of the adaptive bandstop filter, is the normalization constant, and is the order control constant coefficient; Initialize normalization constant and the order control constant coefficient; and by collecting a training data set, using cross validation or grid search methods to optimize the normalization constant on the training data set and the order control constant coefficient, find the optimal normalization constant and a combination of order control constant coefficients; The two-dimensional frequency spectrum of the image Perform a point multiplication operation with the transfer function to obtain the two-dimensional frequency spectrum after filtering ;right A two-dimensional inverse discrete Fourier transform is performed to convert it from the frequency domain back to the spatial domain, and adaptive special filtering is completed.

4. The object recognition detection and real-time tracking system according to claim 3, characterized in that: The method of performing image enhancement on the denoised image includes: Defining Mapping Values , and construct a correction lookup table, for each input pixel value in the denoised image , calculate the corresponding output pixel value ;in, , and is a nonlinear parameter; the input pixel value and output pixel value The corresponding images are stored in a correction lookup table; a mapping operation is performed based on the correction lookup table to obtain a standard image; The mapping operation is performed as follows: for each pixel in the image, the correction lookup table is used for the three channels of R, G, and B to convert its pixel value from the input pixel value Map to output pixel values; recombine the mapped pixel values ​​into a new image, which is the standard image.

5. The object recognition detection and real-time tracking system according to claim 4, characterized in that: The mapping value The definitions include: Preset Mapping Values The search range is discretized into a series of discrete mapping value points, recorded as mapping points; the evaluation index function is defined ; ;in, , and is the weight coefficient of each item, is the maximum pixel value in the image after image enhancement, is the minimum pixel value in the image after image enhancement, is the total number of gray levels of the image after image enhancement, is the gray level index, For the The probability of gray levels, Represents the horizontal gradient of the image after image enhancement. Represents the vertical gradient of the image after image enhancement; Within the search range, a grid containing all mapping points is constructed, and for each mapping point, the value of the corresponding evaluation index function is initialized; each mapping point in the grid is traversed; for the current mapping point, it is used as the mapping value, and a mapping operation is performed to obtain an enhanced image, and the value of the evaluation index function of the enhanced image is calculated, and the value of the corresponding evaluation index function in the grid is updated; After traversing all grid points, find the mapping point with the largest value of the evaluation index function in the grid as the optimal mapping point, and build a new grid near the optimal mapping point; repeatedly search for the mapping point with the largest value of the evaluation index function in the new grid until the accuracy of the grid reaches the preset accuracy threshold; The calculation formula of the grid accuracy is: ;in, The mapping value corresponding to the optimal mapping point is The accuracy of the grid, is the upper limit of the search range, is the lower limit of the search range; is the number of intervals into which the search range is equally divided; The mapping value corresponding to the optimal mapping point is The value of the evaluation index function when for Relative to the mapped value The gradient of for Relative to the mapped value The second-order derivative of and is the equilibrium parameter, and ; is the density control parameter, is the curvature control parameter.

6. The object recognition detection and real-time tracking system according to claim 5, characterized in that: The construction method of the image directed graph model includes: The standard image is divided into several pixel blocks, and for each pixel block, its feature descriptor is extracted; the feature descriptor is a histogram of oriented gradients, a local binary pattern or a scale-invariant feature transform; For each pixel block in the standard image, treat it as a node in the image directed graph model and calculate the similarity between each pair of pixel blocks; According to the spatial position of the pixel blocks in the standard image, a directed edge is connected between each pair of adjacent pixel blocks; the direction of the directed edge is from left to right or from top to bottom; Assign a weight to each directed edge, and the weight value is equal to the similarity between the corresponding two pixel blocks; preset a similarity threshold, for two non-adjacent pixel blocks; if their similarity is greater than or equal to the similarity threshold, also connect a directed edge between the two, thus completing the construction of the directed graph model of the image.

7. The object recognition detection and real-time tracking system according to claim 6, characterized in that: Ways to divide a standard image into several pixel blocks include: Initialize a sliding window of a fixed size; start from the upper left corner of the standard image and slide the sliding window from left to right or from top to bottom on the image; For the area of ​​the standard image covered by the current sliding window, the texture complexity of the pixel values ​​inside it is calculated; the calculation method of texture complexity includes: For the area of ​​the standard image covered by the current sliding window, a gray-level co-occurrence matrix is ​​constructed, and the texture complexity is calculated based on the gray-level co-occurrence matrix. ; ;in, is the contrast of the area, is the entropy of the pixels in the region, is the gray level co-occurrence matrix Line The element value of the column, is the distance weighted kernel function; is the distance attenuation factor; is the pixel weighting function; Pixel weighting function ;in, For the The pixel corresponding to the row is The direction of the pixel corresponding to the column, is the preset main direction of interest; A uniform threshold is preset. If the texture complexity of the area is less than the uniform threshold, the current sliding window is regarded as a pixel block; If the texture complexity of the region is greater than or equal to the uniform threshold, the sliding window is segmented; The splitting is done as follows: The current sliding window is divided into four equal sub-windows in the horizontal and vertical directions. For each sub-window, it is repeatedly determined whether it needs to be split again until the texture complexity of all sub-windows is less than the uniform threshold. These sub-windows are regarded as pixel blocks; the sliding window is continued until the entire standard image is covered and several pixel blocks are obtained.

8. The object recognition detection and real-time tracking system according to claim 7, characterized in that: The cluster analysis method includes: Initialize the clustering parameters of cluster analysis, which include the number of clusters, the maximum number of local minimization q, the rate of change adjustment parameter b1, and the neighborhood adjustment parameter b2; Randomly select r nodes from the image directed graph model as the initial cluster centers; calculate the minimum distance from all nodes to the r cluster centers, and assign each node to the cluster where the nearest cluster center is located based on the calculated minimum distance; calculate the overall intra-cluster sum of squares of the current cluster and locally minimize the current cluster; The way to perform local minimization is to randomly select a node , remove it from its cluster, define the node Neighborhood ; Neighborhood Contains nodes Distance less than Nodes, where is the maximum distance between two nodes in the image directed graph model, in the neighborhood In the calculation, the node Redistribute the sum of squares in the population to the other r-1 clusters, choose the redistribution method that minimizes the sum of squares in the population, and repeat until q local minimizations are performed; in this process, dynamically adjust the size of b2 so that the neighborhood The size of is gradually reduced; if a cluster with a smaller overall sum of squares than the current cluster is found in the process of local minimization, the current cluster is replaced by this cluster; Repeat until the rate of change of the sum of squares within the population is less than b1, and finally r clusters are obtained, each cluster corresponds to a potential target object, and a total of n potential target objects are obtained. The value of r is equal to the value of n; For each cluster, the weighted average feature descriptor of all nodes in it is calculated as the feature description of the corresponding potential target object; The calculation formula of the weighted average feature descriptor is: ;in, For the The weighted average feature descriptor of clusters, is the corresponding node in the cluster The weight of For Node feature descriptor.

9. The object recognition detection and real-time tracking system according to claim 8, characterized in that: The method for detecting the target object includes: Define a detection sliding window, scan the current frame standard image using the detection sliding window, and generate candidate target regions; for each candidate target region, extract its feature descriptor, record it as the candidate region feature descriptor, and match the candidate region feature descriptor with the obtained feature descriptions of the n potential target objects; the matching method is: Calculate the distance between the feature descriptor of the candidate region and the weighted average feature descriptor of each potential target object as the matching score between the corresponding potential target object and the candidate target region; The potential target object and the candidate target area with the highest matching score are marked, that is, the corresponding candidate target area is marked as the potential target object; A confidence threshold is preset, and candidate target regions whose matching scores with all potential target objects are less than the confidence threshold are judged as non-target objects and eliminated; for several candidate target regions marked as the same potential target object, post-processing is performed to obtain the corresponding target object; Post-processing methods include: Based on the matching scores of several candidate target regions and the same potential target object, the aspect ratio of the candidate target region and the area of ​​the candidate target region, an optimal candidate target region is screened out and marked as the potential target object; Screening methods include: Set the matching score threshold, remove candidate target areas whose matching scores are greater than the matching score threshold, set the upper and lower bounds of the aspect ratio [as_min, as_max], and for the remaining candidate areas, remove candidate target areas whose aspect ratios are not within the upper and lower bounds [as_min, as_max]; Set the upper and lower bounds of the area [ar_min, ar_max], and remove candidate target areas whose areas are not within the upper and lower bounds [ar_min, ar_max]; The candidate target area with the largest matching score among the remaining candidate target areas is taken as the most suitable candidate target area.

10. The object recognition detection and real-time tracking system according to claim 9, characterized in that: The method of tracking the detected target object includes: Generate N2 particles, each particle represents a state of the target object, and assign an initial weight to each particle, which is usually set to 1 / N2; for each particle, predict the state at the next moment according to the motion model, that is, predict the particle; extract the observation feature corresponding to each predicted particle, and calculate the observation likelihood of each predicted particle, that is, the similarity between the observation feature and the motion model; update the weight of each particle according to the observation likelihood; calculate the optimal state of the target according to the states of all particles and the corresponding weights, and take the mean of the sum of weights as the optimal state; and resample; the resampling method is residual resampling, systematic resampling or stratified sampling; after resampling, the weights of all particles are reset to 1 / N2; repeat the tracking of the detected target object.

Images