A target tracking method in machine vision integrated system

Abstract

The present invention belongs to the field of computer vision technology and discloses a target tracking method in a machine vision integrated system; the method comprises: acquiring a video stream and performing frame processing on the video stream; performing target detection and feature extraction in an initial frame to acquire target information, the target information including a target position and target features; acquiring target information of a next frame based on the target information of a previous frame; comparing the target features of the previous frame with the target features of the next frame to determine whether the target is lost; if the target is not lost, not updating the target information of the next frame; if the target is lost, combining motion trajectory prediction and trajectory interpolation to update the target information of the next frame; the present invention realizes precise positioning and continuous tracking of targets in complex scenes, can effectively overcome interference factors such as occlusion and illumination changes, improves the robustness and real-time performance of target tracking, and enhances the intelligent perception capability of the machine vision system.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and more particularly to a target tracking method in a machine vision integrated system. Background Art

[0002] With the rapid development of intelligent technology, machine vision has been widely used in industrial automation, security monitoring, unmanned driving and other fields; target tracking, as one of the key technologies in machine vision, aims to monitor and analyze the motion state of the target in real time to provide support for system decision-making and control; in traditional vision systems, target detection and tracking often rely on static image processing, resulting in poor adaptability to dynamic scenes; with the advancement of deep learning and image processing technology, target tracking methods are gradually developing in a more efficient and accurate direction; these new methods can handle challenges such as complex backgrounds, occlusions and lighting changes, improving the robustness and real-time performance of target tracking.

[0003] The patent with publication number CN110163889A discloses a target tracking method, a target tracking device, and a target tracking equipment; including: performing target detection on the current frame of a video image to obtain a head detection frame set and a human body detection frame set; associating the head detection frame set with the human body detection frame set to obtain a target detection frame set in the current frame; determining a prediction frame set in the current frame based on the trajectory speed of each existing trajectory in the video image; and for each target detection frame in the target detection frame set, determining a matching prediction frame in the prediction frame set, and performing target tracking based on the matching result between the target detection frame and the prediction frame; this invention can effectively improve the target tracking accuracy in video images and realize real-time and high-precision target tracking.

[0004] However, although the above technology can achieve target tracking, it only relies on simple head detection frames and human body detection frames, and lacks the extraction and utilization of richer features such as target shape, color, and texture, which may lead to missed detection or false detection, thereby affecting the tracking effect; at the same time, the above technology predicts the frame set based on the speed of the existing trajectory. If the target motion trajectory changes significantly, it will lead to inaccurate prediction, which will affect the matching process; in addition, the above technology does not provide a special processing mechanism for target loss. Once the target is lost, it is difficult to resume tracking.

[0005] In view of this, the present invention proposes a target tracking method in a machine vision integrated system to solve the above problems. Summary of the Invention

[0006] In order to overcome the above-mentioned defects of the prior art and achieve the above-mentioned purpose, the present invention provides the following technical solution: a target tracking method in a machine vision integrated system, the method comprising:

[0007] S1: Obtain the video stream, extract the initial frame from the video stream, perform grayscale analysis on the initial frame, calculate the time interval, and divide the video stream into frames according to the time interval;

[0008] S2: Target detection and feature extraction are performed in the initial frame to obtain target information, including target position and target features;

[0009] S3: Track the target based on the target information of the previous frame, obtain the target position of the next frame, and calculate the target features of the next frame;

[0010] S4: Compare the target features of the previous frame with the target features of the next frame, calculate the tracking confidence, and determine whether the target is lost based on the tracking confidence; if the target is not lost, do not update the target information of the next frame;

[0011] S5: If the target is lost, the motion trajectory prediction method and trajectory interpolation algorithm are used to predict the target position of the next frame respectively, the prediction results are integrated into the trajectory, the target position of the next frame is updated, and the target features of the next frame are calculated based on the updated target position.

[0012] Furthermore, the video stream is a series of image frames arranged in chronological order;

[0013] The method for calculating the time interval includes:

[0014] Obtain an initial frame of the video stream, which is the zeroth frame of the video stream; grayscale the initial frame to obtain a grayscale image; preset a grayscale level, and perform a modulo operation on the grayscale value corresponding to each pixel in the grayscale image according to the grayscale level to obtain the grayscale level corresponding to each pixel; the modulo operation process is as follows: divide the grayscale value corresponding to each pixel in the grayscale image by the grayscale level, and use the corresponding integer part as the grayscale level of the corresponding pixel;

[0015] The grayscale levels corresponding to the horizontally adjacent pixels in the grayscale image are regarded as a set of grayscale sets, and the number of grayscale sets corresponding to each set is counted and marked as the frequency of occurrence; the size of the grayscale set is constructed The matrix, is the grayscale level, and different elements in the matrix correspond to a different set of grayscale sets; the value of each element in the matrix is replaced by the occurrence frequency of the corresponding grayscale set to obtain the grayscale co-occurrence matrix corresponding to the grayscale image; each element in the grayscale co-occurrence matrix is marked as , Indicates grayscale With grayscale The corresponding frequency of occurrence when horizontally adjacent in the grayscale image, , ;

[0016] According to the gray-level co-occurrence matrix of the gray-level image, the texture feature data corresponding to the gray-level image is calculated; according to the texture feature data corresponding to the gray-level image, the time interval is calculated; according to the time interval, the video stream is frame-processed.

[0017] Furthermore, the expression of contrast is: Where, is the contrast;

[0018] The expression of entropy is: Where, is entropy;

[0019] The expression for uniformity is: Where, for uniformity;

[0020] The expression for energy is: Where, for energy;

[0021] The expression for the time interval is: Where, is the time interval, 、 、 、 All are preset weight coefficients.

[0022] Furthermore, the method for obtaining the target position includes:

[0023] Using the trained target detection model, mark the target in the initial frame with a rectangular box; randomly obtain the coordinates of the three vertices corresponding to the rectangular box; add the vertical coordinates corresponding to the two vertices with the same horizontal coordinate, and then divide by 2 to obtain the target vertical coordinate; add the horizontal coordinates corresponding to the two vertices with the same vertical coordinate, and then divide by 2 to obtain the target horizontal coordinate; use the target horizontal coordinate and target vertical coordinate as the target position;

[0024] The target features include shape feature data, color feature data and texture statistics;

[0025] The shape feature data includes target perimeter and target convexity;

[0026] The method for obtaining the target perimeter is as follows: using an edge detection algorithm to detect the edge pixels corresponding to the target in the initial frame; obtaining the coordinates corresponding to each edge pixel; based on the coordinates of each edge pixel, calculating the Euclidean distance between each two adjacent edge pixels and marking them as adjacent distances; and adding each adjacent distance in sequence to obtain the target perimeter;

[0027] The target convexity is obtained by: using the convex hull algorithm to calculate the convex hull vertices corresponding to the initial frame; obtaining the coordinates of each convex hull vertex using the built-in coordinate system of the initial frame; calculating the Euclidean distance between each two adjacent convex hull vertices based on the coordinates of each convex hull vertex, and marking them as the convex hull distance; adding each convex hull distance in sequence to obtain the convex hull perimeter; dividing the target perimeter by the convex hull perimeter to obtain the target convexity;

[0028] The method for obtaining texture statistics is: constructing a gray-level co-occurrence matrix corresponding to the target image, and calculating the texture feature data corresponding to the target image; and using the texture feature data corresponding to the target image as texture statistics.

[0029] Furthermore, the color feature data includes a color histogram and an average color;

[0030] The methods for obtaining a color histogram include:

[0031] According to the detected edge pixels, the target image is segmented from the initial frame; the target image is converted from RGB color space to color space, Color spaces include aisle, Channel and Channel; get the corresponding channel of each pixel in the target image value, Values include value, Value and value; preset bucket size, divide 100 by the bucket size to get The bucket size corresponding to the channel is marked as Bucket size; Divide 255 by the bucket size to get Channel and The bucket size corresponding to the channel is marked as Bucket size; count the number of pixels in the target image and mark them as target points; calculate Channel Histogram 、 Channel Histogram and Channel Histogram , and as a color histogram;

[0032] Channel Histogram The expression is: Where, for Channel corresponds to The frequency of the bucket, is the target number, is the Dirac function, when When ,when When , is the ceiling function, The target image pixels value, for Bucket size, ;

[0033] Channel Histogram The expression is: Where, for Channel corresponds to The frequency of the bucket, The target image pixels value, for Bucket size;

[0034] Channel Histogram The expression is: Where, for Channel corresponds to The frequency of the bucket, The target image pixels value;

[0035] The method for obtaining the average color is: the corresponding average color of each pixel in the target image The values are added in sequence and then divided by the target number of points to get the average Value; the corresponding value of each pixel in the target image The values are added in sequence and then divided by the target number of points to get the average value; the corresponding value of each pixel in the target image The values are added in sequence and then divided by the target number of points to get the average value; average Value, average Value and average value as the average color.

[0036] Furthermore, the step of obtaining the target position of the next frame includes:

[0037] Step S301: Calculate the length and width of the rectangular frame, and use the target position of the previous frame as the center position to construct the current window;

[0038] Step S302: Mark all pixels in the current window that are not at the center as moving points; construct a window corresponding to each moving point based on the length, width, and moving point, and mark it as a moving window; calculate the color histogram corresponding to each moving window and mark it as moving data; mark the color histogram in the target information corresponding to the previous frame as target data;

[0039] Step S303: moving the current window according to the movement data and the target data, that is, updating the corresponding center position and calculating the corresponding position change;

[0040] Step S304: looping steps S302 to S303 until the position change is less than a preset change threshold, then the loop ends and the center position of the current window is used as the target position of the next frame.

[0041] Furthermore, in step S301, the method for calculating the length and width of the rectangular frame is as follows: randomly obtaining two vertices on the same diagonal line in the rectangular frame and marking them as diagonal points, obtaining coordinates corresponding to the diagonal points and marking them as diagonal coordinates; subtracting the smaller horizontal coordinate from the larger horizontal coordinate of the two diagonal coordinates to obtain the length; and subtracting the smaller vertical coordinate from the larger vertical coordinate of the two diagonal coordinates to obtain the width;

[0042] In step S303, the method for updating the center position corresponding to the current window includes:

[0043] ;

[0044] Where, The center position of the current window after updating. The center position of the current window before updating. For the The similarity between mobile data and target data, For the The moving point corresponding to the moving data, is the amount of mobile data, ;

[0045] The expression of position change is: Where, is the position change;

[0046] According to the current window corresponding to the target position of the next frame, the target features corresponding to the next frame are calculated.

[0047] Furthermore, the method for determining whether the target is lost includes:

[0048] The target features of the previous frame and the target features of the next frame are used as analysis data, and the analysis data is input into the trained tracking analysis model to predict the corresponding tracking confidence; a confidence threshold is preset, and the confidence threshold is weighted and smoothed to obtain a smoothing threshold; the tracking confidence is compared with the smoothing threshold; if the tracking confidence is less than the smoothing threshold, it is judged that the target is lost; if the tracking confidence is greater than or equal to the smoothing threshold, it is judged that the target is not lost;

[0049] The expression of the smoothing threshold is: Where, For the The corresponding smoothing threshold when judging whether the target is lost, is the scale factor, For the The corresponding tracking confidence when judging whether the target is lost, is the smoothing factor, For the The corresponding smooth confidence when judging whether the target is lost, is the confidence threshold;

[0050] The expression for the scale factor is: Where, is the minimum function, is the maximum value function, is the basic scale factor, is the sensitivity factor, For the The corresponding smooth confidence when judging whether the target is lost, ;

[0051] The expression of the smoothing factor is: Where, is the minimum value of the smoothing factor, is the maximum value of the smoothing factor, is an exponential function, is the amplitude factor, For the The corresponding tracking confidence when judging whether the target is lost.

[0052] Furthermore, the method for updating the target position of the next frame includes:

[0053] Get the historical position, which is the target position obtained at the historical moment; use the historical position and the target position corresponding to the previous frame as prediction data, input the prediction data into the trained position prediction model, predict the target position corresponding to the next frame, and mark it as the first predicted position, which includes the first horizontal coordinate and the first vertical coordinate; the position prediction model is an RNN neural network model:

[0054] Obtaining the frame corresponding to each target position in the predicted data and the timestamp corresponding to the next frame, and treating each target position in the predicted data and the corresponding acquisition time as a set of position sets, wherein the position sets correspond to the target positions one-to-one; calculating the target position corresponding to the next frame based on each set of position sets and the timestamp corresponding to the next frame, and marking it as a second predicted position; the second predicted position includes a second abscissa and a second ordinate;

[0055] Add the first horizontal coordinate to the second horizontal coordinate and divide the result by 2 to obtain an updated horizontal coordinate; add the first vertical coordinate to the second vertical coordinate and divide the result by 2 to obtain an updated vertical coordinate; update the target position of the next frame according to the updated horizontal coordinate and the updated vertical coordinate;

[0056] The method for calculating the target features of the next frame based on the updated target position is as follows: construct an update window based on the length and width of the rectangular frame and taking the updated target position of the next frame as the center position; and calculate the target features corresponding to the next frame based on the update window.

[0057] Furthermore, the training process of the location prediction model includes:

[0058] Pre-continuous acquisition target locations and construct a location training set, , To predict the number of target positions in the data; based on the position training set, train a position prediction model to predict the target position corresponding to the next frame;

[0059] Preset sliding step size J and sliding window length E, ; The target positions in the position training set are converted into multiple training samples using the sliding window method, and each training sample includes E target positions; each training sample is used as the input of the position prediction model, and the target position after the predicted sliding step J is used as the output. The actual J target positions corresponding to each training sample are used as the predicted targets. The prediction results are evaluated using the mean absolute percentage error (MAPE) to generate a position prediction model that predicts the target position corresponding to the next frame based on the target position;

[0060] The expression of the second horizontal axis is: Where, is the second horizontal axis, is the timestamp corresponding to the next frame, For the prediction data The timestamp corresponding to the frame, For the prediction data The timestamp corresponding to the frame, For the prediction data The target horizontal coordinate corresponding to the frame, , , is the number of frames in the prediction data;

[0061] The expression of the second ordinate is: Where, is the second vertical coordinate, For the prediction data The target vertical coordinate corresponding to the frame.

[0062] The technical effects and advantages of the target tracking method in a machine vision integrated system of the present invention are as follows:

[0063] Through target detection, feature extraction and dynamic tracking based on target features, accurate positioning and continuous tracking of targets in complex scenarios are achieved; the use of technologies such as video frame processing, color histogram matching and motion trajectory prediction can effectively overcome interference factors such as occlusion and lighting changes, and improve the robustness and real-time performance of target tracking; at the same time, deep learning technology is used to analyze and evaluate the tracking results, and the tracking threshold is dynamically adjusted to adaptively respond to complex target motion states, further improving tracking accuracy; it not only enhances the intelligent perception capability of the machine vision system, but also provides an effective target tracking solution for application scenarios such as intelligent monitoring and unmanned driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a flow chart of a target tracking method in a machine vision integrated system according to embodiment 1 of the present invention;

[0065] Figure 2 This is a flow chart of a method for obtaining a target position in the next frame according to embodiment 1 of the present invention. DETAILED DESCRIPTION

[0066] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0067] Example 1

[0068] See also Figure 1 As shown, the target tracking method in a machine vision integrated system described in this embodiment includes:

[0069] S1: Obtain the video stream, extract the initial frame from the video stream, perform grayscale analysis on the initial frame, calculate the time interval, and divide the video stream into frames according to the time interval.

[0070] The video stream is a series of image frames arranged in time sequence. The video stream is obtained by a machine vision integration system, and the machine vision integration system obtains the video stream through a camera.

[0071] Methods for calculating time intervals include:

[0072] Obtaining an initial frame of the video stream, which is the zeroth frame (first image) of the video stream, i.e., the frame captured when the video starts playing or processing; performing grayscale processing (such as weighted averaging, maximum value, Luma method, etc.) on the initial frame to obtain a grayscale image; presetting the number of grayscale levels, which is pre-set by those skilled in the art based on actual conditions; performing a modulo operation on the grayscale value corresponding to each pixel in the grayscale image based on the grayscale level to obtain the grayscale level corresponding to each pixel; the modulo operation comprises dividing the grayscale value corresponding to each pixel in the grayscale image by the grayscale level, and using the corresponding integer part as the grayscale level of the corresponding pixel;

[0073] The grayscale levels corresponding to the horizontally adjacent pixels in the grayscale image are regarded as a set of grayscale sets, and the number of grayscale sets corresponding to each set is counted and marked as the frequency of occurrence; the size of the grayscale set is constructed The matrix, is the grayscale level, and different elements in the matrix correspond to a different set of grayscale sets; the value of each element in the matrix is replaced by the occurrence frequency of the corresponding grayscale set to obtain the grayscale co-occurrence matrix corresponding to the grayscale image; each element in the grayscale co-occurrence matrix is marked as , Indicates grayscale With grayscale The corresponding frequency of occurrence when horizontally adjacent in the grayscale image, , .

[0074] For example, the grayscale level corresponding to the grayscale image is , where the grayscale set corresponding to the first row is and , the grayscale set corresponding to the second row is and , the grayscale set corresponding to the third row is and Grayscale set The corresponding frequency of occurrence is 2, that is, the first row and second column in the gray-level co-occurrence matrix is 2; the gray-level set The corresponding frequency of occurrence is 2, that is, the second row and third column in the gray level co-occurrence matrix is 2; the gray level set The corresponding frequency of occurrence is 1, that is, the second row and first column in the gray level co-occurrence matrix is 1; the gray level set The corresponding frequency of occurrence is 1, that is, the third row and second column in the gray-level co-occurrence matrix is 1; since the rest of the gray-level sets do not exist, the corresponding frequency of occurrence is 0, that is, the value of the corresponding element in the gray-level co-occurrence matrix is 0; therefore, the gray-level co-occurrence matrix corresponding to the gray-level image is .

[0075] According to the gray-level co-occurrence matrix of the gray-level image, texture feature data corresponding to the gray-level image is calculated; and according to the texture feature data corresponding to the gray-level image, a time interval is calculated.

[0076] Texture feature data include contrast, entropy, uniformity and energy;

[0077] The expression for contrast is: Where, is the contrast;

[0078] The expression of entropy is: Where, is entropy;

[0079] The expression for uniformity is: Where, for uniformity;

[0080] The expression for energy is: Where, For energy.

[0081] The expression for the time interval is: Where, is the time interval, 、 、 、 All are preset weight coefficients;

[0082] The specific value of the weight coefficient in the formula can be set according to actual conditions. The weight coefficient reflects the degree of influence of texture feature data on the time interval. Those skilled in the art can preset the corresponding weight coefficient according to the actual degree of influence of texture feature data on the time interval, so as to accurately adjust the time interval in the frame processing, thereby optimizing the efficiency and accuracy of the frame processing.

[0083] It should be noted that the greater the contrast, the more obvious the change in grayscale levels in the grayscale image, and the more complex the texture of the initial frame, and vice versa; the greater the entropy, the richer the amount of information contained in the grayscale image, and the more complex the texture of the initial frame, and vice versa; the greater the uniformity, the smaller the change between grayscale values in the grayscale image, and the simpler the texture of the initial frame, and vice versa; the lower the energy, the more irregular the texture in the grayscale image, and the more complex the texture of the initial frame, and vice versa; complex textures usually contain more details and changes, so in order to capture these details and changes more frequently and effectively, it is necessary to reduce the time interval to avoid missing important details and changes, which helps to improve the accuracy of subsequent analysis.

[0084] S2: Perform target detection and feature extraction in the initial frame to obtain target information, which includes target position and target features.

[0085] Methods for obtaining target locations include:

[0086] Using the trained target detection model, the target in the initial frame is marked with a rectangular box; the three vertex coordinates corresponding to the rectangular box are randomly obtained through the built-in coordinate system of the initial frame; the vertical coordinates corresponding to the two vertex coordinates with the same horizontal coordinate are added, and then divided by 2 to obtain the target vertical coordinate; the horizontal coordinates corresponding to the two vertex coordinates with the same vertical coordinate are added, and then divided by 2 to obtain the target horizontal coordinate; the target horizontal coordinate and the target vertical coordinate are used as the target position.

[0087] The training methods for the target detection model include:

[0088] Prefetch The initial frame includes the target, is an integer greater than 1; The initial frame containing the target is marked as a training image; the target in the training image is annotated with a rectangular frame; the annotated training image is divided into a training set and a test set, with 70% of the training images used as the training set and 30% of the training images used as the test set. The target detection model is trained using the training set and tested using the test set. A coordinate error threshold is preset. When the mean value of the predicted coordinate errors of all training images in the test set is less than the coordinate error threshold, the target detection model training is completed and the target detection model is output. The calculation formula for the predicted coordinate error includes:

[0089] ;

[0090] Where, is the predicted coordinate error, is the number of the training image, is the number of training images in the test set, For the In the image The predicted horizontal coordinates of the vertices, For the In the image The actual horizontal coordinates of the vertices, For the In the image The predicted vertical coordinates of the vertices, For the In the image The actual vertical coordinates of the vertices. The rectangular box has 4 vertices in total. , ; The coordinate error threshold is pre-set according to the accuracy required by the target detection model;

[0091] The above target detection model is specifically a convolutional neural network model.

[0092] Methods for obtaining target features include:

[0093] Target features include shape feature data, color feature data and texture statistics.

[0094] The shape feature data includes the target perimeter and target convexity;

[0095] The method for obtaining the target perimeter is as follows: use an edge detection algorithm (such as Canny edge detection, Sobel operator, Prewitt operator, etc.) to detect the edge pixels corresponding to the target in the initial frame; obtain the coordinates corresponding to each edge pixel through the built-in coordinate system of the initial frame; calculate the Euclidean distance between each two adjacent edge pixels based on the coordinates of each edge pixel, and mark them as adjacent distances; add each adjacent distance in sequence to obtain the target perimeter.

[0096] The target convexity is obtained by using a convex hull algorithm (such as Graham's scan method, Jarvis's walk method, or Andrew's algorithm) to calculate the convex hull vertices corresponding to the initial frame. The coordinates of each convex hull vertex are obtained using the initial frame's built-in coordinate system. Based on the coordinates of each convex hull vertex, the Euclidean distance between each two adjacent convex hull vertices is calculated and marked as the convex hull distance. Each convex hull distance is added together to obtain the convex hull perimeter. The target convexity is obtained by dividing the target perimeter by the convex hull perimeter.

[0097] Color feature data includes color histogram and average color;

[0098] The methods for obtaining the color histogram include:

[0099] According to the detected edge pixels, the target image is segmented from the initial frame; the target image is converted from RGB color space to color space, Color spaces include aisle, Channel and Channel, color space conversion method is an existing technology, which will not be described in detail here; get the corresponding color space of each pixel in the target image. value, Values include (brightness) value, (Red-Green component) value and (yellow-blue component) value; preset bucket volume, which is preset by technical personnel in this field according to actual conditions; divide 100 by the bucket volume to obtain The bucket size corresponding to the channel is marked as Bucket size, where each bucket represents a brightness range; divide 255 by the bucket size to get Channel and The bucket size corresponding to the channel is marked as Bucket size, where each bucket represents a red-green component interval or a yellow-blue component interval; count the number of pixels in the target image and mark them as target points; calculate Channel Histogram 、 Channel Histogram and Channel Histogram , and as a color histogram;

[0100] Channel Histogram The expression is: Where, for Channel corresponds to The frequency of the bucket, is the target number, is the Dirac function, when When ,when When , is the ceiling function, The target image pixels value, for Bucket size, ;

[0101] Channel Histogram The expression is: Where, for Channel corresponds to The frequency of the bucket, The target image pixels value, for Bucket size;

[0102] Channel Histogram The expression is: Where, for Channel corresponds to The frequency of the bucket, The target image pixels value.

[0103] The method for obtaining the average color is: the corresponding average color of each pixel in the target image The values are added in sequence and then divided by the target number of points to get the average value; the corresponding value of each pixel in the target image The values are added in sequence and then divided by the target number of points to get the average value; the corresponding value of each pixel in the target image The values are added in sequence and then divided by the target number of points to get the average value; average Value, average Value and average value as the average color.

[0104] The method for obtaining texture statistics is: constructing a gray-level co-occurrence matrix corresponding to the target image, and calculating the texture feature data corresponding to the target image; and using the texture feature data corresponding to the target image as texture statistics.

[0105] S3: Track the target based on the target information of the previous frame, obtain the target position of the next frame, and calculate the target features of the next frame.

[0106] The steps to obtain the target position of the next frame include:

[0107] Step S301: Calculate the length and width of the rectangular frame, and use the target position of the previous frame as the center position to construct the current window;

[0108] Step S302: Mark all pixels in the current window that are not at the center as moving points; construct a window corresponding to each moving point based on the length, width, and moving point, and mark it as a moving window; calculate the color histogram corresponding to each moving window and mark it as moving data; mark the color histogram in the target information corresponding to the previous frame as target data;

[0109] Step S303: moving the current window according to the movement data and the target data, that is, updating the corresponding center position and calculating the corresponding position change;

[0110] Step S304: loop steps S302 to S303 until the position change is less than a preset change threshold, then the loop ends and the center position of the current window is used as the target position of the next frame; the change threshold is pre-set by those skilled in the art according to actual conditions.

[0111] In the above step S301, the method for calculating the length and width of the rectangular box is: randomly obtain two vertices on the same diagonal line in the rectangular box, mark them as diagonal points, obtain the coordinates corresponding to the diagonal points, and mark them as diagonal coordinates; subtract the horizontal coordinate with the smaller value from the horizontal coordinate with the larger value of the two diagonal coordinates to obtain the length; subtract the vertical coordinate with the smaller value from the vertical coordinate with the larger value of the two diagonal coordinates to obtain the width.

[0112] In the above step S303, the method for updating the center position corresponding to the current window includes:

[0113] ;

[0114] Where, The center position of the current window after updating. The center position of the current window before updating. For the The similarity between the mobile data and the target data is calculated using similarity calculation methods (such as Bhattacharyya distance, chi-square distance, etc.). For the The moving point corresponding to the moving data, is the amount of mobile data, .

[0115] The expression of position change is: Where, is the position change.

[0116] According to the current window corresponding to the target position of the next frame, the target features corresponding to the next frame are calculated.

[0117] S4: Compare the target features of the previous frame with the target features of the next frame, calculate the tracking confidence, and determine whether the target is lost based on the tracking confidence; if the target is not lost, do not update the target information of the next frame.

[0118] Methods for determining whether the target is lost include:

[0119] The target features of the previous frame and the target features of the next frame are used as analysis data, and the analysis data are input into the trained tracking analysis model to predict the corresponding tracking confidence; a confidence threshold is preset, and the confidence threshold is preset by technical personnel in this field according to actual conditions; the confidence threshold is weighted and smoothed to obtain a smoothing threshold; the tracking confidence is compared with the smoothing threshold; if the tracking confidence is less than the smoothing threshold, it is judged that the target is lost, indicating that the reliability of target tracking is low, and it is necessary to re-acquire the target information of the next frame; if the tracking confidence is greater than or equal to the smoothing threshold, it is judged that the target is not lost, indicating that the reliability of target tracking is high, and there is no need to re-acquire the target information of the next frame.

[0120] The training process of the tracking analysis model includes:

[0121] Pre-collection Group analysis data, The corresponding tracking confidence level is set for each group of analysis data. is an integer greater than 1, and the analysis data and the corresponding tracking confidence are converted into a corresponding set of feature vectors; the tracking confidence corresponding to the analysis data is collected by those skilled in the art during the historical target tracking process. The target features of the next frame are obtained under the condition of the corresponding target features of the previous frame in each set of analysis data, and compared with the target features of the next frame in the corresponding analysis data, and the corresponding tracking confidence is analyzed and used as the tracking confidence of the corresponding analysis data. The corresponding tracking confidence levels are set for each group of analysis data in turn;

[0122] Each set of feature vectors is used as the input of the tracking analysis model. The tracking analysis model outputs a set of predicted tracking confidences corresponding to each set of analysis data. The actual tracking confidence corresponding to each set of analysis data is used as the prediction target. The actual tracking confidence is the preset tracking confidence corresponding to the analysis data. The training goal is to minimize the sum of the prediction errors of all analysis data. The calculation formula for the prediction error is: Where, is the prediction error, is the group number of the eigenvector corresponding to the analyzed data, For the The prediction tracking confidence corresponding to the group analysis data, For the The actual tracking confidence corresponding to the group analysis data; the tracking analysis model is trained until the sum of the prediction errors reaches convergence and the training is stopped.

[0123] The above-mentioned tracking and analysis model is specifically a deep neural network model; it includes an input layer, a hidden layer and an output layer; each hidden layer includes multiple neurons, each neuron is connected to the neurons in the next layer, and the connection contains weights, which determine the importance and influence of data transmission in the neural network; an activation function is applied to each neuron between the hidden layer and the output layer. The activation function introduces nonlinearity, allowing the network to learn more complex patterns and features.

[0124] The expression of the smoothing threshold is: Where, For the The corresponding smoothing threshold when judging whether the target is lost, is the scale factor, For the The corresponding tracking confidence when judging whether the target is lost, is the smoothing factor, For the The corresponding smooth confidence when judging whether the target is lost, is the confidence threshold.

[0125] The expression for the scale factor is: Where, is the minimum function, is the maximum value function, is the basic scale factor, is the sensitivity factor, For the The corresponding smooth confidence when judging whether the target is lost, The basic proportional factor and sensitivity factor are preset by those skilled in the art according to actual conditions.

[0126] The expression of the smoothing factor is: Where, is the minimum value of the smoothing factor, is the maximum value of the smoothing factor, is an exponential function, is the amplitude factor, For the The corresponding tracking confidence when determining whether the target is lost; the minimum value of the smoothing factor, the maximum value of the smoothing factor and the amplitude factor are pre-set by those skilled in the art according to actual conditions.

[0127] S5: If the target is lost, the motion trajectory prediction method and trajectory interpolation algorithm are used to predict the target position of the next frame respectively, the prediction results are integrated into the trajectory, the target position of the next frame is updated, and the target features of the next frame are calculated based on the updated target position.

[0128] Methods for updating the target position for the next frame include:

[0129] Obtain the historical position, which is the target position obtained at the historical moment; use the historical position and the target position corresponding to the previous frame as prediction data, input the prediction data into the trained position prediction model, predict the target position corresponding to the next frame, and mark it as the first predicted position, which includes the first horizontal coordinate and the first vertical coordinate; the position prediction model is an RNN neural network model, and the training process of the position prediction model includes:

[0130] Pre-continuous acquisition target locations and build a location training set, , To predict the number of target positions in the data; based on the position training set, train a position prediction model to predict the target position corresponding to the next frame;

[0131] According to the actual experience of those skilled in the art, the sliding step size J and the sliding window length E are preset. ; The target positions in the position training set are converted into multiple training samples using the sliding window method, and each training sample includes E target positions; each training sample is used as the input of the position prediction model, and the target position after the predicted sliding step J is used as the output. The actual J target positions corresponding to each training sample are used as the prediction targets. The prediction results are evaluated using the mean absolute percentage error (MAPE) for model accuracy. When the calculated MAPE is less than the preset MAPE, the position prediction model training corresponding to the target position is completed; the calculation formula of the mean absolute percentage error (MAPE) is: Where, For the The predicted target position corresponds to the predicted target, For the The predicted target location, is the number of predicted target positions; a position prediction model is generated to predict the target position corresponding to the next frame based on the target position.

[0132] For example, the position training set A contains 10 target positions. , For the target locations, , define the length of the sliding window as 3, use the sliding window to construct 7 training samples, each training sample contains 3 consecutive target positions, and the next target position of the 3 consecutive target positions is used as the prediction target; for example: training sample , training samples The corresponding prediction target is ; Training samples , training samples The corresponding prediction target is ; And so on, it is used to train the location prediction model.

[0133] Obtain the frame corresponding to each target position in the predicted data and the timestamp corresponding to the next frame, and use each target position in the predicted data and the corresponding acquisition time as a group of position sets, and the position sets and target positions correspond one-to-one; calculate the target position corresponding to the next frame based on each group of position sets and the timestamp corresponding to the next frame, and mark it as the second predicted position; the second predicted position includes a second horizontal coordinate and a second vertical coordinate.

[0134] The expression of the second horizontal axis is: Where, is the second horizontal axis, is the timestamp corresponding to the next frame, For the prediction data The timestamp corresponding to the frame, For the prediction data The timestamp corresponding to the frame, For the prediction data The target horizontal coordinate corresponding to the frame, , , is the number of frames in the prediction data.

[0135] The expression of the second ordinate is: Where, is the second vertical coordinate, For the prediction data The target vertical coordinate corresponding to the frame.

[0136] Add the first horizontal coordinate to the second horizontal coordinate and divide the result by 2 to obtain an updated horizontal coordinate; add the first vertical coordinate to the second vertical coordinate and divide the result by 2 to obtain an updated vertical coordinate; update the target position of the next frame according to the updated horizontal coordinate and the updated vertical coordinate;

[0137] The method for calculating the target features of the next frame based on the updated target position is as follows: construct an update window based on the length and width of the rectangular frame and taking the updated target position of the next frame as the center position; and calculate the target features corresponding to the next frame based on the update window.

[0138] This embodiment achieves precise positioning and continuous tracking of targets in complex scenarios through target detection, feature extraction, and dynamic tracking based on target features. It utilizes technologies such as video frame processing, color histogram matching, and motion trajectory prediction to effectively overcome interference factors such as occlusion and illumination changes, thereby improving the robustness and real-time performance of target tracking. At the same time, deep learning technology is used to analyze and evaluate tracking results and dynamically adjust tracking thresholds to adaptively respond to complex target motion states, further improving tracking accuracy. This not only enhances the intelligent perception capabilities of machine vision systems, but also provides an effective target tracking solution for application scenarios such as intelligent monitoring and unmanned driving.

[0139] Example 2

[0140] The present application also provides an electronic device. The electronic device may include one or more processors and one or more memories. The memories may store computer-readable code that, when executed by the one or more processors, may execute the target tracking method in a machine vision integrated system as described above.

[0141] The method or system according to the embodiment of the present application can also be implemented with the aid of the architecture of the electronic device shown in the present application. The electronic device may include a bus, one or more CPUs, ROM, RAM, a communication port connected to a network, input / output, a hard disk, etc. The storage device in the electronic device, such as a ROM or a hard disk, can store a target tracking method in a machine vision integrated system provided in the present application. Furthermore, the electronic device may also include a user interface. Of course, the architecture shown in the present application is only exemplary. When implementing different devices, one or more components in the electronic device shown in the present application may be omitted according to actual needs.

[0142] Example 3

[0143] As shown, one embodiment of the present application discloses a computer-readable storage medium. The computer-readable storage medium stores computer-readable instructions. When the computer-readable instructions are executed by a processor, the target tracking method in a machine vision integration system according to the embodiment of the present application described with reference to the above figures can be executed. The storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc.

[0144] Furthermore, according to embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the present application provides a non-transitory machine-readable storage medium storing machine-readable instructions capable of being executed by a processor to perform the steps corresponding to the methods provided herein, such as a method for object tracking in a machine vision integration system. When executed by a central processing unit (CPU), this computer program performs the functions defined in the methods of the present application.

[0145] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

[0146] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A target tracking method in a machine vision integrated system, characterized in that: Methods include: S1: Obtain the video stream, extract the initial frame from the video stream, perform grayscale analysis on the initial frame, calculate the time interval, and divide the video stream into frames according to the time interval; The video stream is a series of image frames arranged in time sequence; The method for calculating the time interval includes: Obtain an initial frame of the video stream, which is the zeroth frame of the video stream; grayscale the initial frame to obtain a grayscale image; preset a grayscale level, and perform a modulo operation on the grayscale value corresponding to each pixel in the grayscale image according to the grayscale level to obtain the grayscale level corresponding to each pixel; the modulo operation process is as follows: divide the grayscale value corresponding to each pixel in the grayscale image by the grayscale level, and use the corresponding integer part as the grayscale level of the corresponding pixel; The grayscale levels corresponding to the horizontally adjacent pixels in the grayscale image are regarded as a set of grayscale sets, and the number of grayscale sets corresponding to each set is counted and marked as the frequency of occurrence; the size of the grayscale set is constructed The matrix, is the grayscale level, and different elements in the matrix correspond to a different set of grayscale sets; the value of each element in the matrix is replaced by the occurrence frequency of the corresponding grayscale set to obtain the grayscale co-occurrence matrix corresponding to the grayscale image; each element in the grayscale co-occurrence matrix is marked as , Indicates grayscale With grayscale The corresponding frequency of occurrence when horizontally adjacent in the grayscale image, , ; Calculating texture feature data corresponding to the grayscale image based on the grayscale co-occurrence matrix of the grayscale image; calculating the time interval based on the texture feature data corresponding to the grayscale image; The expression for the time interval is: Where, is the time interval, is the contrast, is entropy, For uniformity, For energy, 、 、 、 All are preset weight coefficients; The expression for contrast is: ; The expression of entropy is: ; The expression for uniformity is: ; The expression for energy is: ; S2: Target detection and feature extraction are performed in the initial frame to obtain target information, including target position and target features; S3: Track the target based on the target information of the previous frame, obtain the target position of the next frame, and calculate the target features of the next frame; S4: Compare the target features of the previous frame with the target features of the next frame, calculate the tracking confidence, and determine whether the target is lost based on the tracking confidence; if the target is not lost, do not update the target information of the next frame; S5: If the target is lost, the motion trajectory prediction method and trajectory interpolation algorithm are used to predict the target position of the next frame respectively, the prediction results are integrated into the trajectory, the target position of the next frame is updated, and the target features of the next frame are calculated based on the updated target position.

2. The target tracking method in a machine vision integrated system according to claim 1, characterized in that: Methods for obtaining target locations include: Using the trained target detection model, mark the target in the initial frame with a rectangular box; randomly obtain the coordinates of the three vertices corresponding to the rectangular box; add the vertical coordinates corresponding to the two vertices with the same horizontal coordinate, and then divide by 2 to obtain the target vertical coordinate; add the horizontal coordinates corresponding to the two vertices with the same vertical coordinate, and then divide by 2 to obtain the target horizontal coordinate; use the target horizontal coordinate and target vertical coordinate as the target position; The target features include shape feature data, color feature data and texture statistics; The shape feature data includes target perimeter and target convexity; The method for obtaining the target perimeter is as follows: using an edge detection algorithm to detect the edge pixels corresponding to the target in the initial frame; obtaining the coordinates corresponding to each edge pixel; based on the coordinates of each edge pixel, calculating the Euclidean distance between each two adjacent edge pixels and marking them as adjacent distances; and adding each adjacent distance in sequence to obtain the target perimeter; The target convexity is obtained by: using the convex hull algorithm to calculate the convex hull vertices corresponding to the initial frame; obtaining the coordinates of each convex hull vertex using the built-in coordinate system of the initial frame; calculating the Euclidean distance between each two adjacent convex hull vertices based on the coordinates of each convex hull vertex, and marking them as the convex hull distance; adding each convex hull distance in sequence to obtain the convex hull perimeter; dividing the target perimeter by the convex hull perimeter to obtain the target convexity; The method for obtaining texture statistics is: constructing a gray-level co-occurrence matrix corresponding to the target image, and calculating the texture feature data corresponding to the target image; and using the texture feature data corresponding to the target image as texture statistics.

3. The target tracking method in a machine vision integrated system according to claim 2, characterized in that: The color feature data includes a color histogram and an average color; The methods for obtaining a color histogram include: Segment the target image from the initial frame based on the detected edge pixels; Use the color space conversion method to convert the target image from RGB color space to color space, Color spaces include aisle, Channel and Channel; get the corresponding channel of each pixel in the target image value, Values include value, Value and value; preset bucket size, divide 100 by the bucket size to get The bucket size corresponding to the channel is marked as Bucket size; Divide 255 by the bucket size to get Channel and The bucket size corresponding to the channel is marked as Bucket size; count the number of pixels in the target image and mark them as target points; calculate Channel Histogram 、 Channel Histogram and Channel Histogram , and as a color histogram; Channel Histogram The expression is: Where, for Channel corresponds to The frequency of the bucket, is the target number, is the Dirac function, when When ,when When , is the ceiling function, The target image pixels value, for Bucket size, ; Channel Histogram The expression is: Where, for Channel corresponds to The frequency of the bucket, The target image pixels value, for Bucket size; Channel Histogram The expression is: Where, for Channel corresponds to The frequency of the bucket, The target image pixels value; The method for obtaining the average color is: the corresponding average color of each pixel in the target image The values are added in sequence and then divided by the target number of points to get the average value; the corresponding value of each pixel in the target image The values are added in sequence and then divided by the target number of points to get the average value; the corresponding value of each pixel in the target image The values are added in sequence and then divided by the target number of points to get the average value; average Value, average Value and average value as the average color.

4. The target tracking method in a machine vision integrated system according to claim 3, characterized in that: The step of obtaining the target position of the next frame includes: Step S301: Calculate the length and width of the rectangular frame, and use the target position of the previous frame as the center position to construct the current window; Step S302: Mark all pixels in the current window that are not at the center as moving points; construct a window corresponding to each moving point based on the length, width, and moving point, and mark it as a moving window; calculate the color histogram corresponding to each moving window and mark it as moving data; mark the color histogram in the target information corresponding to the previous frame as target data; Step S303: moving the current window according to the movement data and the target data, that is, updating the corresponding center position and calculating the corresponding position change; Step S304: looping steps S302 to S303 until the position change is less than a preset change threshold, then the loop ends and the center position of the current window is used as the target position of the next frame.

5. The target tracking method in a machine vision integrated system according to claim 4, characterized in that: In step S301, the method for calculating the length and width of the rectangular frame is as follows: randomly obtain two vertices on the same diagonal line in the rectangular frame and mark them as diagonal points, obtain coordinates corresponding to the diagonal points and mark them as diagonal coordinates; subtract the smaller horizontal coordinate from the larger horizontal coordinate of the two diagonal coordinates to obtain the length; and subtract the smaller vertical coordinate from the larger vertical coordinate of the two diagonal coordinates to obtain the width; In step S303, the method for updating the center position corresponding to the current window includes: ； Where, The center position of the current window after updating. The center position of the current window before updating. For the The similarity between mobile data and target data, For the The moving point corresponding to the moving data, is the amount of mobile data, ; The expression of position change is: Where, is the position change; According to the current window corresponding to the target position of the next frame, the target features corresponding to the next frame are calculated.

6. The target tracking method in a machine vision integrated system according to claim 5, characterized in that: Methods for determining whether the target is lost include: The target features of the previous frame and the target features of the next frame are used as analysis data, and the analysis data is input into the trained tracking analysis model to predict the corresponding tracking confidence; a confidence threshold is preset, and the confidence threshold is weighted and smoothed to obtain a smoothing threshold; the tracking confidence is compared with the smoothing threshold; if the tracking confidence is less than the smoothing threshold, it is judged that the target is lost; if the tracking confidence is greater than or equal to the smoothing threshold, it is judged that the target is not lost; The expression of the smoothing threshold is: Where, For the The corresponding smoothing threshold when judging whether the target is lost, is the scale factor, For the The corresponding tracking confidence when judging whether the target is lost, is the smoothing factor, For the The corresponding smooth confidence when judging whether the target is lost, is the confidence threshold; The expression for the scale factor is: Where, is the minimum function, is the maximum value function, is the basic scale factor, is the sensitivity factor, For the The corresponding smooth confidence when judging whether the target is lost, ; The expression of the smoothing factor is: Where, is the minimum value of the smoothing factor, is the maximum value of the smoothing factor, is an exponential function, is the amplitude factor, For the The corresponding tracking confidence when judging whether the target is lost.

7. The target tracking method in a machine vision integrated system according to claim 6, characterized in that: The method for updating the target position of the next frame includes: Get the historical position, which is the target position obtained at the historical moment; use the historical position and the target position corresponding to the previous frame as prediction data, input the prediction data into the trained position prediction model, predict the target position corresponding to the next frame, and mark it as the first predicted position, which includes the first horizontal coordinate and the first vertical coordinate; the position prediction model is an RNN neural network model: Obtaining the frame corresponding to each target position in the predicted data and the timestamp corresponding to the next frame, and treating each target position in the predicted data and the corresponding acquisition time as a set of position sets, wherein the position sets correspond to the target positions one-to-one; calculating the target position corresponding to the next frame based on each set of position sets and the timestamp corresponding to the next frame, and marking it as a second predicted position; the second predicted position includes a second abscissa and a second ordinate; Add the first horizontal coordinate to the second horizontal coordinate and divide the result by 2 to obtain an updated horizontal coordinate; add the first vertical coordinate to the second vertical coordinate and divide the result by 2 to obtain an updated vertical coordinate; update the target position of the next frame according to the updated horizontal coordinate and the updated vertical coordinate; The method for calculating the target features of the next frame based on the updated target position is as follows: construct an update window based on the length and width of the rectangular frame and taking the updated target position of the next frame as the center position; and calculate the target features corresponding to the next frame based on the update window.

8. The target tracking method in a machine vision integrated system according to claim 7, characterized in that: The training process of the location prediction model includes: Pre-continuous acquisition target locations and construct a location training set, , To predict the number of target positions in the data; based on the position training set, train a position prediction model to predict the target position corresponding to the next frame; Preset sliding step size J and sliding window length E, ; The target positions in the position training set are converted into multiple training samples using the sliding window method, and each training sample includes E target positions; each training sample is used as the input of the position prediction model, and the target position after the predicted sliding step J is used as the output. The actual J target positions corresponding to each training sample are used as the predicted targets. The prediction results are evaluated using the mean absolute percentage error (MAPE) to generate a position prediction model that predicts the target position corresponding to the next frame based on the target position; The expression of the second horizontal axis is: Where, is the second horizontal axis, is the timestamp corresponding to the next frame, For the prediction data The timestamp corresponding to the frame, For the prediction data The timestamp corresponding to the frame, For the prediction data The target horizontal coordinate corresponding to the frame, , , is the number of frames in the prediction data; The expression of the second ordinate is: Where, is the second vertical coordinate, For the prediction data The target vertical coordinate corresponding to the frame.

Images