Target tracking method in machine vision integrated system

Abstract

The invention belongs to the technical field of computer vision, and discloses a target tracking method in a machine vision integrated system. The method comprises the following steps: acquiring a video stream, and performing framing processing on the video stream; target detection and feature extraction are carried out in the initial frame, target information is obtained, and the target information comprises a target position and a target feature; obtaining the target information of the next frame according to the target information of the previous frame; comparing the target feature of the previous frame with the target feature of the next frame, and judging whether the target is lost or not; if the target is not lost, the target information of the next frame is not updated; if the target is lost, the target information of the next frame is updated in combination with motion trail prediction and trail interpolation; according to the method, accurate positioning and continuous tracking of the target in a complex scene are realized, interference factors such as shielding and illumination variation can be effectively overcome, the robustness and the real-time performance of target tracking are improved, and the intelligent sensing capability of a machine vision system is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and more specifically, to a target tracking method in a machine vision integrated system. Background Art

[0002] With the rapid development of intelligent technology, machine vision has been widely used in industrial automation, security monitoring, unmanned driving and other fields; target tracking, as one of the key technologies in machine vision, aims to monitor and analyze the motion state of the target in real time to provide support for system decision-making and control; in traditional visual systems, target detection and tracking often rely on static image processing, resulting in poor adaptability to dynamic scenes; with the advancement of deep learning and image processing technology, target tracking methods are gradually developing in a more efficient and accurate direction; these new methods can handle challenges such as complex backgrounds, occlusions and lighting changes, and improve the robustness and real-time performance of target tracking.

[0003] The patent with publication number CN110163889A discloses a target tracking method, a target tracking device, and a target tracking equipment; including: performing target detection on the current frame of a video image to obtain a head detection frame set and a human body detection frame set; associating the head detection frame set with the human body detection frame set to obtain a target detection frame set in the current frame; determining a prediction frame set in the current frame according to the trajectory speed of each existing trajectory in the video image; and for each target detection frame in the target detection frame set, determining a prediction frame matching it in the prediction frame set, and performing target tracking based on the matching result of the target detection frame and the prediction frame; this invention can effectively improve the target tracking accuracy in video images and realize real-time and high-precision target tracking.

[0004] However, although the above technology can achieve target tracking, it only relies on simple head detection frames and human body detection frames, and lacks the extraction and utilization of richer features such as target shape, color, texture, etc., which will lead to missed detection or false detection, thereby affecting the tracking effect; at the same time, the above technology predicts the frame set through the speed of the existing trajectory. If the target motion trajectory changes greatly, it will lead to inaccurate prediction, which will affect the matching process; in addition, the above technology does not provide a special processing mechanism for target loss. Once the target is lost, it is difficult to resume tracking.

[0005] In view of this, the present invention proposes a target tracking method in a machine vision integrated system to solve the above problem. Summary of the invention

[0006] In order to overcome the above-mentioned defects of the prior art and achieve the above-mentioned purpose, the present invention provides the following technical solution: a target tracking method in a machine vision integrated system, the method comprising:

[0007] S1: Get the video stream, extract the initial frame from the video stream, perform grayscale analysis on the initial frame, calculate the time interval, and perform frame processing on the video stream according to the time interval;

[0008] S2: Target detection and feature extraction are performed in the initial frame to obtain target information, which includes target position and target features;

[0009] S3: Track the target according to the target information of the previous frame, obtain the target position of the next frame, and calculate the target features of the next frame;

[0010] S4: Compare the target features of the previous frame with the target features of the next frame, calculate the tracking confidence, and determine whether the target is lost based on the tracking confidence; if the target is not lost, the target information of the next frame is not updated;

[0011] S5: If the target is lost, the motion trajectory prediction method and trajectory interpolation algorithm are used to predict the target position of the next frame respectively, the prediction results are integrated into the trajectory, the target position of the next frame is updated, and the target features of the next frame are calculated based on the updated target position.

[0012] Further, the video stream is a series of image frames arranged in time sequence;

[0013] The method for calculating the time interval comprises:

[0014] The initial frame of the video stream is obtained, and the initial frame is the zeroth frame of the video stream; the initial frame is grayed to obtain a gray image; the gray level is preset, and a modulo operation is performed on the gray value corresponding to each pixel in the gray image according to the gray level to obtain the gray level corresponding to each pixel; the modulo operation process is: the gray value corresponding to each pixel in the gray image is divided by the gray level, and the corresponding integer part is used as the gray level of the corresponding pixel;

[0015] The gray levels corresponding to the horizontally adjacent pixels in the grayscale image are regarded as a set of grayscale sets, and the number of grayscale sets corresponding to each group is counted and marked as the frequency of occurrence; the size of the construction is The matrix of is the grayscale level, and different elements in the matrix correspond to a set of different grayscale sets; replace the value of each element in the matrix with the occurrence frequency of the corresponding grayscale set to obtain the grayscale co-occurrence matrix corresponding to the grayscale image; mark each element in the grayscale co-occurrence matrix as , Indicates grayscale With grayscale The corresponding frequency of occurrence when horizontally adjacent in the grayscale image, , ;

[0016] According to the grayscale co-occurrence matrix of the grayscale image, the texture feature data corresponding to the grayscale image is calculated; according to the texture feature data corresponding to the grayscale image, the time interval is calculated; according to the time interval, the video stream is frame-processed.

[0017] Furthermore, the expression of contrast is: ; In the formula, is the contrast;

[0018] The expression of entropy is: ; In the formula, is entropy;

[0019] The expression for uniformity is: ; In the formula, for uniformity;

[0020] The expression of energy is: ; In the formula, for energy;

[0021] The expression for the time interval is: ; In the formula, is the time interval, , , , All are preset weight coefficients.

[0022] Furthermore, the method for obtaining the target position includes:

[0023] Use the trained target detection model to mark the target in the initial frame with a rectangular box; randomly obtain the coordinates of the three vertices corresponding to the rectangular box; add the ordinates corresponding to the two vertices with the same horizontal coordinate, and then divide by 2 to obtain the target vertical coordinate; add the horizontal coordinates corresponding to the two vertices with the same vertical coordinate, and then divide by 2 to obtain the target horizontal coordinate; use the target horizontal coordinate and the target vertical coordinate as the target position;

[0024] The target features include shape feature data, color feature data and texture statistical data;

[0025] The shape feature data includes target perimeter and target convexity;

[0026] The method for obtaining the perimeter of the target is as follows: using the edge detection algorithm to detect the edge pixel points corresponding to the target in the initial frame; obtaining the coordinates corresponding to each edge pixel point; according to the coordinates of each edge pixel point, respectively calculating the Euclidean distance between each two adjacent edge pixel points and marking them as adjacent distances; adding each adjacent distance in turn to obtain the perimeter of the target;

[0027] The method for obtaining the target convexity is as follows: using the convex hull algorithm to calculate the convex hull vertices corresponding to the initial frame; obtaining the coordinates corresponding to each convex hull vertex through the built-in coordinate system of the initial frame; calculating the Euclidean distance between each two adjacent convex hull vertices according to the coordinates corresponding to each convex hull vertex, and marking them as convex hull distances; adding each convex hull distance in turn to obtain the convex hull perimeter; dividing the target perimeter by the convex hull perimeter to obtain the target convexity;

[0028] The method for obtaining texture statistical data is: constructing a gray-level co-occurrence matrix corresponding to the target image, and calculating the texture feature data corresponding to the target image; and using the texture feature data corresponding to the target image as texture statistical data.

[0029] Further, the color feature data includes a color histogram and an average color;

[0030] The methods for obtaining the color histogram include:

[0031] According to the detected edge pixels, the target image is segmented from the initial frame; the target image is converted from the RGB color space to Color space, Color spaces include aisle, Channel and Channel; Get the corresponding channel of each pixel in the target image value, Values ​​include value, Value and value; preset bucket size, divide 100 by the bucket size to get The bucket size corresponding to the channel is marked as Bucket size; divide 255 by the bucket size to get Channel and The bucket size corresponding to the channel is marked as Bucket size; count the number of pixels in the target image and mark them as target points; calculate Channel Histogram , Channel Histogram and Channel Histogram , and as a color histogram;

[0032] Channel Histogram The expression is: ; In the formula, for Channel corresponds to The frequency of the bucket, is the target number, is the Dirac function, when When ,when When , is the ceiling function, The target image Pixels value, for Bucket size, ;

[0033] Channel Histogram The expression is: ; In the formula, for Channel corresponds to The frequency of the bucket, The target image Pixels value, for Bucket size;

[0034] Channel Histogram The expression is: ; In the formula, for Channel corresponds to The frequency of the bucket, The target image Pixels value;

[0035] The method for obtaining the average color is: The values ​​are added in sequence and then divided by the target number of points to obtain the average value; the corresponding value of each pixel in the target image The values ​​are added in sequence and then divided by the target number of points to obtain the average value; the corresponding value of each pixel in the target image The values ​​are added in sequence and then divided by the target number of points to obtain the average value; average Value, average Value and average value as the average color.

[0036] Furthermore, the step of obtaining the target position of the next frame includes:

[0037] Step S301: Calculate the length and width of the rectangular frame, and use the target position of the previous frame as the center position to construct the current window;

[0038] Step S302: Mark all pixels in the current window that are not at the center as moving points; construct windows corresponding to each moving point based on the length, width and moving point, and mark them as moving windows; calculate the color histogram corresponding to each moving window, and mark them as moving data; mark the color histogram in the corresponding target information of the previous frame as target data;

[0039] Step S303: moving the current window according to the movement data and the target data, that is, updating the corresponding center position, and calculating the corresponding position change;

[0040] Step S304: looping steps S302 to S303 until the position change is less than a preset change threshold, the loop ends, and the center position of the current window is used as the target position of the next frame.

[0041] Further, in step S301, the method for calculating the length and width of the rectangular frame is as follows: randomly obtain two vertices located on the same diagonal line in the rectangular frame and mark them as diagonal points, obtain coordinates corresponding to the diagonal points and mark them as diagonal coordinates; subtract the smaller horizontal coordinate from the larger horizontal coordinate of the two diagonal coordinates to obtain the length; subtract the smaller vertical coordinate from the larger vertical coordinate of the two diagonal coordinates to obtain the width;

[0042] In step S303, the method for updating the center position corresponding to the current window includes:

[0043] ;

[0044] In the formula, The center position of the current window after updating. The center position of the current window before updating. For the The similarity between the mobile data and the target data, For the The moving point corresponding to the moving data is is the amount of mobile data, ;

[0045] The expression of position change is: ; In the formula, is the position change;

[0046] According to the current window corresponding to the target position of the next frame, the target features corresponding to the next frame are calculated.

[0047] Furthermore, the method for determining whether the target is lost includes:

[0048] The target features of the previous frame and the target features of the next frame are used as analysis data, and the analysis data are input into the trained tracking analysis model to predict the corresponding tracking confidence; the confidence threshold is preset, and the confidence threshold is weighted and smoothed to obtain the smoothing threshold; the tracking confidence is compared with the smoothing threshold; if the tracking confidence is less than the smoothing threshold, it is judged that the target is lost; if the tracking confidence is greater than or equal to the smoothing threshold, it is judged that the target is not lost;

[0049] The expression of the smoothing threshold is: ; In the formula, For the The corresponding smoothing threshold when judging whether the target is lost is: is the scale factor, For the The corresponding tracking confidence when judging whether the target is lost. is the smoothing factor, For the The corresponding smooth confidence when judging whether the target is lost, is the confidence threshold;

[0050] The expression for the scaling factor is: ; In the formula, is the minimum function, is the maximum value function, is the basic scale factor, is the sensitivity factor, For the The corresponding smooth confidence when judging whether the target is lost, ;

[0051] The expression of the smoothing factor is: ; In the formula, is the minimum value of the smoothing factor, is the maximum value of the smoothing factor, is an exponential function, is the amplitude factor, For the The corresponding tracking confidence when judging whether the target is lost.

[0052] Furthermore, the method for updating the target position of the next frame includes:

[0053] Get the historical position, which is the target position obtained at the historical moment; use the historical position and the target position corresponding to the previous frame as prediction data, input the prediction data into the trained position prediction model, predict the target position corresponding to the next frame, and mark it as the first prediction position, which includes the first horizontal coordinate and the first vertical coordinate; the position prediction model is the RNN neural network model:

[0054] Obtaining the frame corresponding to each target position in the predicted data and the timestamp corresponding to the next frame, taking each target position in the predicted data and the corresponding acquisition time as a set of position sets, wherein the position sets correspond to the target positions one by one; calculating the target position corresponding to the next frame according to each set of position sets and the timestamp corresponding to the next frame, and marking it as a second predicted position; the second predicted position includes a second abscissa and a second ordinate;

[0055] Add the first horizontal coordinate to the second horizontal coordinate and divide the result by 2 to obtain an updated horizontal coordinate; add the first vertical coordinate to the second vertical coordinate and divide the result by 2 to obtain an updated vertical coordinate; update the target position of the next frame according to the updated horizontal coordinate and the updated vertical coordinate;

[0056] The method for calculating the target features of the next frame based on the updated target position is: construct an update window according to the length and width of the rectangular frame and take the target position of the next frame after the update as the center position; and calculate the target features corresponding to the next frame according to the update window.

[0057] Furthermore, the training process of the location prediction model includes:

[0058] Pre-continuous acquisition target locations and construct a location training set. , The number of target positions in the predicted data; based on the position training set, a position prediction model is trained to predict the target position corresponding to the next frame;

[0059] Preset sliding step size J and sliding window length E, ; The target positions in the position training set are converted into multiple training samples using the sliding window method, and each training sample includes E target positions; each training sample is used as the input of the position prediction model, and the target position after the predicted sliding step length J is used as the output. The actual J target positions corresponding to each training sample are used as the prediction targets. The mean absolute percentage error MAPE is used to evaluate the model accuracy of the prediction results, and a position prediction model is generated based on the target position to predict the target position corresponding to the next frame;

[0060] The expression of the second horizontal axis is: ; In the formula, is the second horizontal axis, is the timestamp corresponding to the next frame, For the prediction data The timestamp corresponding to the frame, For the prediction data The timestamp corresponding to the frame, For the prediction data The target horizontal coordinate corresponding to the frame, , , is the number of frames in the prediction data;

[0061] The expression of the second ordinate is: ; In the formula, is the second ordinate, For the prediction data The target vertical coordinate corresponding to the frame.

[0062] Technical effects and advantages of a target tracking method in a machine vision integrated system of the present invention:

[0063] Through target detection, feature extraction and dynamic tracking based on target features, accurate positioning and continuous tracking of targets in complex scenarios are achieved; the use of technologies such as video frame processing, color histogram matching and motion trajectory prediction can effectively overcome interference factors such as occlusion and lighting changes, and improve the robustness and real-time performance of target tracking; at the same time, deep learning technology is used to analyze and evaluate the tracking results, and the tracking threshold is dynamically adjusted to adaptively respond to complex target motion states, further improving the tracking accuracy; it not only improves the intelligent perception capability of the machine vision system, but also provides an effective target tracking solution for application scenarios such as intelligent monitoring and unmanned driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a flow chart of a target tracking method in a machine vision integrated system according to Embodiment 1 of the present invention;

[0065] Figure 2 This is a flow chart of a method for obtaining a target position in a next frame according to Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] Example 1

[0068] See also Figure 1 As shown, the target tracking method in a machine vision integrated system described in this embodiment includes:

[0069] S1: Get the video stream, extract the initial frame from the video stream, perform grayscale analysis on the initial frame, calculate the time interval, and perform frame processing on the video stream according to the time interval.

[0070] The video stream is a series of image frames arranged in time sequence. The video stream is acquired through a machine vision integration system, and the machine vision integration system acquires the video stream through a camera.

[0071] Methods for calculating time intervals include:

[0072] Obtain the initial frame of the video stream, which is the zeroth frame (the first image) of the video stream, that is, the frame captured when the video starts to be played or processed; perform grayscale processing (such as weighted average method, maximum value method, Luma method, etc.) on the initial frame to obtain a grayscale image; preset the grayscale level, which is preset by a person skilled in the art according to actual conditions; perform a modulo operation on the grayscale value corresponding to each pixel in the grayscale image according to the grayscale level, and obtain the grayscale level corresponding to each pixel; the process of the modulo operation is: divide the grayscale value corresponding to each pixel in the grayscale image by the grayscale level, and use the corresponding integer part as the grayscale level of the corresponding pixel;

[0073] The gray levels corresponding to the horizontally adjacent pixels in the grayscale image are regarded as a set of grayscale sets, and the number of grayscale sets corresponding to each group is counted and marked as the frequency of occurrence; the size of the construction is The matrix of is the grayscale level, and different elements in the matrix correspond to a set of different grayscale sets; replace the value of each element in the matrix with the occurrence frequency of the corresponding grayscale set to obtain the grayscale co-occurrence matrix corresponding to the grayscale image; mark each element in the grayscale co-occurrence matrix as , Indicates grayscale With grayscale The corresponding frequency of occurrence when horizontally adjacent in the grayscale image, , .

[0074] Exemplarily, the grayscale level corresponding to the grayscale image is , where the grayscale set corresponding to the first row is and , the grayscale set corresponding to the second row is and , the grayscale set corresponding to the third row is and ; Grayscale set The corresponding frequency of occurrence is 2, that is, the first row and second column in the gray-level co-occurrence matrix is ​​2; the gray-level set The corresponding frequency of occurrence is 2, that is, the second row and third column in the gray-level co-occurrence matrix is ​​2; the gray-level set The corresponding occurrence frequency is 1, that is, the first column of the second row in the gray level co-occurrence matrix is ​​1; the gray level set The corresponding frequency of occurrence is 1, that is, the third row and second column in the gray-level co-occurrence matrix is ​​1; since the other gray-level sets do not exist, the corresponding frequency of occurrence is 0, that is, the value of the corresponding element in the gray-level co-occurrence matrix is ​​0; therefore, the gray-level co-occurrence matrix corresponding to the gray-level image is .

[0075] According to the gray-level co-occurrence matrix of the gray-level image, the texture feature data corresponding to the gray-level image is calculated; according to the texture feature data corresponding to the gray-level image, the time interval is calculated.

[0076] Texture feature data include contrast, entropy, uniformity, and energy;

[0077] The expression for contrast is: ; In the formula, is the contrast;

[0078] The expression of entropy is: ; In the formula, is entropy;

[0079] The expression for uniformity is: ; In the formula, for uniformity;

[0080] The expression of energy is: ; In the formula, For energy.

[0081] The expression for the time interval is: ; In the formula, is the time interval, , , , All are preset weight coefficients;

[0082] The specific value of the weight coefficient in the formula can be set according to actual conditions. The weight coefficient reflects the degree of influence of texture feature data on the time interval. Technical personnel in this field can preset the corresponding weight coefficient according to the actual degree of influence of texture feature data on the time interval, so as to accurately adjust the time interval in the frame processing, thereby optimizing the efficiency and accuracy of the frame processing.

[0083] It should be noted that the greater the contrast, the more obvious the change of grayscale in the grayscale image, the more complex the texture of the initial frame, and vice versa; the greater the entropy, the richer the information contained in the grayscale image, the more complex the texture of the initial frame, and vice versa; the greater the uniformity, the smaller the change between grayscale values ​​in the grayscale image, the simpler the texture of the initial frame, and vice versa; the lower the energy, the more irregular the texture in the grayscale image, the more complex the texture of the initial frame, and vice versa; complex textures usually contain more details and changes, so in order to capture these details and changes more frequently and effectively, it is necessary to reduce the time interval to avoid missing important details and changes, which will help improve the accuracy of subsequent analysis.

[0084] S2: Perform target detection and feature extraction in the initial frame to obtain target information, which includes target position and target features.

[0085] Methods for obtaining target locations include:

[0086] The trained target detection model is used to mark the target in the initial frame with a rectangular box. The three vertex coordinates corresponding to the rectangular box are randomly obtained through the built-in coordinate system of the initial frame. The vertical coordinates corresponding to the two vertex coordinates with the same horizontal coordinate are added, and then divided by 2 to obtain the target vertical coordinate. The horizontal coordinates corresponding to the two vertex coordinates with the same vertical coordinate are added, and then divided by 2 to obtain the target horizontal coordinate. The target horizontal coordinate and the target vertical coordinate are used as the target position.

[0087] The training methods for the target detection model include:

[0088] Pre-fetch The initial frame includes the target, is an integer greater than 1; The initial frame including the target is marked as a training image; the target in the training image is marked with a rectangular frame; the marked training image is divided into a training set and a test set, 70% of the training images are used as the training set, and 30% of the training images are used as the test set, the training set is used to train the target detection model, and the test set is used to test the target detection model; a coordinate error threshold is preset, and when the mean value of the predicted coordinate errors of all training images in the test set is less than the coordinate error threshold, the target detection model training is completed, and the target detection model is output; wherein, the calculation formula for the predicted coordinate error includes:

[0089] ;

[0090] In the formula, is the predicted coordinate error, is the number of the training image, is the number of training images in the test set, For the In the image The predicted horizontal coordinates of the vertices, For the In the image The actual horizontal coordinates of the vertices, For the In the image The predicted ordinate of each vertex, For the In the image The actual vertical coordinates of the vertices. The rectangular box has 4 vertices in total. , ; The coordinate error threshold is preset according to the accuracy required by the target detection model;

[0091] The above target detection model is specifically a convolutional neural network model.

[0092] Methods for obtaining target features include:

[0093] Target features include shape feature data, color feature data and texture statistics.

[0094] The shape feature data include the target perimeter and the target convexity;

[0095] The method for obtaining the target perimeter is as follows: using an edge detection algorithm (such as Canny edge detection, Sobel operator, Prewitt operator, etc.) to detect the edge pixel points corresponding to the target in the initial frame; obtaining the coordinates corresponding to each edge pixel point through the built-in coordinate system of the initial frame; calculating the Euclidean distance between each two adjacent edge pixel points according to the coordinates of each edge pixel point, and marking them as adjacent distances; adding each adjacent distance in turn to obtain the target perimeter.

[0096] The method for obtaining the target convexity is as follows: using a convex hull algorithm (such as Graham scanning method, Jarvis walking method, Andrew algorithm, etc.) to calculate the convex hull vertices corresponding to the initial frame; obtaining the coordinates corresponding to each convex hull vertex through the built-in coordinate system of the initial frame; according to the coordinates corresponding to each convex hull vertex, respectively calculating the Euclidean distance between each two adjacent convex hull vertices, and marking them as convex hull distances; adding each convex hull distance in turn to obtain the convex hull perimeter; dividing the target perimeter by the convex hull perimeter to obtain the target convexity.

[0097] Color feature data includes color histogram and average color;

[0098] The methods for obtaining the color histogram include:

[0099] According to the detected edge pixels, the target image is segmented from the initial frame; the target image is converted from the RGB color space to Color space, Color spaces include aisle, Channel and Channel, color space conversion method is an existing technology, which will not be described in detail here; get the corresponding color space of each pixel in the target image value, Values ​​include (brightness) value, (Red-Green Component) Value and (yellow-blue component) value; preset bucket volume, which is preset by technicians in this field according to actual conditions; divide 100 by the bucket volume to obtain The bucket size corresponding to the channel is marked as Bucket size, where each bucket represents a brightness range; divide 255 by the bucket size to get Channel and The bucket size corresponding to the channel is marked as Bucket size, where each bucket represents a red-green component interval or a yellow-blue component interval; count the number of pixels in the target image and mark them as target points; calculate Channel Histogram , Channel Histogram and Channel Histogram , and as a color histogram;

[0100] Channel Histogram The expression is: ; In the formula, for Channel corresponds to The frequency of the bucket, is the target number, is the Dirac function, when When ,when When , is the ceiling function, The target image Pixels value, for Bucket size, ;

[0101] Channel Histogram The expression is: ; In the formula, for Channel corresponds to The frequency of the bucket, The target image Pixels value, for Bucket size;

[0102] Channel Histogram The expression is: ; In the formula, for Channel corresponds to The frequency of the bucket, The target image Pixels value.

[0103] The method for obtaining the average color is: The values ​​are added in sequence and then divided by the target number of points to obtain the average value; the corresponding value of each pixel in the target image The values ​​are added in sequence and then divided by the target number of points to obtain the average value; the corresponding value of each pixel in the target image The values ​​are added in sequence and then divided by the target number of points to obtain the average value; average Value, average Value and average value as the average color.

[0104] The method for obtaining texture statistical data is: constructing a gray-level co-occurrence matrix corresponding to the target image, and calculating the texture feature data corresponding to the target image; and using the texture feature data corresponding to the target image as texture statistical data.

[0105] S3: Track the target based on the target information of the previous frame, obtain the target position of the next frame, and calculate the target features of the next frame.

[0106] The steps to obtain the target position of the next frame include:

[0107] Step S301: Calculate the length and width of the rectangular frame, and use the target position of the previous frame as the center position to construct the current window;

[0108] Step S302: Mark all pixels in the current window that are not at the center as moving points; construct windows corresponding to each moving point based on the length, width and moving point, and mark them as moving windows; calculate the color histogram corresponding to each moving window, and mark them as moving data; mark the color histogram in the corresponding target information of the previous frame as target data;

[0109] Step S303: moving the current window according to the movement data and the target data, that is, updating the corresponding center position, and calculating the corresponding position change;

[0110] Step S304: looping steps S302 to S303 until the position change is less than a preset change threshold, the loop ends, and the center position of the current window is used as the target position of the next frame; the change threshold is preset by a person skilled in the art according to actual conditions.

[0111] In the above step S301, the method for calculating the length and width of the rectangular box is: randomly obtain two vertices on the same diagonal line in the rectangular box, mark them as diagonal points, obtain the coordinates corresponding to the diagonal points, and mark them as diagonal coordinates; subtract the smaller horizontal coordinate from the larger horizontal coordinate of the two diagonal coordinates to obtain the length; subtract the smaller vertical coordinate from the larger vertical coordinate of the two diagonal coordinates to obtain the width.

[0112] In the above step S303, the method for updating the center position corresponding to the current window includes:

[0113] ;

[0114] In the formula, The center position of the current window after updating. The center position of the current window before updating. For the The similarity between the mobile data and the target data is calculated using similarity calculation methods (such as Bhattacharyya distance, chi-square distance, etc.). For the The moving point corresponding to the moving data is is the amount of mobile data, .

[0115] The expression of position change is: ; In the formula, is the position change.

[0116] According to the current window corresponding to the target position of the next frame, the target features corresponding to the next frame are calculated.

[0117] S4: Compare the target features of the previous frame with the target features of the next frame, calculate the tracking confidence, and determine whether the target is lost based on the tracking confidence; if the target is not lost, the target information of the next frame is not updated.

[0118] Methods for determining whether the target is lost include:

[0119] The target features of the previous frame and the target features of the next frame are used as analysis data, and the analysis data are input into the trained tracking analysis model to predict the corresponding tracking confidence; a confidence threshold is preset, and the confidence threshold is preset by technical personnel in this field according to actual conditions; the confidence threshold is weighted and smoothed to obtain a smoothing threshold; the tracking confidence is compared with the smoothing threshold; if the tracking confidence is less than the smoothing threshold, it is judged that the target is lost, indicating that the reliability of target tracking is low, and it is necessary to re-acquire the target information of the next frame; if the tracking confidence is greater than or equal to the smoothing threshold, it is judged that the target is not lost, indicating that the reliability of target tracking is high, and it is not necessary to re-acquire the target information of the next frame.

[0120] The training process of the tracking analysis model includes:

[0121] Pre-collection Group analysis data, The corresponding tracking confidence level is set for each group of analysis data. is an integer greater than 1, and the analysis data and the corresponding tracking confidence are converted into a corresponding set of feature vectors; the tracking confidence corresponding to the analysis data is collected by technicians in the historical target tracking process. The target features of the next frame are obtained under the condition of the corresponding target features of the previous frame in each group of analysis data, and compared with the target features of the next frame in the corresponding analysis data, and the corresponding tracking confidence is analyzed and used as the tracking confidence of the corresponding analysis data. The corresponding tracking confidence levels are set for the group analysis data in turn;

[0122] Each set of feature vectors is used as the input of the tracking analysis model. The tracking analysis model uses a set of predicted tracking confidences corresponding to each set of analysis data as output, and the actual tracking confidences corresponding to each set of analysis data as the prediction target. The actual tracking confidences are the preset tracking confidences corresponding to the analysis data. The training target is to minimize the sum of the prediction errors of all analysis data. The calculation formula for the prediction error is: ; In the formula, is the prediction error, is the group number of the eigenvector corresponding to the analyzed data, For the The prediction tracking confidence corresponding to the group analysis data, For the The actual tracking confidence corresponding to the group analysis data; the tracking analysis model is trained until the sum of the prediction errors reaches convergence and the training is stopped.

[0123] The above-mentioned tracking and analysis model is specifically a deep neural network model; it includes an input layer, a hidden layer and an output layer; each hidden layer includes multiple neurons, each neuron is connected to the neurons in the next layer, and the connection contains weights, which determine the importance and influence of data transmission in the neural network; an activation function is applied to each neuron between the hidden layer and the output layer, and the activation function introduces nonlinearity, allowing the network to learn more complex patterns and features.

[0124] The expression of the smoothing threshold is: ; In the formula, For the The corresponding smoothing threshold when judging whether the target is lost is: is the scale factor, For the The corresponding tracking confidence when judging whether the target is lost. is the smoothing factor, For the The corresponding smooth confidence when judging whether the target is lost, is the confidence threshold.

[0125] The expression for the scaling factor is: ; In the formula, is the minimum function, is the maximum value function, is the basic scale factor, is the sensitivity factor, For the The corresponding smooth confidence when judging whether the target is lost, The basic proportional factor and sensitivity factor are preset by technicians in this field according to actual conditions.

[0126] The expression of the smoothing factor is: ; In the formula, is the minimum value of the smoothing factor, is the maximum value of the smoothing factor, is an exponential function, is the amplitude factor, For the The corresponding tracking confidence when judging whether the target is lost; the minimum value of the smoothing factor, the maximum value of the smoothing factor and the amplitude factor are preset by technical personnel in this field according to actual conditions.

[0127] S5: If the target is lost, the motion trajectory prediction method and trajectory interpolation algorithm are used to predict the target position of the next frame respectively, the prediction results are integrated into the trajectory, the target position of the next frame is updated, and the target features of the next frame are calculated based on the updated target position.

[0128] Methods for updating the target position for the next frame include:

[0129] Obtain the historical position, which is the target position obtained at the historical moment; use the historical position and the target position corresponding to the previous frame as prediction data, input the prediction data into the trained position prediction model, predict the target position corresponding to the next frame, and mark it as the first prediction position, which includes the first horizontal coordinate and the first vertical coordinate; the position prediction model is an RNN neural network model, and the training process of the position prediction model includes:

[0130] Pre-continuous acquisition target locations and construct a location training set. , The number of target positions in the predicted data; based on the position training set, a position prediction model is trained to predict the target position corresponding to the next frame;

[0131] According to the actual experience of those skilled in the art, the sliding step size J and the sliding window length E are preset. ; The target position in the position training set is converted into multiple training samples using the sliding window method, and each training sample includes E target positions; each training sample is used as the input of the position prediction model, and the target position after the predicted sliding step length J is used as the output, and the actual J target positions corresponding to each training sample are used as the prediction target. The prediction results are evaluated by the mean absolute percentage error MAPE for model accuracy. When the calculated MAPE is less than the preset MAPE, the position prediction model training corresponding to the target position is completed; the calculation formula of the mean absolute percentage error MAPE is: ; In the formula, For the The predicted target corresponding to the predicted target position is For the The predicted target location, is the number of predicted target positions; a position prediction model is generated to predict the target position corresponding to the next frame based on the target position.

[0132] Exemplarily, the position training set A includes 10 target positions. , For the target locations, , define the length of the sliding window as 3, use the sliding window to construct 7 training samples, each training sample contains 3 consecutive target positions, and the next target position of the 3 consecutive target positions is used as the prediction target; for example: training sample , training samples The corresponding prediction target is ; Training samples , training samples The corresponding prediction target is ; And so on, it is used to train the location prediction model.

[0133] Obtain the frame corresponding to each target position in the predicted data and the timestamp corresponding to the next frame, and take each target position in the predicted data and the corresponding acquisition time as a group of position sets, and the position sets correspond to the target positions one by one; calculate the target position corresponding to the next frame according to each group of position sets and the timestamp corresponding to the next frame, and mark it as the second predicted position; the second predicted position includes a second horizontal coordinate and a second vertical coordinate.

[0134] The expression of the second horizontal axis is: ; In the formula, is the second horizontal axis, is the timestamp corresponding to the next frame, For the prediction data The timestamp corresponding to the frame, For the prediction data The timestamp corresponding to the frame, For the prediction data The target horizontal coordinate corresponding to the frame, , , is the number of frames in the prediction data.

[0135] The expression of the second ordinate is: ; In the formula, is the second ordinate, For the prediction data The target vertical coordinate corresponding to the frame.

[0136] Add the first horizontal coordinate to the second horizontal coordinate and divide the result by 2 to obtain an updated horizontal coordinate; add the first vertical coordinate to the second vertical coordinate and divide the result by 2 to obtain an updated vertical coordinate; update the target position of the next frame according to the updated horizontal coordinate and the updated vertical coordinate;

[0137] The method for calculating the target features of the next frame based on the updated target position is: construct an update window according to the length and width of the rectangular frame and take the target position of the next frame after the update as the center position; and calculate the target features corresponding to the next frame according to the update window.

[0138] This embodiment achieves accurate positioning and continuous tracking of targets in complex scenarios through target detection, feature extraction, and dynamic tracking based on target features. It utilizes technologies such as video frame processing, color histogram matching, and motion trajectory prediction to effectively overcome interference factors such as occlusion and illumination changes, thereby improving the robustness and real-time performance of target tracking. At the same time, it uses deep learning technology to analyze and evaluate the tracking results, and dynamically adjusts the tracking threshold to adaptively respond to complex target motion states, further improving tracking accuracy. It not only improves the intelligent perception capability of the machine vision system, but also provides an effective target tracking solution for application scenarios such as intelligent monitoring and unmanned driving.

[0139] Example 2

[0140] The present application also provides an electronic device. The electronic device may include one or more processors and one or more memories. The memories store computer-readable codes, and when the computer-readable codes are executed by the one or more processors, the target tracking method in the machine vision integrated system as described above may be executed.

[0141] The method or system according to the implementation mode of the present application can also be implemented with the aid of the architecture of the electronic device shown in the present application. The electronic device may include a bus, one or more CPUs, ROM, RAM, a communication port connected to a network, input / output, a hard disk, etc. A storage device in the electronic device, such as a ROM or a hard disk, can store a target tracking method in a machine vision integrated system provided in the present application. Furthermore, the electronic device may also include a user interface. Of course, the architecture shown in the present application is only exemplary. When implementing different devices, one or more components in the electronic device shown in the present application may be omitted according to actual needs.

[0142] Example 3

[0143] As shown, one embodiment of the present application discloses a computer-readable storage medium. Computer-readable instructions are stored on the computer-readable storage medium. When the computer-readable instructions are executed by the processor, a target tracking method in a machine vision integration system according to an embodiment of the present application described with reference to the above figures can be executed. The storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache). Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0144] In addition, according to the implementation of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the present application provides a non-transitory machine-readable storage medium, which stores machine-readable instructions, and the machine-readable instructions can be executed by a processor to execute instructions corresponding to the method steps provided by the present application, for example: a target tracking method in a machine vision integration system. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are executed.

[0145] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0146] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A target tracking method in a machine vision integrated system, characterized in that: Methods include: S1: Get the video stream, extract the initial frame from the video stream, perform grayscale analysis on the initial frame, calculate the time interval, and perform frame processing on the video stream according to the time interval; S2: Target detection and feature extraction are performed in the initial frame to obtain target information, which includes target position and target features; S3: Track the target according to the target information of the previous frame, obtain the target position of the next frame, and calculate the target features of the next frame; S4: Compare the target features of the previous frame with the target features of the next frame, calculate the tracking confidence, and determine whether the target is lost based on the tracking confidence; If the target is not lost, the target information for the next frame will not be updated; S5: If the target is lost, the motion trajectory prediction method and trajectory interpolation algorithm are used to predict the target position of the next frame respectively, the prediction results are integrated into the trajectory, the target position of the next frame is updated, and the target features of the next frame are calculated based on the updated target position.

2. The target tracking method in a machine vision integrated system according to claim 1, characterized in that: The video stream is a series of image frames arranged in time sequence; The method for calculating the time interval comprises: The initial frame of the video stream is obtained, and the initial frame is the zeroth frame of the video stream; the initial frame is grayed to obtain a gray image; the gray level is preset, and a modulo operation is performed on the gray value corresponding to each pixel in the gray image according to the gray level to obtain the gray level corresponding to each pixel; the modulo operation process is: the gray value corresponding to each pixel in the gray image is divided by the gray level, and the corresponding integer part is used as the gray level of the corresponding pixel; The gray levels corresponding to the horizontally adjacent pixels in the grayscale image are regarded as a set of grayscale sets, and the number of grayscale sets corresponding to each group is counted and marked as the frequency of occurrence; the size of the construction is The matrix of is the grayscale level, and different elements in the matrix correspond to a set of different grayscale sets; replace the value of each element in the matrix with the occurrence frequency of the corresponding grayscale set to obtain the grayscale co-occurrence matrix corresponding to the grayscale image; mark each element in the grayscale co-occurrence matrix as , Indicates grayscale With grayscale The corresponding frequency of occurrence when horizontally adjacent in the grayscale image, , ; According to the gray-level co-occurrence matrix of the gray-level image, the texture feature data corresponding to the gray-level image is calculated; according to the texture feature data corresponding to the gray-level image, the time interval is calculated.

3. The target tracking method in a machine vision integrated system according to claim 2, characterized in that: The expression for contrast is: ; In the formula, is the contrast; The expression of entropy is: ; In the formula, is entropy; The expression for uniformity is: ; In the formula, for uniformity; The expression of energy is: ; In the formula, for energy; The expression for the time interval is: ; In the formula, is the time interval, , , , All are preset weight coefficients.

4. The target tracking method in a machine vision integrated system according to claim 3, characterized in that: Methods for obtaining the target location include: Use the trained target detection model to mark the target in the initial frame with a rectangular box; randomly obtain the coordinates of the three vertices corresponding to the rectangular box; add the ordinates corresponding to the two vertices with the same horizontal coordinate, and then divide by 2 to obtain the target vertical coordinate; add the horizontal coordinates corresponding to the two vertices with the same vertical coordinate, and then divide by 2 to obtain the target horizontal coordinate; use the target horizontal coordinate and the target vertical coordinate as the target position; The target features include shape feature data, color feature data and texture statistical data; The shape feature data includes target perimeter and target convexity; The method for obtaining the perimeter of the target is as follows: using the edge detection algorithm to detect the edge pixel points corresponding to the target in the initial frame; obtaining the coordinates corresponding to each edge pixel point; according to the coordinates of each edge pixel point, respectively calculating the Euclidean distance between each two adjacent edge pixel points and marking them as adjacent distances; adding each adjacent distance in turn to obtain the perimeter of the target; The method for obtaining the target convexity is as follows: using the convex hull algorithm to calculate the convex hull vertices corresponding to the initial frame; obtaining the coordinates corresponding to each convex hull vertex through the built-in coordinate system of the initial frame; calculating the Euclidean distance between each two adjacent convex hull vertices according to the coordinates corresponding to each convex hull vertex, and marking them as convex hull distances; adding each convex hull distance in turn to obtain the convex hull perimeter; dividing the target perimeter by the convex hull perimeter to obtain the target convexity; The method for obtaining texture statistical data is: constructing a gray-level co-occurrence matrix corresponding to the target image, and calculating the texture feature data corresponding to the target image; and using the texture feature data corresponding to the target image as texture statistical data.

5. The target tracking method in a machine vision integrated system according to claim 4, characterized in that: The color feature data includes a color histogram and an average color; The methods for obtaining the color histogram include: Segment the target image from the initial frame based on the detected edge pixels; The color space conversion method is used to convert the target image from RGB color space to Color space, Color spaces include aisle, Channel and Channel; Get the corresponding channel of each pixel in the target image value, Values ​​include value, Value and value; preset bucket size, divide 100 by the bucket size to get The bucket size corresponding to the channel is marked as Bucket size; divide 255 by the bucket size to get Channel and The bucket size corresponding to the channel is marked as Bucket size; count the number of pixels in the target image and mark them as target points; calculate Channel Histogram , Channel Histogram and Channel Histogram , and as a color histogram; Channel Histogram The expression is: ; In the formula, for Channel corresponds to The frequency of the bucket, is the target number, is the Dirac function, when When ,when When , is the ceiling function, The target image Pixels value, for Bucket size, ; Channel Histogram The expression is: ; In the formula, for Channel corresponds to The frequency of the bucket, The target image Pixels value, for Bucket size; Channel Histogram The expression is: ; In the formula, for Channel corresponds to The frequency of the bucket, The target image Pixels value; The method for obtaining the average color is: The values ​​are added in sequence and then divided by the target number of points to obtain the average value; the corresponding value of each pixel in the target image The values ​​are added in sequence and then divided by the target number of points to obtain the average value; the corresponding value of each pixel in the target image The values ​​are added in sequence and then divided by the target number of points to obtain the average value; average Value, average Value and average value as the average color.

6. The target tracking method in a machine vision integrated system according to claim 5, characterized in that: The step of obtaining the target position of the next frame comprises: Step S301: Calculate the length and width of the rectangular frame, and use the target position of the previous frame as the center position to construct the current window; Step S302: Mark all pixels in the current window that are not at the center as moving points; construct windows corresponding to each moving point based on the length, width and moving point, and mark them as moving windows; calculate the color histogram corresponding to each moving window, and mark them as moving data; mark the color histogram in the corresponding target information of the previous frame as target data; Step S303: moving the current window according to the movement data and the target data, that is, updating the corresponding center position, and calculating the corresponding position change; Step S304: looping steps S302 to S303 until the position change is less than a preset change threshold, the loop ends, and the center position of the current window is used as the target position of the next frame.

7. The target tracking method in a machine vision integrated system according to claim 6, characterized in that: In step S301, the method for calculating the length and width of the rectangular frame is as follows: randomly obtain two vertices located on the same diagonal line in the rectangular frame and mark them as diagonal points, obtain coordinates corresponding to the diagonal points and mark them as diagonal coordinates; subtract the smaller horizontal coordinate from the larger horizontal coordinate of the two diagonal coordinates to obtain the length; subtract the smaller vertical coordinate from the larger vertical coordinate of the two diagonal coordinates to obtain the width; In step S303, the method for updating the center position corresponding to the current window includes: ； In the formula, The center position of the current window after updating. The center position of the current window before updating. For the The similarity between the mobile data and the target data, For the The moving point corresponding to the moving data is is the amount of mobile data, ; The expression of position change is: ; In the formula, is the position change; According to the current window corresponding to the target position of the next frame, the target features corresponding to the next frame are calculated.

8. The target tracking method in a machine vision integrated system according to claim 7, characterized in that: Methods for determining whether the target is lost include: The target features of the previous frame and the target features of the next frame are used as analysis data, and the analysis data are input into the trained tracking analysis model to predict the corresponding tracking confidence; the confidence threshold is preset, and the confidence threshold is weighted and smoothed to obtain the smoothing threshold; the tracking confidence is compared with the smoothing threshold; if the tracking confidence is less than the smoothing threshold, it is judged that the target is lost; if the tracking confidence is greater than or equal to the smoothing threshold, it is judged that the target is not lost; The expression of the smoothing threshold is: ; In the formula, For the The corresponding smoothing threshold when judging whether the target is lost is: is the scale factor, For the The corresponding tracking confidence when judging whether the target is lost. is the smoothing factor, For the The corresponding smooth confidence when judging whether the target is lost, is the confidence threshold; The expression for the scaling factor is: ; In the formula, is the minimum function, is the maximum value function, is the basic scale factor, is the sensitivity factor, For the The corresponding smooth confidence when judging whether the target is lost, ; The expression of the smoothing factor is: ; In the formula, is the minimum value of the smoothing factor, is the maximum value of the smoothing factor, is an exponential function, is the amplitude factor, For the The corresponding tracking confidence when judging whether the target is lost.

9. The target tracking method in a machine vision integrated system according to claim 8, characterized in that: The method for updating the target position of the next frame comprises: Get the historical position, which is the target position obtained at the historical moment; use the historical position and the target position corresponding to the previous frame as prediction data, input the prediction data into the trained position prediction model, predict the target position corresponding to the next frame, and mark it as the first prediction position, which includes the first horizontal coordinate and the first vertical coordinate; the position prediction model is the RNN neural network model: Obtaining the frame corresponding to each target position in the predicted data and the timestamp corresponding to the next frame, taking each target position in the predicted data and the corresponding acquisition time as a set of position sets, wherein the position sets correspond to the target positions one by one; calculating the target position corresponding to the next frame according to each set of position sets and the timestamp corresponding to the next frame, and marking it as a second predicted position; the second predicted position includes a second abscissa and a second ordinate; Add the first horizontal coordinate to the second horizontal coordinate and divide the result by 2 to obtain an updated horizontal coordinate; add the first vertical coordinate to the second vertical coordinate and divide the result by 2 to obtain an updated vertical coordinate; update the target position of the next frame according to the updated horizontal coordinate and the updated vertical coordinate; The method for calculating the target features of the next frame based on the updated target position is: construct an update window according to the length and width of the rectangular frame and take the target position of the next frame after the update as the center position; and calculate the target features corresponding to the next frame according to the update window.

10. The target tracking method in a machine vision integrated system according to claim 9, characterized in that: The training process of the location prediction model includes: Pre-continuous acquisition target locations and construct a location training set. , The number of target positions in the predicted data; based on the position training set, a position prediction model is trained to predict the target position corresponding to the next frame; Preset sliding step size J and sliding window length E, ; The target positions in the position training set are converted into multiple training samples using the sliding window method, and each training sample includes E target positions; each training sample is used as the input of the position prediction model, and the target position after the predicted sliding step length J is used as the output. The actual J target positions corresponding to each training sample are used as the prediction targets. The mean absolute percentage error MAPE is used to evaluate the model accuracy of the prediction results, and a position prediction model is generated based on the target position to predict the target position corresponding to the next frame; The expression of the second horizontal axis is: ; In the formula, is the second horizontal axis, is the timestamp corresponding to the next frame, For the prediction data The timestamp corresponding to the frame, For the prediction data The timestamp corresponding to the frame, For the prediction data The target horizontal coordinate corresponding to the frame, , , is the number of frames in the prediction data; The expression of the second ordinate is: ; In the formula, is the second ordinate, For the prediction data The target vertical coordinate corresponding to the frame.

Images