Target tracking method and device, intelligent selling cabinet and storage medium

Abstract

The invention relates to the technical field of target tracking, and discloses a target tracking method and device, an intelligent selling cabinet and a storage medium. Performing pixel weighting on the original image data to obtain target image data; performing rotating frame labeling on the target image data to obtain labeling information of the target image data; according to the annotation information, performing feature learning on the target image data to obtain smear feature information; obtaining a target detection model according to the smear feature information; and obtaining a target tracking result according to the target detection model. According to the method, the target image data with the smear features are obtained after pixel weighting is carried out on the original data, then rotating frame labeling and feature learning are carried out on the target image data, so that a target detection model learns recognition of the smear features, and understanding of a target motion mode is enhanced. In this way, the target can be accurately detected and tracked in real time in a complex scene where the target rapidly moves.

Description

Technical Field

[0001] The present application relates to the field of target tracking technology, and in particular to a target tracking method, device, smart vending cabinet and storage medium. Background Art

[0002] In the design and implementation of smart vending machines, how to efficiently identify and track the movement trajectory of goods during the purchase process is a key technical challenge. Smart vending machines need to determine in real time whether consumers are taking out or putting back goods, which is not only related to the inventory management of goods, but also involves the accuracy of consumption records and the optimization of user experience. Therefore, accurate object trajectory recognition methods are particularly important.

[0003] At present, the inventors have found that the commonly used moving target tracking methods have the following problems: (1) Due to noise, light changes and unclear object contours, the accuracy of the detection results is easily affected by environmental interference; (2) The target cannot be detected and tracked in real time in complex scenes. Summary of the invention

[0004] The implementation methods of the present application mainly solve the technical problem of how to detect and track targets in real time and accurately in complex scenarios where the targets are moving rapidly.

[0005] To solve the above technical problems, a technical solution adopted in the implementation mode of the present application is: to provide a target tracking method, the method comprising: performing pixel weighting on the original image data to obtain target image data; performing rotation box annotation on the target image data to obtain annotation information of the target image data; performing feature learning on the target image data based on the annotation information to obtain ghost feature information; obtaining a target detection model based on the ghost feature information; and obtaining a target tracking result based on the target detection model.

[0006] In some embodiments, the pixel weighting of the original image data to obtain the target image data includes: performing a preprocessing operation on the original image data to obtain processed original image data; selecting a preset number of frames of the original image data to perform pixel-by-pixel weighting operations to generate the weighted target image data.

[0007] In some embodiments, the target image data is annotated with a rotating box to obtain the annotation information of the target image data, including: performing region extraction based on the target image data to obtain a region of interest; performing minimum enclosing rectangle calculation on the region of interest to obtain boundary information of the rotating box; and generating the annotation information of the target image data based on the boundary information.

[0008] In some embodiments, the feature learning is performed on the target image data according to the annotation information to obtain the ghost feature information, including: constructing a data set and a label set of the target image according to the target image data and the annotation information; dividing the data set in a preset ratio according to the data set and the label set to obtain a training set, a validation set and a test set, and a label set corresponding to the training set, the validation set and the test set; training a preset convolutional neural network according to the training set and the label set corresponding to the training set, and updating and saving weight parameters; verifying the trained convolutional neural network according to the validation set and the label set corresponding to the validation set to obtain a loss value of the validation set; when the loss value of the validation set is within a preset range for a preset number of consecutive iterations, saving the weight parameters as the ghost feature information.

[0009] In some embodiments, obtaining a target detection model based on the ghost feature information includes: inputting the ghost feature information as an initialization weight into the target detection model to initialize the network weights to obtain the target detection model; and fine-tuning the parameters of the target detection model to obtain the target detection model.

[0010] In some embodiments, obtaining the target tracking result based on the target detection model includes: inputting the test set and the label set corresponding to the test set into the target detection model for identification to obtain the detection result of the target to be tracked; based on the detection result, tracking the target to be tracked through the target tracking model to obtain the target tracking result.

[0011] In some embodiments, based on the detection result, the target to be tracked is tracked by a target tracking model to obtain the target tracking result, including: inputting the detection result into the target tracking model for initialization; the initialized target tracking model performs target positioning and update processing on the target to be tracked in subsequent frames to obtain the target tracking result.

[0012] To solve the above technical problems, another technical solution adopted in the implementation mode of the present application is: to provide a target tracking device, the target tracking device comprising: an acquisition module, the acquisition module is used to perform pixel weighting on the original image data to obtain target image data; a labeling module, the labeling module is used to perform rotating box labeling on the target image data to obtain labeling information of the target image data; a feature learning module, the feature learning module is used to perform feature learning on the target image data according to the labeling information to obtain ghost feature information; a target detection module, the template detection module is used to obtain a target detection model according to the ghost feature information; a target tracking module, the target tracking module is used to obtain a target tracking result according to the target detection model.

[0013] To solve the above technical problems, another technical solution adopted in the implementation mode of the present application is: to provide a smart vending cabinet, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described above.

[0014] In order to solve the above technical problems, another technical solution adopted in the implementation mode of the present application is: providing a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are executed by the smart vending cabinet, the smart vending cabinet executes the method described above.

[0015] Different from the related art, the present application provides a target tracking method, device, smart vending machine and storage medium. Target image data is obtained by pixel weighting the original image data; rotating frame annotation is performed on the target image data to obtain annotation information of the target image data; feature learning is performed on the target image data according to the annotation information to obtain smear feature information; a target detection model is obtained according to the smear feature information; and a target tracking result is obtained according to the target detection model. Target image data with smear features is obtained by pixel weighting the original data, and then rotating frame annotation and feature learning are performed on the target image data, so that the target detection model learns to recognize the smear features, enhances the understanding of the target motion pattern, and facilitates subsequent target detection and tracking. In this way, the present application realizes accurate and real-time detection and tracking of targets in complex scenes where targets move rapidly. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] One or more embodiments are exemplarily described by corresponding drawings, which do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and the figures in the drawings do not constitute proportional limitations unless otherwise stated.

[0017] Figure 1 is a flow chart of a target tracking method provided by an embodiment of the present application;

[0018] Figure 2 is a flowchart of a pixel weighting process provided by an embodiment of the present application;

[0019] Figure 3 is a flow chart of the rotation box annotation process provided by an embodiment of the present application;

[0020] Figure 4 is a flow chart of a feature learning process provided by an embodiment of the present application;

[0021] Figure 5 is a schematic block diagram of the structure of a target tracking device provided in an embodiment of the present application;

[0022] Figure 6 It is a schematic diagram of the hardware structure of the smart vending cabinet for executing the target tracking method provided in the embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0024] It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, all within the scope of protection of the present application. In addition, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a different order than the module division in the device schematic diagram or the order in the flow chart.

[0025] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used in this specification includes any and all combinations of one or more of the related listed items.

[0026] See also Figure 1 , Figure 1 : is a flow chart of a target tracking method provided by an embodiment of the present application. Figure 1 As shown, the method includes steps S101-S105:

[0027] S101: Perform pixel weighting on original image data to obtain target image data.

[0028] See also Figure 2 , Figure 2 is a flowchart of the pixel weighting process provided by the embodiment of the present application. Figure 2 As shown, the pixel weighting of the original image data to obtain the target image data includes steps S1011-S1012:

[0029] S1011: performing a preprocessing operation on the original image data to obtain processed original image data.

[0030] First, obtain continuous video frames from the target scene. There should be enough video frames to ensure that the target's motion trajectory can be captured. In order to accurately track fast-moving targets, the frame rate of the video frame needs to be high enough (generally speaking, 30 frames per second or higher). Extract continuous image frames from the video. Each frame of the image is usually a color image (RGB format) or a grayscale image (single channel), depending on the requirements of target detection and tracking.

[0031] After obtaining the original image data, the image data can be preprocessed to eliminate unnecessary noise or convert the image data into a format suitable for subsequent calculations. The preprocessing operations provided in this embodiment include: Grayscale: If the original image is in color, it can be converted into a grayscale image. After grayscale, the image contains only brightness information and removes color information, which can simplify calculations in many image processing tasks. Denoising: Video frames may contain noise, especially in low light or motion blur. Methods such as median filtering and Gaussian filtering can be used to remove noise and smooth the image. For example, applying Gaussian blur: Smoothing the image using a Gaussian kernel through a convolution operation reduces high-frequency noise. Normalization: Normalizing the pixel values ​​of the image so that the pixel range of the image is uniform (such as between 0 and 1). A common method is to divide the pixel value by 255 (for 8-bit images) or use mean and standard deviation normalization to make the image data suitable for model processing. Image alignment: For videos in dynamic scenes, images may need to be registered or corrected to avoid image offsets caused by motion. This step is particularly important for tracking tasks, especially when the target is moving quickly.

[0032] Through the above preprocessing steps, the consistency of continuous frame data is ensured, and clean and standardized input data are prepared for subsequent weighted method operations.

[0033] S1012: Selecting a preset number of frames of the original image data to perform pixel-by-pixel weighted operations to generate the weighted target image data.

[0034] After preprocessing the original image data, you can select a preset number of frames (for example, 5 consecutive frames) of original image data for weighted operations. This weighted method can retain the information of stationary objects, highlight the dynamically changing parts, and form a smear phenomenon. The weighted operations include: Acquisition of continuous frames: Extract 5 consecutive frames of images from the original image data, assuming that these frames are img(n), img(n-1), img(n-2), img(n-3), and img(n-4). Pixel-level weighted operations: For each pixel, use the weighted method to obtain the weighted target image data ds_img(x, y). The calculation formula is as follows:

[0035] ds_img(x,y)

[0036] =[img(n)(x,y)+img(n-1)(x,y)+img(n-2)(x,y)

[0037] +img(n-3)(x,y)+img(n-4)(x,y)] / 5

[0038] Among them, (x, y) represents the coordinates of the pixel currently being processed, and the value of each pixel is the average of the corresponding pixel values ​​of these five frames.

[0039] In addition, the linear weighting method can also be used for pixel-level weighting operations, and the calculation formula is as follows:

[0040] ds_img(x,y)

[0041] =[w0·img(n)(x,y)+w1·img(n-1)(x,y)+w2

[0042] ·img(n-2)(x,y)+w3 ·img(n-3)(x,y)+w4

[0043] ·img(n-4)(x,y)] / (w0+w1+w2+w3+w4)

[0044] Among them, w0 to w4 are weights of different frames, which can be designed to decay over time.

[0045] It should be noted that in the weighted method operation, the pixel values ​​of multiple frames are combined. When an object moves quickly between frames, its position in different frames will change. By adding or weighted averaging the pixel values ​​of these different frames, the pixel information of the moving object will be superimposed together to form a smear effect. Since the weighted method operation mainly processes each pixel one by one, the pixel values ​​of stationary objects in multiple frames remain consistent, so after weighted synthesis, the pixel values ​​of these areas will not change. For moving objects, their positions in different frames will cause changes in pixel values, thereby generating smears in the weighted results. In addition, the weighted method emphasizes the pixel features of the changing parts (i.e., moving objects), making these areas more obvious in the final weighted image. Smears usually appear as blurred paths that can reflect the direction and speed of the object's movement. The positions of fast-moving objects in different frames will form a directional track. Through weighted processing, this directionality is more prominent in the image, making it easier for the tracking system to identify the object's motion pattern. It can be understood that the smear feature is an important visual clue that can help understand and analyze the movement of objects in dynamic scenes. By capturing these features, object detection and tracking systems can more effectively cope with rapidly changing environments.

[0046] S102: Annotate the target image data with a rotating frame to obtain annotation information of the target image data.

[0047] See also Figure 3 , Figure 3 Flowchart of the rotating frame marking process provided by the embodiment of the present application. Figure 3 As shown, the target image data is annotated with a rotating frame to obtain the annotation information of the target image data, including steps S1021-S1023:

[0048] S1021: Perform region extraction according to the target image data to obtain a region of interest.

[0049] First, use a fixed threshold or an adaptive threshold to binarize the image to separate the target area from the background area. Common methods include global thresholding (such as Otsu's method) or adaptive threshold segmentation. Second, use the Canny edge detection algorithm or the Sobel operator to detect edges in the image. By detecting edges, the outline of the target object can be clearly identified, which helps to extract the region of interest (ROI). Finally, extract contours from the edge detection results using algorithms such as findContours (OpenCV function). Contours represent the boundaries of each independent area in the image. Select and extract the region of interest (ROI) based on the contour area or shape characteristics. For example, you can exclude too small noise areas by setting a minimum area threshold.

[0050] Through region extraction, irrelevant parts of the image are removed and the focus is on the target area, reducing the amount of calculation in subsequent steps. Focusing on the region of interest can improve the accuracy of subsequent target detection, especially for images with small targets or complex backgrounds.

[0051] S1022: Calculate the minimum bounding rectangle of the region of interest to obtain boundary information of the rotation box.

[0052] The minimum bounding rectangle (MBR) is a rectangular box used to accurately enclose the target object. For tilted or rotated targets, the minimum bounding rectangle not only considers the horizontal and vertical boundaries of the target, but also the rotation angle of the target. Its implementation steps include: First, contour extraction: from the region of interest (ROI) extracted in step S1021, use the contour extraction algorithm (such as findContours in OpenCV) to obtain the edge or contour of the target object. Each contour represents an independent area and may correspond to an object in the image. Secondly, calculate the minimum bounding rectangle: for each detected contour, use the minimum bounding rectangle calculation (minAreaRect()) to determine the minimum rectangular box of the contour. This rectangular box is the rectangle that is closest to the edge of the target object, and the rectangle can be rotated (that is, it can have an angle). Among them, the result returned by the minAreaRect() function includes: the center coordinates of the rectangle: the center point position of the rectangle, usually (x, y) coordinates. The width and height of the rectangle: the length and width of the rectangle can be different. Rotation angle: the rotation angle relative to the horizontal axis, indicating the direction of the rectangular box. Finally, the parameters of the rotation box are obtained: Based on the minimum enclosing rectangle result calculated above, the coordinates of the four vertices and the rotation angle of the rotation box can be extracted. This information will be used for subsequent annotation generation.

[0053] By calculating the minimum enclosing rectangle, the bounding box of the object can be effectively matched to the shape of the object. Even if the object is rotated or tilted, the minimum enclosing rectangle can still accurately surround the target. In addition, the rotating box can handle the positioning problem of the target at different angles and adapt to the change of the object's direction.

[0054] S1023: Generate annotation information of the target image data according to the boundary information.

[0055] After calculating the boundary information of the rotating box in step S1022, this information is converted into a standard annotation format for subsequent training or evaluation. First, the rotating box information is extracted: according to the result of the minimum circumscribed rectangle, the coordinates of the four corner points of the rotating box (x1, y1), (x2, y2), (x3, y3), (x4, y4) and the rotation angle are obtained. Secondly, the annotation information is generated, and the annotation information includes: target category: if it is a multi-category target detection task, the category label of the target needs to be attached, such as "car", "pedestrian", etc. Rotating box boundary information: including the four vertex coordinates and rotation angle of the rotating box, as the position and posture information of the target in the image. Save format: save the annotation information in a standard format (such as XML, JSON, CSV, etc.) for subsequent processing and use. Finally, storage and application: save the rotating box annotation information to a database or file system for use in training and reasoning. This information can not only be used for target detection, but also for subsequent target tracking, behavior recognition and other tasks.

[0056] S103: Perform feature learning on the target image data according to the annotation information to obtain smear feature information.

[0057] See also Figure 4 , Figure 4 is a flow chart of the feature learning process provided by the embodiment of the present application. Figure 4 As shown, according to the annotation information, feature learning is performed on the target image data to obtain the smear feature information, including steps S1031-S1035:

[0058] S1031: Construct a data set and a label set of the target image according to the target image data and the annotation information.

[0059] After obtaining the target image data and annotation information through the above steps S101 and S102, they are constructed into a target image data set and a label set for learning the smear feature information. It can be understood that these target image data are target image data with smear features. The label set corresponds to the image data set one by one, and the label of each image contains the target type in the image and the corresponding bounding box information (such as a rotation box or a rectangular box). The annotation information includes information such as the specific category of the target, the position of the box, and the rotation angle.

[0060] S1032: Divide the data set into a preset ratio according to the data set and the label set to obtain a training set, a validation set, and a test set, as well as label sets corresponding to the training set, the validation set, and the test set.

[0061] It is understandable that the data set division usually uses a preset ratio, such as: 70% for training set, 15% for validation set, and 15% for test set. Random division can also be used to ensure data diversity. The divided training set is used to train the convolutional neural network (CNN) model, the validation set is used to regularly verify the model performance and adjust the hyperparameters during the training process, and the test set is used to finally evaluate the model performance and does not participate in the training process. The label set corresponds to the data set division one by one. For the training set, validation set, and test set, the content of the label set matches the image data.

[0062] S1033: According to the training set and the label set corresponding to the training set, a preset convolutional neural network is trained, and weight parameters are updated and saved.

[0063] The specific implementation process of this step is as follows: Network selection and initialization: Select a suitable convolutional neural network to extract features from the image and perform target positioning and classification. In many applications, using a model pre-trained on a large-scale dataset (such as ImageNet) as the starting model can accelerate the training process and improve the learning effect. This model is initialized by learning general image features (such as edges, textures, etc.). Preprocessing of input data: Take each frame of the image as input, the image is normalized (for example, the pixel value is scaled to between [0,1] or between [-1,1]) and the size is adjusted according to the input requirements of the model. Iterative training: Through multiple iterative training, use optimization algorithms (such as Adam, SGD, etc.) to update the weight parameters in the network, so that the error between the network output and the actual annotation is gradually reduced. During the training process, the weights of the network are continuously updated, and finally a trained convolutional neural network is obtained.

[0064] S1034: Verify the trained convolutional neural network according to the verification set and the label set corresponding to the verification set to obtain a loss value of the verification set.

[0065] The image data and label data in the validation set are input into the trained convolutional neural network, and the loss value of each prediction result is calculated. The validation process is to evaluate the performance of the model by calculating the difference between the predicted box and the true box (such as IOU loss, regression loss, etc.). During the validation process, the loss value (such as cross entropy loss or regression loss) is output. This loss value reflects the performance of the model on the validation set. The lower the loss value, the better the model.

[0066] S1035: When the loss value of the validation set is within a preset range for a preset number of consecutive iterations, the weight parameter is saved as the smear feature information.

[0067] A key link in the training process is to determine whether the model has converged. It is usually judged by observing whether the loss value of the validation set remains within a certain range (for example, less than 0.002) for several consecutive times (for example, 5 or 10 times). If the loss value no longer decreases significantly, or tends to be stable after multiple iterations, it means that the model has converged. It can be understood that when the loss value of the validation set is within the preset range for a preset number of iterations, it means that the model has learned relatively stable features and the weight parameters will be saved. The saved weights contain the smear feature information learned by the model, and these weights are used for subsequent target detection tasks. After the training and verification process is completed, the current optimal network weights are saved. These weights contain the network's learning results for the smear features, that is, the optimal network weights are the learned smear feature information.

[0068] S104: Obtaining a target detection model according to the smear feature information.

[0069] In some embodiments, obtaining a target detection model based on the ghost feature information includes: inputting the ghost feature information as an initialization weight into the target detection model to initialize the network weights to obtain the target detection model; and fine-tuning the parameters of the target detection model to obtain the target detection model.

[0070] It can be understood that, first of all, after the convolutional neural network is trained, the network weights obtained represent the learned smear feature information. These weights contain the model's learning results of the smear feature and can accurately identify the smear phenomenon in the image. In the target detection task, the model needs to learn how to identify targets and locate them. In this step, the learned smear feature weights are used as initialization parameters and input into the target detection model (for example, YOLO, Faster R-CNN, etc.). These initialized weights can help the target detection model to be trained more efficiently on specific tasks, because the smear feature information has provided the model with useful prior knowledge, especially in the target image with smear, the model will be more likely to identify the target features. It can be understood that by using the smear feature information as the initialization weight, the target detection model has gained a preliminary understanding of the smear feature, so that it has better detection capabilities in subsequent tasks.

[0071] In addition, the model needs to be fine-tuned so that it can better adapt to the specific object detection task, especially on new image data. It is understandable that a new, fully annotated training set can be used, which contains the target images in the object detection task (which may or may not have smear features). The labels of this training set include the category, location, rotation box information, etc. of the target. Parameter optimization is performed in the fine-tuning stage using the loss function of the object detection task (such as cross entropy loss, regression loss, etc.). Through the back propagation algorithm, the network adjusts the internal weights according to the new training data, so that the model better fits the needs of the object detection task. Usually, a smaller learning rate is used for fine-tuning, because the initialized weights already provide more suitable features, so there is no need to make large weight updates during fine-tuning, only minor adjustments need to be made on the existing basis. After fine-tuning, the object detection model can not only identify smear features, but also locate and classify targets more accurately. This process enhances the generalization ability of the model and improves the performance of the model in new scenarios.

[0072] It should be noted that since the smear feature information has been loaded during initialization, the model does not need to learn these features from scratch, and can converge faster and shorten the training time. In addition, the target detection model has stronger performance in specific tasks (such as smear detection, fast motion tracking, etc.) with the help of the initialization weights of the smear features, especially in scenes with smear features, the accuracy is improved. Through fine-tuning, the model can better adapt to different target detection tasks, especially when facing different types of targets or different lighting and perspective changes, and maintain good robustness.

[0073] S105: Obtaining a target tracking result according to the target detection model.

[0074] In some embodiments, obtaining the target tracking result based on the target detection model includes: inputting the test set and the label set corresponding to the test set into the target detection model for identification to obtain the detection result of the target to be tracked; based on the detection result, tracking the target to be tracked through the target tracking model to obtain the target tracking result.

[0075] According to the detection result, the target to be tracked is tracked by a target tracking model to obtain the target tracking result, including: inputting the detection result into the target tracking model for initialization; the initialized target tracking model performs target positioning and update processing on the target to be tracked in subsequent frames to obtain the target tracking result.

[0076] Object detection and object tracking are two closely related processes. In order to achieve the target tracking result, it is necessary to first use the target detection model to identify the target, and then use the target tracking algorithm to track the detection result.

[0077] It can be understood that the target detection stage specifically includes: Input data: The test set is used for reasoning of the detection model, and the label set provides the real label for the test set, which usually contains the bounding box of each target, the target category and other information, mainly used to verify the accuracy of the model. Target detection model output: In the target detection stage, the model processes each input test image and outputs the category, position (bounding box coordinates or rotation box parameters), confidence, etc. of each detected target. For each test image, the target detection model may detect multiple targets, and each target has a location information and category. Among them, the detection results of each frame include: target category (for example, people or car lights), bounding box coordinates (x_min, y_min, x_max, y_max) or rotation box parameters (center coordinates, width and height, angle), confidence (the probability that the target is correctly classified).

[0078] The task of the target tracking model is to continuously track the target based on the target detection results. It infers the new position of the target through the target position of the previous frame and the image data of the current frame, and locates and updates the target. Specifically, it includes: Initializing the target tracking model: The target information detected in the target detection stage (such as target category, bounding box coordinates, etc.) is passed as input data to the target tracking model. The target tracking model initializes the state of the tracked target based on the input target detection results. For example, in the first frame of the image, the model will record the position and category of the target and prepare for the subsequent tracking process. For example, assuming that the target detection model recognizes a target (such as "person") with a bounding box of [50,100,150,200] in the first frame, the target tracking model will use this information to initialize the target state. Target positioning and updating in subsequent frames: First, the target tracking model will accept the next frame of the image as input and find the position of the target based on the target detection result of the first frame (that is, the initial position of the target). Secondly, the target tracking model locates the target in the current frame through various methods (such as Kalman filtering, optical flow method or deep learning method, etc.), and updates the state of the target (such as position, speed, etc.) according to the movement of the target. As each frame is processed, the target tracking model continuously updates the target's bounding box or rotation box to ensure that the target's position is tracked correctly. Finally, if the target moves quickly or is obscured, the tracking algorithm predicts and corrects the target's position to maintain tracking accuracy. For each frame, the target tracking model outputs the current target's position (such as updated bounding box coordinates or rotation box), category information, and target ID. The output includes: target ID (used to distinguish different targets), bounding box coordinates or rotation box coordinates and angles of each frame, target speed or motion trajectory, and confidence (during the tracking process, the confidence of the target tracking model can help determine whether the target is lost or mistracked).

[0079] In this embodiment, after the target position is initialized once, target tracking only needs to continuously update the target position, without the need for a new target detection for each frame, which greatly reduces the computational cost. In addition, target detection provides the precise position of the target, while target tracking corrects the target position through tracking and prediction algorithms to ensure the stability and accuracy of the tracking results. Even when the target moves rapidly, is blocked or changes, the target tracking algorithm can cope with these complex situations through updates and predictions, providing a stable tracking effect.

[0080] In the embodiment of the present application, the target image data is obtained by pixel weighting the original image data; the target image data is annotated with a rotating frame to obtain the annotation information of the target image data; the target image data is feature learned according to the annotation information to obtain the smear feature information; the target detection model is obtained according to the smear feature information; and the target tracking result is obtained according to the target detection model. Since the fast-moving object leaves pixel values ​​at different positions in the continuous frames, these information form the target image data with obvious smear features after pixel-level weighting processing of the original image data. The smear feature information is learned by performing rotating frame annotation and feature learning on the target image data. These smear feature information not only provides the position of the object, but also reflects its direction and speed of movement. Using the smear feature information in the target detection model can enable the model to enhance its understanding of the target motion pattern. In a practical environment, when the smear phenomenon occurs, the trained model can be used to quickly adapt and maintain the tracking effect. This feature enables stable tracking capabilities to be maintained in complex scenes (such as fast-paced movement or light changes). In addition, since the learning of the smear feature can be done in advance by pre-training with the data set, no further training is required in the subsequent target detection and target tracking process (i.e., the subsequent target detection model already has the characteristics of identifying the smear feature), and the target detection and tracking can be directly performed on the image data obtained in the actual scene, so as to achieve real-time analysis of the smear feature, timely adjust the tracking strategy for the target, and reduce the risk of tracking loss. In this way, the present application can not only accurately detect and track targets in complex scenes where the target moves quickly, but also detect the status of the target in real time.

[0081] Based on the target tracking method provided in the above embodiment, the present application embodiment further provides a target tracking device. Figure 5 , Figure 5 This is a schematic diagram of the structure of the device. Figure 5 As shown, the device 200 includes: an acquisition module 210, a labeling module 220, a feature learning module 230, a target detection module 240 and a target tracking module 250.

[0082] Among them, the acquisition module 210 is used to perform pixel weighting on the original image data to obtain the target image data; the labeling module 220 is used to perform rotating box labeling on the target image data to obtain the labeling information of the target image data; the feature learning module 230 is used to perform feature learning on the target image data according to the labeling information to obtain the ghost feature information; the target detection module 240, the template detection module is used to obtain the target detection model according to the ghost feature information; the target tracking module 250, the target tracking module is used to obtain the target tracking result according to the target detection model.

[0083] It should be noted that the target tracking device can execute the target tracking method provided in the embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in the target tracking device embodiment, please refer to the target tracking method provided in the embodiment of the present application.

[0084] The present application also provides a smart vending cabinet. Figure 6 , which shows the hardware structure of the smart vending cabinet capable of executing the method described in the above embodiment. The smart vending cabinet 300 includes: at least one processor 310; and a memory 320 in communication with the at least one processor 310. Figure 6 In the embodiment, a processor 310 is taken as an example. The memory 320 stores instructions that can be executed by the at least one processor 310. The instructions are executed by the at least one processor 310 so that the at least one processor 310 can execute the target tracking method described in the above embodiment. The processor 310 and the memory 320 can be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0085] The memory 320 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the target tracking method in the embodiment of the present application. The processor 310 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 320, that is, implementing the target tracking method described in the above embodiment.

[0086] The memory 320 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created according to the use of the computing device, etc. In addition, the memory 320 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 320 may optionally include a memory remotely arranged relative to the processor 310, and these remote memories may be connected to the computing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0087] The one or more modules are stored in the memory 320, and when executed by the one or more processors 310, the target tracking method described in the above embodiment is executed.

[0088] The above product can execute the method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the target tracking method described in any embodiment of the present application.

[0089] The embodiment of the present application also provides a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by one or more processors so that the at least one processor can execute the target tracking method described in the above embodiment. For example, the non-volatile computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CDROM), a magnetic tape, a floppy disk, and an optical data storage device.

[0090] It should be noted that the above described embodiments are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0091] Through the description of the above implementation methods, ordinary technicians in this field can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Ordinary technicians in this field can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Under the concept of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes in different aspects of the present application as described above, which are not provided in detail for the sake of simplicity. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A target tracking method, characterized in that: The method comprises: Perform pixel weighting on the original image data to obtain target image data; Performing a rotating frame annotation on the target image data to obtain annotation information of the target image data; Performing feature learning on the target image data according to the annotation information to obtain smear feature information; Obtaining a target detection model according to the smear feature information; According to the target detection model, a target tracking result is obtained.

2. The target tracking method according to claim 1, characterized in that: The step of performing pixel weighting on the original image data to obtain target image data includes: Performing a preprocessing operation on the original image data to obtain processed original image data; The original image data of a preset number of frames are selected to perform pixel-by-pixel weighted operations to generate the weighted target image data.

3. The target tracking method according to claim 1, characterized in that: The performing rotating frame marking on the target image data to obtain the marking information of the target image data includes: Performing region extraction according to the target image data to obtain a region of interest; Calculating the minimum bounding rectangle of the region of interest to obtain boundary information of the rotation frame; Generate annotation information of the target image data according to the boundary information.

4. The target tracking method according to claim 1, characterized in that: The step of performing feature learning on the target image data according to the annotation information to obtain the smear feature information includes: Constructing a data set and a label set of the target image according to the target image data and the annotation information; According to the data set and the label set, the data set is divided into a preset ratio to obtain a training set, a validation set and a test set, and a label set corresponding to the training set, the validation set and the test set; According to the training set and the label set corresponding to the training set, a preset convolutional neural network is trained, and weight parameters are updated and saved; Verifying the trained convolutional neural network according to the verification set and the label set corresponding to the verification set to obtain a loss value of the verification set; When the loss value of the validation set is within a preset range for a preset number of iterations, the weight parameter is saved as the smear feature information.

5. The target tracking method according to any one of claims 1 to 4, characterized in that: The step of obtaining a target detection model according to the smear feature information includes: Inputting the smear feature information as initialization weights into the target detection model to initialize network weights, thereby obtaining the target detection model; Fine-tune the parameters of the target detection model to obtain the target detection model.

6. The target tracking method according to claim 4, characterized in that: Obtaining a target tracking result according to the target detection model includes: Inputting the test set and the label set corresponding to the test set into the target detection model for identification, and obtaining a detection result of the target to be tracked; According to the detection result, the target to be tracked is tracked by a target tracking model to obtain the target tracking result.

7. The target tracking method according to claim 6, characterized in that: The step of tracking the target to be tracked by using a target tracking model according to the detection result to obtain the target tracking result includes: Inputting the detection result into the target tracking model for initialization; The initialized target tracking model performs target positioning and update processing on the target to be tracked in subsequent frames to obtain the target tracking result.

8. A target tracking device, characterized in that: The target tracking device comprises: An acquisition module, the acquisition module is used to perform pixel weighting on the original image data to obtain target image data; A marking module, wherein the marking module is used to perform a rotating frame marking on the target image data to obtain marking information of the target image data; A feature learning module, wherein the feature learning module is used to perform feature learning on the target image data according to the annotation information to obtain smear feature information; A target detection module, wherein the template detection module is used to obtain a target detection model according to the smear feature information; A target tracking module is used to obtain a target tracking result according to the target detection model.

9. An intelligent vending machine, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are executed by the smart vending cabinet, the smart vending cabinet executes the method according to any one of claims 1 to 7.

Images