A constant velocity linear motion target class estimation method

Abstract

The application discloses a constant-speed linear motion target category estimation method. The method of the application reduces the interference of environmental factors such as light and background on the target category prediction result by averaging the category prediction results of the same target in different image frames, and improves the accuracy of the visual system in target category prediction. A detection interval is set in the camera field of view, images are collected at a fixed frame rate, the category and position information of the target are obtained through a target detection algorithm, and when the target is in the detection interval, the category prediction result of the current frame of the target is stored in a circular queue; when the target first leaves the detection interval, the category prediction results of the target in different frames are averaged to obtain the final category prediction result of the target.

Description

Technical Field

[0001] This invention relates to a target detection vision system, specifically to a method for estimating the category of a target moving at a constant speed in a straight line, thereby improving the accuracy of the target category estimation system in predicting the category of a target moving at a constant speed in a straight line on a conveyor belt. Background Technology

[0002] With the development of computer technology, vision systems based on object detection algorithms have been widely used in various fields such as packaging defect detection and product sorting.

[0003] The introduction of target detection algorithms has improved the efficiency of product classification and sorting on the production line. However, products on the production line are in motion, and at different times, due to environmental interference such as lighting and background, the target detection algorithm's prediction of the target category will deviate. If the prediction result of the interfered image frame is used as the category prediction result, it will increase the false detection and false negative rates of the system and reduce the system's reliability.

[0004] Meanwhile, the products on the production line conveyor belt are mostly in a linear motion state, and the conveyor belt speed is relatively stable. Based on this, this paper proposes a linear uniform motion target category estimation method based on inter-frame mean filtering. By balancing the target category prediction results at different times, the accuracy of the system's target category prediction is improved, and the reliability of the system is increased. Summary of the Invention

[0005] The purpose of this invention is to reduce the interference of environmental factors such as lighting and background on the prediction of moving target categories on a conveyor belt, improve the accuracy of the vision system in predicting target categories, and increase the reliability of the system.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for estimating the category of a target moving in uniform linear motion based on inter-frame mean filtering includes the following steps:

[0008] 1) Acquire images of a specified area along the path of the target on the conveyor belt at a fixed frame rate;

[0009] 2) Perform visual algorithm detection on the acquired image to determine whether a target exists and obtain the category and pixel coordinate prediction results of all targets in the current image; if no target exists in the current image, return to step 1) to reacquire the image;

[0010] 3) Calculate the relative coordinates x of all targets in the current image in the direction of the conveyor belt's movement, and sort the targets according to x from largest to smallest;

[0011] 4) Select targets for processing according to the sorting order. If the current target satisfies x < α, discard the prediction results of the current target and the remaining unprocessed targets in the current frame, and directly return to step 1); if the current target satisfies α ≤ x ≤ 1 - α, store the target's category and pixel coordinate prediction results; if the current target satisfies 1 - α < x ≤ 1 - α + d, average the category prediction results of the current target stored in the previous frame as the category estimation result of the target, and delete all stored category and pixel coordinate prediction results of the current target; if the current target satisfies 1 - α + d < x, do not process the target's category and pixel coordinate prediction results; where α is half the length of the current target in the direction of the conveyor belt in the image, and d is the offset of the target's position in two adjacent frames.

[0012] 5) After the category estimation of one target is completed, select the next target and repeat the operation of step 4); if the prediction results of all targets in the current image have been processed, return to step 1) to acquire the next image and estimate the category of the target in the image.

[0013] The present invention also provides a target category estimation system for implementing the above-described uniform linear motion target category estimation method, comprising:

[0014] The image acquisition module is used to acquire images of a specified area along the target's travel route at a fixed frame rate;

[0015] The visual algorithm detection module is used to perform visual algorithm detection on the acquired image to determine whether there is a target. If there is a target, it obtains the category and pixel coordinate prediction results of all targets in the current image.

[0016] The target sorting module is used to sort the targets in the image and match the sorted targets with the predicted target categories and pixel coordinates.

[0017] The target category prediction result filtering and evaluation module is used to filter the target category prediction results based on the target's relative position in the image and estimate the target's category.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] This invention addresses the problem that environmental factors such as illumination and background can interfere with the category prediction results of moving targets on a conveyor belt in target detection vision systems. By balancing the category prediction results of the target at different times, the interference of environmental factors such as illumination and background on the category prediction of the target at different positions is reduced, thereby improving the accuracy of the vision system in category prediction and increasing the reliability of the system. Attached Figure Description

[0020] Figure 1This is a flowchart of the method of the present invention.

[0021] Figure 2 This is a diagram of a circular queue structure used to store target information in the method of this invention.

[0022] Figure 3 This is a visual system diagram according to an embodiment of the present invention.

[0023] Figure 4 This is the target test diagram of an embodiment of the present invention. Detailed Implementation

[0024] The present invention will be further described and illustrated below with reference to specific embodiments. The embodiments described are merely examples of the content of this disclosure and do not limit the scope of the invention. The technical features of each embodiment in the present invention can be combined accordingly, provided that there is no mutual conflict.

[0025] like Figure 1 As shown, a method for estimating the category of a target moving in uniform linear motion includes the following steps:

[0026] 1) Acquire images of a specified area along the path of the target on the conveyor belt at a fixed frame rate;

[0027] 2) Perform visual algorithm detection on the acquired image to determine whether a target exists and obtain the category and pixel coordinate prediction results of all targets in the current image; if no target exists in the current image, return to step 1) to reacquire the image;

[0028] 3) Calculate the relative coordinates x of all targets in the current image in the direction of the conveyor belt's movement, and sort the targets according to x from largest to smallest;

[0029] 4) Select targets for processing according to the sorting order. If the current target satisfies x < α, discard the prediction results of the current target and the remaining unprocessed targets in the current frame, and directly return to step 1); if the current target satisfies α ≤ x ≤ 1 - α, store the target's category and pixel coordinate prediction results; if the current target satisfies 1 - α < x ≤ 1 - α + d, average the category prediction results of the current target stored in the previous frame as the category estimation result of the target, and delete all stored category and pixel coordinate prediction results of the current target; if the current target satisfies 1 - α + d < x, do not process the target's category and pixel coordinate prediction results; where α is half the length of the current target in the direction of the conveyor belt in the image, and d is the offset of the target's position in two adjacent frames.

[0030] 5) After the category estimation of one target is completed, select the next target and repeat the operation of step 4); if the prediction results of all targets in the current image have been processed, return to step 1) to acquire the next image and estimate the category of the target in the image.

[0031] The visual algorithm used in step 2) is a target detection algorithm based on traditional vision or deep learning to predict the target category and location information. In this embodiment, the YOLOv5s network is selected to predict the category and location of food packaging moving at a constant speed in a straight line on a conveyor belt.

[0032] The target moves in a straight line in the vision system. The coordinates of the target in the image that are in the same direction of movement increase over time. In step 3), the relative coordinates of the target in the image are used as the identifier to distinguish different targets.

[0033] In step 4), the interval (α, 1-α) (α < 0.5) is used as the target detection interval to prevent incomplete target information from interfering with the prediction results when the target partially enters or leaves the field of view. In one embodiment of the present invention, considering the size of the field of view occupied by the food packaging, the interval (0.3, 0.7) (α = 0.3) is selected as the target detection interval to ensure that the target appears in multiple frames within the detection interval.

[0034] The target moves at a constant speed in the vision system, and the positional offset is the same within the same interval. In step 4), a dynamic parameter d is used as the threshold for the target to first leave the detection interval. d is the positional offset of the target within two frames of images, which is dynamically updated during system operation. Assume the target's position in the nth frame is x. n Then the dynamic parameter d at this time n The update formula is as follows:

[0035]

[0036] Where, d n-1 The dynamic parameters are from the previous frame, and t is the order of magnitude of the dynamic parameter d, to prevent abnormal target motion in the vision system from affecting parameter updates.

[0037] like Figure 2 As shown, in step 4), a circular queue Q of length L is used to store target information. Each element in Q is an array. The first element of the array stores the relative coordinate x of the target's motion direction in the current frame, and the remaining elements store the category prediction results of the current target in different frames.

[0038] In a circular queue Q, the front pointer points to the front element and the rear pointer points to the rear element. The front and rear pointers satisfy the following relationship:

[0039] a) When the circular queue Q is empty, i.e., there is no target, front == rear;

[0040] b) When the circular queue Q is full, that is, when the current detection interval reaches the maximum number of targets, (rear+1)%L==front;

[0041] c) Enqueue Q of the circular queue, i.e., when the target first appears in the detection interval, rear = (rear+1)%L;

[0042] d) When the Q element of the circular queue is dequeued, that is, when the target leaves the detection interval for the first time, front = (front+1)%L.

[0043] In step 4), the target's class prediction results across all frames are used as the final result. Assuming the target appears in n image frames within the detection interval, the class prediction results are (p1, p2, ..., p...). n The final category prediction result is:

[0044]

[0045] The category prediction result is a positive integer. The final category result is obtained by averaging the category prediction results of all image frames, adding the offset, and then rounding down.

[0046] The present invention also provides a target category estimation system for implementing the above-mentioned uniform linear motion target category estimation method, including an image acquisition module, a visual algorithm detection module, a target sorting module, and a target category prediction result screening and evaluation module;

[0047] The image acquisition module acquires images of a specified area along the target's path at a fixed frame rate; the visual algorithm detection module performs visual algorithm detection on the acquired images to determine whether a target exists. If a target exists, it obtains the category and pixel coordinate prediction results of all targets in the current image; the target sorting module sorts the targets in the image and maps the sorted targets to the target category and pixel coordinate prediction results; the target category prediction result filtering and evaluation module filters the target category prediction results based on the target's relative position in the image and estimates the target's category.

[0048] The defect sorting system based on machine vision built in this embodiment is as follows: Figure 3As shown, the YOLOv5 object detection algorithm is used to predict the category and location information of food packaging on a conveyor belt, where the food packaging includes two categories: normal and defective. When the system identifies a package as defective at a certain moment, it controls the robot to sort the package. Due to environmental interference such as lighting and background, normal packaging may be considered defective at a certain moment and thus be sorted by the robot. By introducing the method of this invention, the category prediction results of the packaging are balanced at different times, which can reduce the probability of normal packaging being missorted and effectively reduce the false detection rate of the vision system. In this embodiment, each target can capture about 15 frames of images within the detection interval, and normal packaging is tested. Some captured images are shown below. Figure 4 As shown in the figure, the numbers identify the same package captured in different frames. The category prediction results for some normal targets are shown in Table 1 below:

[0049] Table 1 Prediction Results of Categories of Embodiments of the Invention

[0050]

[0051] In Table 1, 0 indicates a predicted normal package, and 1 indicates a predicted defective package. As can be seen from Table 1, if a target is considered a defective package at a certain moment and is immediately sorted, most normal packages will be missorted. This invention effectively reduces the system's missorting rate by balancing the target's category prediction results at different times.

[0052] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. Those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A method for estimating the category of a target moving at a constant velocity in a straight line, characterized in that, Includes the following steps: 1) Acquire images of a specified area along the path of the target on the conveyor belt at a fixed frame rate; 2) Perform visual algorithm detection on the acquired image to determine whether a target exists and obtain the category and pixel coordinate prediction results of all targets in the current image; if no target exists in the current image, return to step 1) to reacquire the image; 3) Calculate the relative coordinates of all targets in the current image along the direction of the conveyor belt's movement. and according to Sort the targets from largest to smallest; 4) Select targets for processing according to the sorting order. If the current target satisfies... If the current target and the remaining unprocessed targets are not predicted in the current frame, then discard the prediction results for the current target and the remaining unprocessed targets, and return directly to step 1); if the current target satisfies If the target satisfies the following conditions, then store the target's category and pixel coordinate prediction results; if the current target satisfies the following conditions, then store the target's category and pixel coordinate prediction results. Then, the average of the class prediction results of the current target stored in the previous frame is used as the class estimate result of the target, and all stored class and pixel coordinate prediction results of the current target are deleted; if the current target satisfies If the target category and pixel coordinate prediction results are not processed, then no processing is performed; where, It is half the length of the current target in the image along the direction of the conveyor belt's movement. This represents the offset of the target's position within two adjacent frames. 5) After the category estimation of one target is completed, select the next target and repeat the operation of step 4); if the prediction results of all targets in the current image have been processed, return to step 1) to acquire the next image and estimate the category of the target in the image.

2. The method according to claim 1, characterized in that, The visual algorithm in step 2) is a machine vision-based target detection algorithm used to predict target category and location information.

3. The method according to claim 1, characterized in that, The target moves in a straight line. At different times, the coordinates of the target on the image that are in the same direction of movement increase with time. In step 3), the relative coordinates of the target on the image are used as the identifier to distinguish different targets.

4. The method according to claim 1, characterized in that, Step 4) uses intervals The detection interval for the target, where This prevents incomplete target information from interfering with the prediction results when the target partially enters or leaves the field of view.

5. The method according to claim 1, characterized in that, The target moves at a constant speed in the vision system, and the target's position offset is the same in both frames. Step 4) uses dynamic parameters. As the threshold for the first departure of the target from the detection range, The position offset of the target within two image frames is dynamically updated during system runtime. Assume the target is in the [missing information - likely a specific frame or region]. The position of the frame is Then the first Frame dynamic parameters The update formula is as follows: ； in, These are the dynamic parameters from the previous frame. For dynamic parameters This is on the order of magnitude, preventing abnormal target motion in the vision system from affecting parameter updates.

6. The method according to claim 1, characterized in that, In step 4), a length of... circular queue Store target information. Each element in the array is a subarray, and the first element of the array stores the relative coordinates of the target's motion direction in the current frame. The remaining elements store the category prediction results for the current target in different frames.

7. The method according to claim 6, characterized in that, The circular queue use The pointer points to the target at the head of the queue, using The pointer points to the target at the rear of the queue. and The following relationship must be satisfied: Circular queue When empty, meaning no target exists, ; Circular queue "Full" means that the current detection interval has reached the maximum target number. ; Circular queue When an element joins the queue, that is, when the target first appears in the detection range. ; Circular queue Element dequeueing occurs when the target first leaves the detection range. .

8. The method according to claim 1, characterized in that, In step 4), the class prediction results of the target across all frames are used as the final result. It is assumed that the number of image frames in which the target appears within the detection interval is... The category prediction result is The final category prediction result is: ； Among them, the category prediction results The value is a positive integer. The final category prediction result is obtained by averaging the category prediction results of all image frames, adding the offset, and then rounding down.

9. A target category estimation system for implementing the uniform linear motion target category estimation method of claim 1, characterized in that, include The image acquisition module is used to acquire images of a specified area along the travel path of the target on the conveyor belt at a fixed frame rate. The visual algorithm detection module is used to perform visual algorithm detection on the acquired image to determine whether there is a target. If there is a target, it obtains the category and pixel coordinate prediction results of all targets in the current image. The target sorting module is used to sort the targets in the image and match the sorted targets with the predicted target categories and pixel coordinates. The target category prediction result filtering and evaluation module is used to filter the target category prediction results based on the target's relative position in the image and estimate the target's category.

Images