Method for improving recall rate of vehicle dynamic inspection

Abstract

The invention provides a method for improving the recall rate of vehicle dynamic inspection. The method comprises the following steps: S1, training a vehicle detection model; s2, adding movement detection: acquiring a rectangular frame of a moving target by using the binarized difference image; s3, adding a detection target historical frame and a moving object historical frame: obtaining an ID (Identity) of the detection frame by utilizing a Kalman tracking method, namely trackID, and judging whether the detection frames are the same target or not according to the trackID; the targets with the same trackID are put into a vector, and a historical frame of each vehicle and historical frames of all moving objects are obtained; s4, matching the moving frame and the vehicle frame: matching the vehicle and the moving object according to the historical frames of the vehicle and the moving object, and determining that the moving object is the vehicle by the matched moving frame; and S5, improving the detection recall rate by matching the upper moving frame. The recall rate of vehicle dynamic detection is improved through mobile detection, vehicle detection is more continuous and not prone to frame loss, vehicle dynamic detection does not need to be carried out when no moving object exists in a picture, and power consumption is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent video processing, and in particular relates to a method for improving the recall rate of vehicle dynamic inspection. Background Art

[0002] With the development of computer technology and the widespread application of computer vision principles, security systems are an important technical means to achieve security control. With the current expansion of security needs, their application in the field of security technology is becoming more and more extensive. However, the current security system mainly relies on people's visual judgment and lacks intelligent analysis of video content. This means that the security system can only complete the time of video storage and recording, and can only provide evidence for post-analysis. The lack of early warning also greatly reduces the significance of ensuring safety.

[0003] The intelligent security system is composed of monitoring, alarm, access control, patrol and broadcast (mainly for fire protection system), etc. The intelligent video analysis technology used in the security system is equivalent to an alarm detector, which can intelligently analyze the image recognition characteristics, monitor and alarm events in real time. At the same time, the alarm information it provides is also richer; therefore, it is most closely integrated with the monitoring system in the security system.

[0004] There are many existing methods to improve the recall rate of vehicle dynamic inspection, such as data: increase the number of data samples, improve data diversity; increase data enhancement methods; model: increase model size to improve detection recall rate; design a better model network to improve recall rate. Other aspects: apply a better loss function; find a better way to distribute positive and negative samples; better sample labeling methods, etc.

[0005] However, the defects of the prior art are:

[0006] There are many existing technologies for improving detection recall rate. Increasing data samples requires data collection and data labeling, which is time-consuming and laborious. Enlarging the model will increase performance, memory and running time. Designing a better and more clever network requires effort and time.

[0007] In addition, the commonly used terms in the prior art include:

[0008] Vehicle motion detection: monitor moving vehicles, that is, alarm when the vehicle moves.

[0009] Recall rate: the ratio of detected correct samples to all positive samples. Summary of the invention

[0010] In order to solve the above problems, the purpose of this application is to improve the accuracy of dynamic inspection of low-power fast-start vehicles.

[0011] Specifically, the present invention provides a method for improving the vehicle dynamic inspection recall rate, the method comprising the following steps:

[0012] S1. Train the vehicle detection model:

[0013] The vehicle detection model is based on the yolov5 framework and the network is yolov5s. The training adopts the open source coco dataset and the vehicle data collected by the user. The vehicle data collected by the user needs to be labeled. After the data is labeled, the vehicle detection model is obtained by training for 300 epochs. The detection time is 50ms.

[0014] S2. Add motion detection: use the binary difference map to obtain the rectangular frame of the moving target;

[0015] S3. Add the detection target history frame and moving object history frame:

[0016] The ID of the detection frame can be obtained by the tracking method, referred to as trackID. The tracking method is Kalman tracking. According to the trackID, it can be determined whether it is the same target; the targets with the same trackID are put into a vector. The vector size can be changed according to the situation. Here, it is assumed to be 5, that is, when the vector size = 5, when the trackID is the same, the vehicle's location information frame is put into the vector, so that the history frame of each vehicle and the history frame of all moving objects are obtained;

[0017] S4. Matching mobile frame and vehicle frame:

[0018] According to the historical frames of the vehicle and the moving object, the vehicle and the moving object are matched, and the matching moving frame can be regarded as the moving object as a vehicle;

[0019] S5. Use matching moving boxes to improve detection recall rate:

[0020] When there is a vehicle detection result, use the vehicle detection result. When the vehicle detection result fails, the matched moving box can be used as the vehicle result, which can improve the recall rate of vehicle detection.

[0021] In the step S1, the vehicle labeling can use labelImg labeling software to label the vehicle location information; the 300 epochs include coco and self-collected data, trained using the yolov5 framework, and the network is yolov5s.

[0022] The step S2 further comprises:

[0023] S2.1, motion detection uses the grayscale images of two frames to make a difference. A vector of size 4 is used to save 4 frames of grayscale images. The width and height of the image are 640x360. The fourth frame is subtracted from the first frame to form a difference image.

[0024] S2.2, binarize the difference image of step S2.1, i.e., the difference between the corresponding pixels of the fourth frame and the first frame. The threshold can be adjusted according to the demand. Assume that the threshold is set to 20, and the values ​​greater than 20 are set to 1, and the values ​​less than 20 are set to 0, so as to obtain a difference binary image;

[0025] S2.3, obtain the rectangular frame of the moving target on the difference binary image of step S2.2, that is, the place where the pixel is 1, first find the edge contour with pixel 1, and then obtain the minimum rectangular frame of the edge contour, so as to obtain the rectangular frame where the picture changes. There are as many rectangular frames as there are moving objects, that is, all moving objects; a threshold can be set to filter the rectangular frames. Suppose the threshold is set to 20, so that the frames less than 20 can be filtered out, that is, the width of the rectangular frame is W and the height is H. When H>20 and W>20, the rectangular frame is retained, otherwise it is filtered out; the moving time is 10ms.

[0026] In step S4, the matching method is specifically as follows:

[0027] S4.1, unify the vehicles and moving boxes on the image to make iou, the formula is iou = A∩B / A∪B, where A∩B is the intersection of the two boxes, and A∪B is the union of the two boxes; when iou>0.8, the moving box is considered to be a vehicle, otherwise it is not a vehicle;

[0028] S4.2, perform step S4.1 on the five historical frames of the vehicle and the moving object. The number of matches is N. When N is greater than or equal to 3, the moving object is considered to be a vehicle.

[0029] Therefore, the advantages of the present application are: the method is simple, motion detection is used to improve the recall rate of vehicle motion detection, making vehicle detection more continuous and less prone to frame loss, and there is no need to perform vehicle motion detection when there are no moving objects in the picture, thereby reducing power consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0031] Figure 1 is a schematic diagram of the process of this method. DETAILED DESCRIPTION

[0032] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail in conjunction with the accompanying drawings.

[0033] This application proposes a method for improving the recall rate of vehicle dynamic inspections, the core of which is a method for improving the recall rate of vehicle dynamic inspections by means of mobile detection. Figure 1 As shown, the main implementation steps are as follows:

[0034] Step S1. Training vehicle detection model

[0035] The vehicle detection model here is based on the yolov5 framework, and the network is yolov5s. The training uses the open source coco dataset and some vehicle data collected by yourself. Your own vehicle data needs to be labeled, and the labelImg labeling software can be used to label the vehicle location information. After the data is labeled, 300 epochs are trained, including coco and self-collected data, and the yolov5 framework is used for training. The network is yolov5s, and the vehicle detection model can be obtained. The detection time is about 50ms.

[0036] Step S2. Add motion detection

[0037] S2.1, motion detection uses the grayscale images of two frames to make a difference. A vector of size 4 is used to save 4 frames of grayscale images. The width and height of the image are 640x360. The fourth frame is subtracted from the first frame (the interval of two frames is because the difference between the two adjacent frames is small, and the interval frame number can also be set according to needs); a difference map is formed.

[0038] S2.2, binarize the difference image of step S2.1, that is, the difference between the corresponding pixels of the fourth frame and the first frame, the threshold is set to 20 (the threshold can be adjusted according to needs), >20 is set to 1, and less than 20 is set to 0, so as to obtain a difference binary image.

[0039] S2.3, obtain the rectangular frame of the moving target on the binary image of step S2.2, that is, where the pixel is 1, first find the edge contour with pixel 1, and then obtain the minimum rectangular frame of the edge contour, so as to obtain the number of rectangular frames where the picture changes as there are moving objects, that is, all moving objects. A threshold can be set to filter the rectangular frames. Assuming that the threshold is set to 20, frames smaller than 20 can be filtered out, that is, the width of the rectangular frame is W and the height is H. When H>20 and W>20, the rectangular frame is retained, otherwise it is filtered out, so that frames smaller than 20 can be filtered out. The moving time is about 10ms.

[0040] Step S3. Add the detection target history frame and the moving object history frame:

[0041] Tracking can be used to obtain the ID of the detection frame, referred to as trackID. The tracking method used here is Kalman tracking. According to trackID, it can be determined whether it is the same target. Put the targets with the same trackID into a vector, and set the vector size to 5 (which can be changed according to your needs). That is, when the vector size = 5, when the trackID is the same, put the vehicle's location information frame into the vector, so that the history frame of each vehicle and the history frame of all moving objects are obtained.

[0042] Step S4. Matching the moving frame and the vehicle frame

[0043] According to the historical frames of the vehicle and the moving object, the vehicle and the moving object are matched. The matched moving frame can be regarded as the moving object as a vehicle. The specific matching method is as follows:

[0044] S4.1, unify the vehicles and moving boxes on the image to make iou, the formula is iou = A∩B / A∪B, where A∩B is the intersection of the two boxes, and A∪B is the union of the two boxes. When iou>0.8, the moving box is considered to be a vehicle, otherwise it is not a vehicle;

[0045] S4.2, perform step S4.1 on the five historical frames of the vehicle and the mobile, and the number of matches is N. When N is greater than or equal to 3, the mobile object is considered to be a vehicle;

[0046] Step S5. Use the matching moving box to improve the detection recall rate:

[0047] When there is a vehicle detection result, use the vehicle detection result. When the vehicle detection result fails, the matched moving box can be used as the vehicle result, which can improve the recall rate of vehicle detection.

[0048] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for improving the vehicle dynamic inspection recall rate, characterized in that: The method comprises the following steps: S1. Train the vehicle detection model: The vehicle detection model is based on the yolov5 framework and the network is yolov5s. The training adopts the open source coco dataset and the vehicle data collected by the user. The vehicle data collected by the user needs to be labeled. After the data is labeled, the vehicle detection model is obtained by training for 300 epochs. The detection time is 50ms. S2. Add motion detection: use the binary difference map to obtain the rectangular frame of the moving target; S3. Add the detection target history frame and moving object history frame: The ID of the detection frame can be obtained by the tracking method, referred to as trackID. The tracking method is Kalman tracking. According to the trackID, it can be determined whether it is the same target; the targets with the same trackID are put into a vector. The vector size can be changed according to the situation. Here, it is assumed to be 5, that is, when the vector size = 5, when the trackID is the same, the vehicle's location information frame is put into the vector, so that the history frame of each vehicle and the history frame of all moving objects are obtained; S4. Matching mobile frame and vehicle frame: According to the historical frames of the vehicle and the moving object, the vehicle and the moving object are matched, and the matching moving frame can be regarded as the moving object as a vehicle; S5. Use matching moving boxes to improve detection recall rate: When there is a vehicle detection result, use the vehicle detection result. When the vehicle detection result fails, the matched moving box can be used as the vehicle result, which can improve the recall rate of vehicle detection.

2. The method for improving the vehicle dynamic inspection recall rate according to claim 1, characterized in that: In the step S1, the vehicle labeling can use labelImg labeling software to label the vehicle location information; the 300 epochs include coco and self-collected data, trained using the yolov5 framework, and the network is yolov5s.

3. The method for improving the vehicle dynamic inspection recall rate according to claim 1, characterized in that: The step S2 further comprises: S2.1, motion detection uses the grayscale images of two frames to make a difference. A vector of size 4 is used to save 4 frames of grayscale images. The width and height of the image are 640x360. The fourth frame is subtracted from the first frame to form a difference image. S2.2, binarize the difference image of step S2.1, i.e., the difference between the corresponding pixels of the fourth frame and the first frame. The threshold can be adjusted according to the demand. Assume that the threshold is set to 20, and the values ​​greater than 20 are set to 1, and the values ​​less than 20 are set to 0, so as to obtain a difference binary image; S2.3, obtain the rectangular frame of the moving target on the difference binary image of step S2.2, that is, the place where the pixel is 1, first find the edge contour with pixel 1, and then obtain the minimum rectangular frame of the edge contour, so as to obtain the rectangular frame where the picture changes. There are as many rectangular frames as there are moving objects, that is, all moving objects; a threshold can be set to filter the rectangular frames. Suppose the threshold is set to 20, so that the frames less than 20 can be filtered out, that is, the width of the rectangular frame is W and the height is H. When H>20 and W>20, the rectangular frame is retained, otherwise it is filtered out; the moving time is 10ms.

4. The method for improving the vehicle dynamic inspection recall rate according to claim 1, characterized in that: In step S4, the matching method is specifically as follows: S4.1, unify the vehicles and moving boxes on the image to make iou, the formula is iou = A∩B / A∪B, where A∩B is the intersection of the two boxes, and A∪B is the union of the two boxes; when iou>0.8, the moving box is considered to be a vehicle, otherwise it is not a vehicle; S4.2, perform step S4.1 on the five historical frames of the vehicle and the moving object. The number of matches is N. When N is greater than or equal to 3, the moving object is considered to be a vehicle.

Images