A target detection method based on improved YOLO

Abstract

The present invention discloses an improved YOLO-based target detection method, comprising: 1) reading in a video stream; 2) inputting an image into a YOLO v4 network; 3) performing target capture on the current frame to obtain the target type and calibration coordinates; 4) using the ViBe algorithm to extract background modeling of the first 15 frames in the video; 5) determining whether the current frame has a YOLO target framed area; 6) converting an RGB image into a grayscale image and performing ViBe target detection on the framed area; 7) segmenting the YOLO framed area targets to construct a YOBE database for target classification; 8) performing conventional ViBe detection on the unframed area and performing similarity matching. If a similarity threshold is met, performing YOLO type determination; 9) repeating the above steps until all image frames are detected. The present invention proposes a YOBE algorithm by combining the YOLO and Vibe algorithms. Compared with the single YOLO algorithm or Vibe algorithm, the YOBE algorithm can better detect small targets or low-pixel targets and effectively solve the shadow and ghosting problems in target extraction.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and specifically is a method which uses the YOLO (You Only Look Once) algorithm as the core and combines it with the ViBe (Visual Background Extractor) algorithm to complete the detection of small targets or low-pixel targets, while solving the shadow and ghosting problems in target extraction. Background Art

[0002] Object detection is the core of the YOLO algorithm, which uses a deep convolutional neural network (CNN) to achieve this. The input image is divided into a grid, and convolutions at a specific level are used to predict the presence, location, and type of objects in the image. Anchor boxes are used to handle objects of varying sizes and aspect ratios. YOLO's performance is limited for small objects and low-resolution images, reducing detection accuracy. Small object detection has important applications in many fields, such as drone monitoring, insect recognition, and medical image analysis. The efficiency and accuracy of the YOLO and ViBe algorithms make them powerful tools for detecting small objects in these practical applications. Summary of the Invention

[0003] The purpose of the present invention is to address the problem that small targets or low-pixel targets are difficult to detect in the conventional YOLO algorithm. The Vibe algorithm is combined with the YOLO algorithm to propose the YOBE algorithm. The YOLO algorithm is used to preliminarily frame the target, and ViBe detects the framed target. A YOBE image database is constructed to store target image information and category information. For small targets and low-pixel images, YOLO forced detection is performed using the YOBE database similarity matching method, and the shadow and ghosting problems in target extraction are solved at the same time.

[0004] To achieve the above objectives, the present invention adopts the following technical solutions.

[0005] An improved YOLO-based target detection method includes the following steps:

[0006] Step 1: Read the video stream to be detected frame by frame from the video library;

[0007] Step 2: Input each frame of the read image into the pre-trained YOLO v4 network model in sequence;

[0008] Step 3: The YOLO v4 network model captures the target in the current frame image, preliminarily frames the target area, and obtains the target type and calibration coordinates;

[0009] Step 4: Use the ViBe algorithm to extract the first 15 frames of images in the video stream as the initial background sample library;

[0010] Step 5: Determine whether the current frame has a YOLO target framed area. If so, expand the framed area by 1.2 times and proceed to step 6. Otherwise, proceed to step 8.

[0011] Step 6: Convert the original RGB image into a grayscale image and perform ViBe target detection on the framed area;

[0012] Step 7: If the target is detected in the framed area ViBe, segment the target in the YOLO framed area to build a YOBE database, and classify the target by obtaining the type;

[0013] Step 8: If there is no framed area in the current frame, ViBe conventional detection is performed on the entire image. If ViBe detects a target, image segmentation is performed to extract the target area, and the current target is matched with the YOBE database for similarity. If the similarity threshold is met, the target is re-entered into YOLO for detection and target classification;

[0014] Step 9: Determine whether the current frame is the last frame of the video stream. If so, end the detection; otherwise, return to step 2 to continue target detection for the next frame.

[0015] Specifically, the pre-trained YOLO v4 network model described in step 2 is trained using the COCO dataset to frame the target in the image. The bounding box contains the object's location information and the confidence score associated with each category. At the same time, the upper left corner coordinates, height, width and category of the target box are extracted. The location information includes the upper left corner x, y coordinate values ​​and width and height.

[0016] Specifically, the YOLO v4 network model described in step 3 captures the target in the current frame image, preliminarily frames the target area, and obtains the target type and calibration coordinates;

[0017] First, the current frame image is input into the convolutional neural network for feature extraction. Each cell generates multiple bounding boxes based on the prediction results, and Non-Maximum Suppression (NMS) is used to eliminate redundant bounding boxes.

[0018] Then, the input current frame image is divided into a fixed-size grid. Each cell is responsible for predicting a set of bounding boxes and the confidence of the corresponding category. Each bounding box contains the location information of the object and the confidence score associated with each category. The softmax function is used to calculate the probability that each bounding box belongs to a different category. The category prediction of each bounding box is calculated by the degree of overlap between the bounding box and the target instance and the category confidence score.

[0019] Finally, the detected object bounding box position, category and corresponding confidence score are output.

[0020] Preferably, in step 4, 15 frames of background modeling starting from the first frame of the video stream are used as the initial background sample library, which can optimize the shadow problem.

[0021] Specifically, steps 5, 6, and 7 detect the target using YOLO and input the detection coordinates and types into ViBe. By default, pixels outside the frame are considered background pixels, and only in-frame detection is performed to segment the image to obtain precise targets. The targets detected by both YOLO and ViBe are input into the YOBE database. YOLO in-frame detection can eliminate shadow problems and improve shadow elimination efficiency.

[0022] Specifically, the construction process of the YOBE database described in step 7 is as follows:

[0023] If ViBe detects an object in the framed area, the following operations are performed on the image within the frame using the ViBe algorithm:

[0024] Set the radius threshold coefficient R = 20, the matching threshold coefficient M = 2, and the difference coefficient Δλ, where represents the pixel value of the i-th row and j-th column of the μ-th frame, represents the pixel value of the i-th row and j-th column in the μ-th sample library; then

[0025]

[0026] Solve the formula (1) by comparing the image of the μth frame with all samples in the initial background sample library to obtain Δλ μ , μ=1~20, compare Δλ in turn μ The relationship with R is recorded, and the number of times R>Δλ is X. If X>M, it is considered that the pixel value being judged has not changed much from the previous pixel value and the distance is not far. That is, the point is defined as a background pixel and not a dynamic area. Otherwise, it means that the pixel point is a foreground pixel.

[0027] The common targets detected by the two algorithms are used to construct a YOBE database, and the common features and categories are sorted out to provide preliminary feature and category screening indexes for the subsequent classification of images segmented by ViBe alone.

[0028] Specifically, if there is no framed area in the current frame described in step 8, ViBe conventional detection is performed on the entire image. In the case of low-pixel images or small targets, YOLO cannot detect the target, and it is necessary to use ViBe's high sensitivity to grayscale pixel changes to perform image segmentation. The RGB image of the corresponding coordinates in the original image is extracted based on the pixel coordinates of the segmented image. The image extracted only by ViBe is matched with the image data in the YOBE database for similarity matching and feature matching. When the similarity threshold reaches 0.8 or above, the image is re-input into YOLO for secondary detection and classified according to the image category in the YOBE database, and a bounding box is given.

[0029] The beneficial effects of the present invention are:

[0030] The present invention combines Vibe with YOLO algorithm to propose YOBE algorithm, uses YOLO algorithm to preliminarily frame the target, and uses ViBe to detect the framed target. A YOBE image database is constructed to store target image information and category information. For small targets and low-pixel images, YOLO forced detection is performed using the YOLO database similarity matching method. This overcomes the problem that conventional YOLO algorithm has difficulty in detecting small targets or low-pixel targets, and solves the shadow and ghosting problems in target extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a flow chart of a target detection method based on improved YOLO of the present invention;

[0032] Figure 2 Detection results of the method of the present invention on small targets; (a) YOLO does not detect the small target, (b) ViBe targets, (c) YOLO combined with ViBe to detect the small target twice;

[0033] Figure 3 These are the effects of the method of the present invention on removing shadows from an image; (a) original image, (b) YOLO image, (c) ViBe image, and (d) YOLO combined with ViBe image. DETAILED DESCRIPTION

[0034] The present invention will be further described in detail below with reference to specific embodiments, but the protection scope of the present invention is not limited to the contents described above.

[0035] Example

[0036] like Figure 1 As shown, the present invention provides a target detection method based on improved YOLO, comprising the following steps:

[0037] Step 1: Read the video stream to be detected frame by frame from the video library. In this embodiment, the video material is captured by mobile phone photography, and the mobile phone model is iPhone 11promax;

[0038] Step 2: Input each frame of the read image into the pre-trained YOLO v4 network model in sequence;

[0039] The YOLO v4 network model is trained using the COCO dataset to bound objects within the image. The bounding box contains the object's location information and the confidence score associated with each category. The upper left corner coordinates, height, width, and category of the target box are also extracted. The location information includes the upper left corner x, y coordinates, width, and height.

[0040] Step 3: The YOLO v4 network model captures the target in the current frame image, preliminarily frames the target area, and obtains the target type and calibration coordinates;

[0041] First, the current frame image is input into the convolutional neural network for feature extraction. Each cell generates multiple bounding boxes based on the prediction results, and Non-Maximum Suppression (NMS) is used to eliminate redundant bounding boxes.

[0042] Then, the input current frame image is divided into a fixed-size grid. Each cell is responsible for predicting a set of bounding boxes and the confidence of the corresponding category. Each bounding box contains the location information of the object and the confidence score associated with each category. The softmax function is used to calculate the probability that each bounding box belongs to a different category. The category prediction of each bounding box is calculated by the degree of overlap between the bounding box and the target instance and the category confidence score.

[0043] Finally, the detected target bounding box position, category and corresponding confidence score are output;

[0044] Step 4: Use the ViBe algorithm to extract the first 15 frames of images in the video stream as the initial background sample library;

[0045] Step 5: Determine whether the current frame has a YOLO target framed area. If so, expand the framed area by 1.2 times and proceed to step 6. Otherwise, proceed to step 8.

[0046] Step 6: Convert the original RGB image into a grayscale image and perform ViBe target detection on the framed area;

[0047] Step 7: If ViBe detects an object in the framed area, the following operations are performed on the image within the frame using the ViBe algorithm:

[0048] Set the radius threshold coefficient R = 20, the matching threshold coefficient M = 2, and the difference coefficient Δλ, where represents the pixel value of the i-th row and j-th column of the μ-th frame, represents the pixel value of the i-th row and j-th column in the μ-th sample library; then

[0049]

[0050] Solve the formula (1) by comparing the image of the μth frame with all samples in the initial background sample library to obtain Δλ μ , μ=1~20, compare Δλ in turn μThe relationship with R is recorded, and the number of times R>Δλ is X. If X>M, it is considered that the pixel value being judged has not changed much from the previous pixel value and the distance is not far. That is, the point is defined as a background pixel and not a dynamic area. Otherwise, it means that the pixel point is a foreground pixel.

[0051] The common targets detected by the two algorithms are used to construct a YOBE database, in which the common features and categories are sorted out, providing an index for preliminary feature and category screening for subsequent image classification by ViBe segmentation.

[0052] Step 8. If there is no framed area in the current frame, perform ViBe regular detection on the entire image. If YOLO cannot detect the target in low-pixel images or small targets, it is necessary to use ViBe's high sensitivity to grayscale pixel changes to perform image segmentation. The segmented image is extracted based on the pixel coordinates to obtain the RGB image of the corresponding coordinates in the original image. The image extracted only by ViBe is matched with the image data in the YOBE database for similarity matching and feature matching. When the similarity threshold reaches 0.8 or above, the image is re-input into YOLO for secondary detection and classified according to the image category in the YOBE database, and a bounding box is given.

[0053] Step 9: Determine whether the current frame is the last frame of the video stream. If so, end the detection; otherwise, return to step 2 to continue target detection for the next frame.

[0054] like Figure 2 、 Figure 3 The figures show the effect of target detection on the video images collected in this embodiment using the method of the present invention. It can be clearly seen from the figures that the YOLO algorithm proposed by the present invention, which combines the YOLO and Vibe algorithms, can better detect small targets or low-pixel targets than the single YOLO algorithm or Vibe algorithm, and effectively solve the shadow and ghosting problems in target extraction.

[0055] The above description is only a preferred example of the present invention and does not limit the present invention in any way. Any simple modification, change, or imitation of the above implementation examples based on the technical content of the present invention shall fall within the scope of protection of the technical solution of the present invention.

Claims

1. A target detection method based on improved YOLO, characterized in that: The following steps are involved: Step 1: Read the video stream to be detected frame by frame from the video library; Step 2: Input each frame of the read image into the pre-trained YOLO v4 network model in sequence; Step 3: The YOLO v4 network model captures the target in the current frame image, preliminarily frames the target area, and obtains the target type and calibration coordinates; Step 4: Use the ViBe algorithm to extract the first 15 frames of images in the video stream as the initial background sample library; Step 5: Determine whether the current frame has a YOLO target framed area. If so, expand the framed area by 1.2 times and proceed to step 6. Otherwise, proceed to step 8. Step 6: Convert the original RGB image into a grayscale image and perform ViBe target detection on the framed area; Step 7: If the target is detected in the framed area ViBe, segment the target in the YOLO framed area to build a YOBE database, and classify the target by obtaining the type; Step 8: If there is no framed area in the current frame, ViBe conventional detection is performed on the entire image. If ViBe detects a target, image segmentation is performed to extract the target area, and the current target is matched with the YOBE database for similarity. If the similarity threshold is met, the target is re-entered into YOLO for detection and target classification; Step 9: Determine whether the current frame is the last frame of the video stream. If so, end the detection; otherwise, return to step 2 to continue target detection for the next frame.

2. A target detection method based on improved YOLO according to claim 1, characterized in that, The pre-trained YOLO v4 network model described in step 2 is trained using the COCO dataset to frame the target in the image. The bounding box contains the object's location information and the confidence score associated with each category. At the same time, the upper left corner coordinates, height, width, and category of the target box are extracted. The location information includes the upper left corner x, y coordinates, width, and height.

3. A target detection method based on improved YOLO according to claim 1, characterized in that, The YOLO v4 network model described in step 3 captures the target in the current frame image, preliminarily frames the target area, and obtains the target type and calibration coordinates; First, the current frame image is input into the convolutional neural network for feature extraction. Each cell generates multiple bounding boxes based on the prediction results, and Non-Maximum Suppression (NMS) is used to eliminate redundant bounding boxes. Then, the input current frame image is divided into a fixed-size grid. Each cell is responsible for predicting a set of bounding boxes and the confidence of the corresponding category. Each bounding box contains the location information of the object and the confidence score associated with each category. The softmax function is used to calculate the probability that each bounding box belongs to a different category. The category prediction of each bounding box is calculated by the degree of overlap between the bounding box and the target instance and the category confidence score. Finally, the detected object bounding box position, category and corresponding confidence score are output.

4. A target detection method based on improved YOLO according to claim 1, characterized in that, The construction process of the YOBE database described in step 7 is as follows: If ViBe detects an object in the framed area, the following operations are performed on the image within the frame using the ViBe algorithm: Set the radius threshold coefficient R = 20, the matching threshold coefficient M = 2, and the difference coefficient Δλ, where represents the pixel value of the pixel at the i-th row and j-th column of the μ-th frame, represents the pixel value of the pixel in the i-th row and j-th column of the μ-th sample library; then Solve the formula (1) by comparing the image of the μth frame with all samples in the initial background sample library to obtain Δλ μ , μ=1~20, compare Δλ in turn μ The relationship between the value of R and the value of R is recorded. The number of times R>Δλ is X. If X>M, it is considered that the pixel value being judged has not changed much from the previous pixel value and the distance is not far. That is, the pixel point is defined as a background pixel and is not a dynamic area. Otherwise, it means that the pixel point is a foreground pixel. The common targets detected by the two algorithms are used to construct a YOBE database, and the common features and categories are sorted out to provide preliminary feature and category screening indexes for the subsequent classification of images segmented by ViBe alone.

5. A target detection method based on improved YOLO according to claim 1, characterized in that, If there is no framed area in the current frame as described in step 8, then ViBe regular detection is performed on the entire image. In the case of low-pixel images or small targets, YOLO cannot detect the target. It is necessary to use ViBe's high sensitivity to grayscale pixel changes to perform image segmentation. The RGB image of the corresponding coordinates in the original image is extracted based on the pixel coordinates of the segmented image. The image extracted only by ViBe is matched with the image data in the YOBE database for similarity matching and feature matching. When the similarity threshold reaches above 0.8, the image is re-input into YOLO for secondary detection and classified according to the image category in the YOBE database, and a bounding box is given.

Images