Image recognition method and system based on multi-target detection, and storage medium

Abstract

This invention discloses an image recognition method, system, and storage medium based on multi-target detection, relating to the field of image visual recognition technology. It addresses the problem that current vehicle detection methods cannot detect vehicles obscured from behind, thus failing to fully acquire information about vehicles behind. The method involves: constructing a detection image of the vehicle to be detected; calculating the vehicle height based on the detection image; determining whether a following vehicle exists within the detection area behind the vehicle in the detection image; segmenting the vehicle border of the following vehicle in the following vehicle detection image; determining whether the vehicle to be detected is obscuring the following vehicle based on the recognition results of the front border area and the windshield border area; and determining whether the following vehicle has passed the previous road camera based on real-time vehicle video of all vehicles, and obtaining the license plate number of the following vehicle based on the determination result. This invention achieves accurate identification of information about vehicles behind that are obscured by the vehicle in front.

Description

Technical Field

[0001] This invention belongs to the field of image visual recognition technology, specifically an image recognition method, system, and storage medium based on multi-target detection. Background Technology

[0002] Vehicle detection refers to the technical process of automatically identifying and locating the position and boundaries of vehicles in images or videos. By analyzing visual information in images, such as color, shape, texture, or depth features, detection algorithms can find the rectangular region or outline where a vehicle is located in each frame, thereby determining the presence and location of the vehicle. Vehicle detection is often used as the foundation for applications such as traffic monitoring, autonomous driving, and road congestion analysis, providing input data for subsequent tasks such as speed measurement, license plate recognition, or tracking.

[0003] In existing technologies, vehicle detection methods typically rely on the complete visible area of ​​a vehicle for judgment. When the vehicle in front is a high-rise truck or a large vehicle, the vehicle behind may be completely obscured, and the camera cannot directly capture the image of the vehicle behind, resulting in a blind spot for following vehicles. At the same time, due to the blind spot, the license plate and driver's facial information of the vehicle behind often cannot be captured by the lane camera, resulting in the inability to obtain complete vehicle information of the vehicle behind.

[0004] Therefore, this invention proposes an image recognition method, system, and storage medium based on multi-target detection. Summary of the Invention

[0005] The purpose of this invention is to propose an image recognition method, system, and storage medium based on multi-target detection to solve the problems mentioned in the background art.

[0006] The technical problem to be solved by this invention is:

[0007] How to determine the following vehicle and obtain its information when the vehicle in front is obstructing the view of the vehicle in front.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] Image recognition methods based on multi-object detection, including:

[0010] Step S1: Construct an image of the vehicle to be detected based on real-time vehicle video, and then calculate the vehicle height based on the image.

[0011] Step S2: Determine whether there is a following vehicle in the rear detection area of ​​the image to be detected by the total number of suspected vehicle pixels and the number of pixels on the roof.

[0012] Step S3: Segment the vehicle border of the following vehicle in the following vehicle detection image, and determine whether the vehicle to be detected occludes the following vehicle based on the recognition results of the front border area and the windshield border area.

[0013] Step S4: Based on the real-time vehicle video of all vehicles, determine whether the following vehicle has passed the previous road camera, and obtain the license plate number of the following vehicle based on the determination result.

[0014] Further, step S1 includes the following sub-steps:

[0015] Step S11: Set up a laser vehicle detector within the shooting area of ​​the road camera. When the vehicle to be detected in the road triggers the laser vehicle detector, the road camera captures real-time vehicle video of the vehicle to be detected.

[0016] Step S12: Obtain the output frame rate corresponding to the road camera;

[0017] If the output frame rate is greater than or equal to the upper limit of the frame rate, a fixed number of images will be randomly selected as video frame images of the road within a fixed time interval.

[0018] If the output frame rate is less than the upper limit of the frame rate, the output frame rate is multiplied by a fixed duration, and the calculated result is used as the video frame image of the road.

[0019] Step S13: Draw a perpendicular line from half the length of the video frame image to the long side of the video frame, and draw a perpendicular line from half the width of the video frame image to the wide side of the video frame. Record the pixel point corresponding to the intersection of the two perpendicular lines as the center pixel point of the video frame image.

[0020] Step S14: When there are no vehicles driving on the road, the road camera takes a picture of the road. Step S13 is repeated to obtain the center pixel of the road image, and an initial marker point that coincides with the center pixel is set in the road.

[0021] Furthermore, step S1 also includes the following sub-steps:

[0022] Step S15: Starting from the initial marker point, set positioning marker points at fixed intervals and number all positioning marker points.

[0023] Step S16: Identify the vehicle bounding box of the vehicle to be detected in all video frame images, and connect the pixels with the maximum vertical coordinate in the vehicle bounding box to obtain the roof pixel sequence of the vehicle to be detected in the video frame images. The video frame image corresponding to the intersection of the center pixel and the roof pixel sequence is taken as the detection image of the vehicle to be detected.

[0024] Step S17: When the center pixel intersects with the roof pixel sequence, obtain the positioning marker point with the smallest distance to the rear of the vehicle, and then obtain the distance between the positioning marker point and the initial marker point, and record it as the actual displacement distance of the vehicle to be detected.

[0025] Step S18: Obtain the horizontal distance between the road camera and the initial marker point and the vertical height of the road camera, and calculate the vehicle height of the vehicle to be detected;

[0026] Step S19: When the vehicle height is greater than or equal to the height threshold, proceed to step S2;

[0027] No action is taken when the vehicle height is below the height threshold.

[0028] Further, step S2 includes the following sub-steps:

[0029] Step S21: Mark the vehicle border in the image to be detected, and extend it by a fixed pixel length in the positive Y-axis direction with the vehicle roof pixel sequence as the baseline. Use the extended area as the rear detection area of ​​the image to be detected.

[0030] Step S22: Obtain the pixel values ​​of all pixels in the image to be detected, then convert the image to be detected into a grayscale image to be detected and obtain the pixel grayscale values ​​of all pixels in the detection area behind the grayscale image to be detected.

[0031] Step S23: Convert the road image captured by the road camera when there are no vehicles on the road into a grayscale road image, obtain the grayscale value of all pixels in the grayscale road image, sum the grayscale values ​​and take the average value to calculate the average grayscale value of the grayscale road image.

[0032] Step S24: Subtract the average gray value from the gray values ​​of all pixels in the rear detection area and take the absolute value to calculate the gray difference of all pixels.

[0033] Furthermore, step S2 also includes the following sub-steps:

[0034] Step S25: If the grayscale difference is greater than the minimum grayscale difference, then the corresponding pixel in the rear detection area is determined to be a suspected vehicle pixel, and the total number of suspected vehicle pixels is counted, and then proceed to step S26.

[0035] If the grayscale difference is less than or equal to the minimum grayscale difference, it is determined that there is no following vehicle in the rear detection area;

[0036] Step S26: Obtain the real-time speed of the vehicle to be detected and the speed range of the vehicle in the current road; wherein, the speed range includes a first speed range, a second speed range and a third speed range.

[0037] When the real-time vehicle speed falls within the first speed range, the first image following the image to be detected is selected as the following vehicle detection image, based on the chronological order of the real-time vehicle video. When the real-time vehicle speed falls within the second speed range, the third image following the image to be detected is selected as the following vehicle detection image, based on the chronological order of the real-time vehicle video. When the real-time vehicle speed falls within the third speed range, the fifth image following the image to be detected is selected as the following vehicle detection image, based on the chronological order of the real-time vehicle video.

[0038] Step S27, repeat step S16 to obtain the number of roof pixel sequences and the corresponding number of pixels in the rear detection area of ​​the following vehicle detection image;

[0039] Step S28: If the number of roof pixel sequences in the vehicle detection image is one, and the total number of suspected vehicle pixels is less than the number threshold, then it is determined that there is no following vehicle in the rear detection area; if the number of roof pixel sequences in the vehicle detection image is greater than or equal to two, or the total number of suspected vehicle pixels is greater than or equal to the number threshold, then it is determined that there is a following vehicle in the rear detection area, and proceed to step S3.

[0040] Furthermore, step S3 includes the following sub-steps:

[0041] Step S31: Repeat step S16 to identify the vehicle border of the following vehicle in the following vehicle detection image; if there is a front border area and a windshield border area in the vehicle border of the following vehicle, then the front border area and the windshield border area are cropped; if there is no front border area or windshield border area in the vehicle border of the following vehicle, then it is determined that the vehicle to be detected is occluding the following vehicle, and proceed to step S4.

[0042] Step S32: Obtain the color corresponding to the license plate type for all license plates, and convert the color of all license plates to the standard license plate grayscale value;

[0043] Step S33: Convert the front border region into a grayscale image, and then extract all rectangular regions of the front border region; if there are two grayscale values ​​in any rectangular region, proceed to step S34; if there are more than two grayscale values ​​in all rectangular regions, it is determined that the vehicle to be detected is occluding the following vehicle, and proceed to step S4.

[0044] Step S34: The rectangular area of ​​the following vehicle is identified using a license plate recognition algorithm; when the rectangular area is the license plate of the following vehicle, the license plate number of the following vehicle is identified; when the rectangular area is not the license plate of the following vehicle, no operation is performed.

[0045] Further, step S4 includes the following sub-steps:

[0046] Step S41: Obtain the outline of the upper border area corresponding to the following vehicle, and convert the pixel values ​​of all pixels in the upper border area to grayscale values.

[0047] Step S42: Obtain the real-time vehicle video corresponding to all vehicles captured by the previous road camera, and repeat step S16 to obtain the upper border area of ​​all vehicles in the real-time vehicle video. At the same time, obtain the region outline corresponding to the upper border area of ​​all vehicles and the grayscale value of all pixels.

[0048] Step S43: Compare the outline of the upper border area of ​​all vehicles with the outline of the upper border area of ​​the following vehicles.

[0049] If the outline of the upper border area of ​​any vehicle completely overlaps with the outline of the upper border area of ​​the following vehicle, proceed to step S44.

[0050] If the outline of the upper border area of ​​all vehicles does not coincide with the outline of the upper border area of ​​the following vehicle, no operation will be performed.

[0051] Step S44: Number the pixels in the upper border area of ​​all vehicles. Similarly, number the pixels in the upper border area of ​​the following vehicles. Subtract the gray values ​​of the pixels with the same number and take the absolute value to calculate the pixel gray value difference between all pixels. Count the number of pixels whose pixel gray value difference is greater than the difference threshold.

[0052] If the number of pixels exceeds the threshold, no operation is performed.

[0053] If the number of pixels is less than or equal to the number threshold, the corresponding vehicle is determined to be a suspected following vehicle, and the process proceeds to step S45.

[0054] Step S45: Obtain the interval distance between the camera on the previous road and the camera on the current road, the lower limit of the current road interval, the first time point when the camera on the previous road captures the suspected following vehicle, and the second time point when the camera on the current road captures the suspected following vehicle.

[0055] Furthermore, step S4 also includes the following sub-steps:

[0056] Step S46: Divide the interval distance by the upper limit of the current road interval to calculate the minimum travel time for the vehicle to travel from the position of the previous road camera to the position of the current road camera. Similarly, divide the interval distance by the lower limit of the current road interval to calculate the maximum travel time for the vehicle to travel from the position of the previous road camera to the position of the current road camera.

[0057] Step S47: Add the minimum travel time to the first time node and use the calculated time node as the minimum endpoint value of the arrival time interval. Similarly, add the maximum travel time to the first time node and use the calculated time node as the maximum endpoint value of the arrival time interval. Construct the arrival time interval of the suspected following vehicle based on the maximum endpoint value and the minimum endpoint value.

[0058] Step S48: When the second time node belongs to the arrival time interval, determine that the suspected following vehicle is the following vehicle, and proceed to step S49.

[0059] When the second time point does not fall within the arrival time interval, the suspected following vehicle is determined not to be a following vehicle.

[0060] Step S49: Convert the real-time vehicle video corresponding to the following vehicle captured by the previous road camera into video frame images, and repeat steps S31 to S34 to obtain the license plate number of the following vehicle.

[0061] The present invention also provides an image recognition system based on multi-target detection, comprising:

[0062] The data acquisition module is used to acquire real-time vehicle videos from the current road camera and the previous road camera in the road.

[0063] The video segmentation module is used to segment real-time vehicle video into video frame images;

[0064] The height detection module is used to determine whether the height of the vehicle to be detected is greater than a height threshold.

[0065] The image segmentation module is used to segment the rear detection region in the image to be detected;

[0066] The vehicle detection module is used to determine whether there are following vehicles in the rear detection area;

[0067] The information investigation module is used to obtain the license plate number of the following vehicle based on the border area;

[0068] The occlusion detection module is used to determine whether the vehicle under test occludes the following vehicle;

[0069] The vehicle detection module is used to determine whether a suspected following vehicle is a true following vehicle based on upstream and downstream road cameras;

[0070] The database module stores the license plate type and corresponding color for all license plates, the current speed range of the road, the distance between adjacent road cameras, and the vertical height of the road cameras.

[0071] The present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the image recognition method described above.

[0072] Compared with the prior art, the beneficial effects of the present invention are:

[0073] 1. This invention constructs an image of the vehicle to be detected based on real-time vehicle video, and then calculates the vehicle height based on the image. By using the total number of suspected vehicle pixels and the number of pixels on the roof, it determines whether there is a following vehicle in the rear detection area of ​​the image, thereby achieving accurate determination of the following situation.

[0074] 2. This invention segments the vehicle border of the following vehicle in the following vehicle detection image, determines whether the vehicle to be detected is obscuring the following vehicle based on the recognition results of the front border area and the windshield border area, determines whether the following vehicle has passed the previous road camera based on the real-time vehicle video of all vehicles, and obtains the license plate number of the following vehicle based on the determination result, thereby achieving accurate identification of the information of the obscured vehicle behind. Attached Figure Description

[0075] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0076] Figure 1 This is a flowchart of the method of the present invention;

[0077] Figure 2 This is an example diagram of the vehicle frame in this invention;

[0078] Figure 3 This is an example diagram showing the vertical height of the road camera and the vehicle height in this invention;

[0079] Figure 4 This is an overall system block diagram of the present invention;

[0080] Figure 5 This is a schematic diagram of the electronic device in this invention. Detailed Implementation

[0081] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0082] Example 1: Please refer to Figures 1-3As shown, the technical solution provided by this invention is: an image recognition method based on multi-target detection. This method is used to detect vehicles violating traffic rules on the road. Since the vehicle to be detected obstructs the following vehicle, causing the following vehicle to violate traffic rules, it is necessary to obtain the license plate number of the following vehicle. The method detects the height of the vehicle to be detected as captured by a road camera to determine whether the vehicle height is greater than a height threshold. Based on the determination result, it is determined whether there is a following vehicle behind the vehicle to be detected, and whether the license plate number of the following vehicle can be obtained from the real-time vehicle video captured by the road camera. If the license plate number of the following vehicle is detected, no operation is performed. If the license plate number of the following vehicle cannot be detected, the vehicle features of the following vehicle are extracted to extract the corresponding license plate number of the following vehicle.

[0083] In this embodiment, the image recognition method is specifically as follows:

[0084] Step S1: Construct an image of the vehicle to be detected based on real-time vehicle video, and then calculate the vehicle height based on the image.

[0085] Among them, real-time vehicle video can be collected by road cameras deployed directly above the road, and the height of the road cameras is greater than the maximum height limit of the current lane;

[0086] In this embodiment, step S1 includes the following sub-steps:

[0087] Step S11: Set up a laser vehicle detector within the shooting area of ​​the road camera. When the vehicle to be detected in the road triggers the laser vehicle detector, the road camera captures real-time vehicle video of the vehicle to be detected.

[0088] It should be noted that when the vehicle to be detected first enters the shooting area of ​​the road camera, the laser vehicle detector is triggered and the road camera starts to shoot real-time vehicle video; when the vehicle to be detected leaves the shooting area of ​​the road camera, the road camera stops shooting real-time vehicle video.

[0089] Step S12: Obtain the output frame rate corresponding to the road camera;

[0090] If the output frame rate is greater than or equal to the upper limit of the frame rate, a fixed number of images will be randomly selected as video frame images of the road within a fixed time interval.

[0091] If the output frame rate is less than the upper limit of the frame rate, the output frame rate is multiplied by a fixed duration, and the calculated result is used as the video frame image of the road.

[0092] In practice, the output frame rate is the number of images captured by the road camera per second; the maximum endpoint value minus the minimum endpoint value of the fixed duration interval is denoted as the fixed duration, which is one second; in actual operation, the fixed number can be five video frames.

[0093] Step S13: Draw a perpendicular line from half the length of the video frame image to the long side of the video frame, and draw a perpendicular line from half the width of the video frame image to the wide side of the video frame. Record the pixel point corresponding to the intersection of the two perpendicular lines as the center pixel point of the video frame image.

[0094] Step S14: When there are no vehicles driving on the road, the road camera takes a road image, repeats step S13, obtains the center pixel of the road image, and sets an initial marker point in the road that coincides with the center pixel.

[0095] Step S15: Starting from the initial marker point, set positioning marker points at fixed intervals and number all positioning marker points.

[0096] The fixed length can be 50 centimeters;

[0097] Step S16, as follows Figure 2 As shown, the vehicle bounding box of the vehicle to be detected is identified in all video frame images, and the pixels with the maximum vertical coordinate in the vehicle bounding box are connected to obtain the roof pixel sequence of the vehicle to be detected in the video frame image. The video frame image corresponding to the intersection of the center pixel and the roof pixel sequence is taken as the detection image of the vehicle to be detected.

[0098] The vehicle bounding box includes the top bounding box, front bounding box, and windshield bounding box of the vehicle to be detected. In practice, vehicle bounding boxes in video frames can be identified using object detection algorithms. Then, the corresponding bounding box pixels are marked, and the closed area obtained by connecting these pixels is used as the region of the vehicle to be detected in the video frame image, which is then cropped to obtain the vehicle image. The region contained in the bounding box of the vehicle to be detected is the same as the region contained in the bounding box of the following vehicles. Identifying vehicle bounding boxes in video frames using object detection algorithms is existing technology; Faster R-CNN or YOLO object detection algorithms can be used for this purpose, and will not be elaborated upon here. If there are following vehicles, only the bounding box of the vehicle to be detected is identified.

[0099] Step S17: When the center pixel point intersects with the roof pixel sequence, obtain the positioning marker point with the smallest distance to the rear of the vehicle, and then obtain the distance between the positioning marker point and the initial marker point, and record it as the actual displacement distance WY of the vehicle to be detected.

[0100] Step S18, as follows Figure 3 As shown, the horizontal distance SP between the road camera and the initial marker point and the vertical height CZ of the road camera are obtained. The vehicle height CG of the vehicle to be detected is then calculated using the following formula:

[0101] CG = WY × CZ / SP;

[0102] Step S19: When the vehicle height is greater than or equal to the height threshold, proceed to step S2;

[0103] No action is taken when the vehicle height is below the height threshold;

[0104] It should be noted that when the vehicle height is greater than or equal to the height threshold, the road camera cannot capture real-time video of the vehicle behind it that is adjacent to the vehicle being detected; the vehicle height is less than the maximum height limit of the current lane.

[0105] Step S2: Determine whether there is a following vehicle in the rear detection area of ​​the image to be detected by the total number of suspected vehicle pixels and the number of pixels on the roof.

[0106] In this embodiment, step S2 includes the following sub-steps:

[0107] Step S21: Mark the vehicle border in the image to be detected, and extend it by a fixed pixel length in the positive Y-axis direction with the vehicle roof pixel sequence as the baseline. Use the extended area as the rear detection area of ​​the image to be detected.

[0108] Step S22: Obtain the pixel values ​​of all pixels in the image to be detected, then convert the image to be detected into a grayscale image to be detected and obtain the pixel grayscale values ​​of all pixels in the detection area behind the grayscale image to be detected.

[0109] The image to be detected can be converted into a grayscale image by a grayscale conversion formula. The grayscale conversion formula is as follows: grayscale value = 0.3×R + 0.59×G + 0.11×B, where R is the red channel value of the pixel, G is the green channel value of the pixel, and B is the blue channel value of the pixel.

[0110] Step S23: Convert the road image captured by the road camera when there are no vehicles on the road into a grayscale road image, obtain the grayscale value of all pixels in the grayscale road image, sum the grayscale values ​​and take the average value to calculate the average grayscale value of the grayscale road image.

[0111] Specifically, the grayscale values ​​of both the following vehicle and the vehicle under test are different from the average grayscale value of the road.

[0112] Step S24: Subtract the average gray value from the gray value of all pixels in the rear detection area and take the absolute value to calculate the gray value difference of all pixels.

[0113] Step S25: If the grayscale difference is greater than the minimum grayscale difference, then the corresponding pixel in the rear detection area is determined to be a suspected vehicle pixel, and the total number of suspected vehicle pixels is counted, and then proceed to step S26.

[0114] If the grayscale difference is less than or equal to the minimum grayscale difference, it is determined that there is no following vehicle in the rear detection area;

[0115] It should be specifically noted that when the road camera captures real-time video of the vehicle to be detected, if there is a vehicle following behind the vehicle to be detected, the vehicle to be detected cannot completely block the following vehicle. Therefore, when there is a following vehicle, there will be a color change in the detection area behind the vehicle to be detected.

[0116] Step S26: Obtain the real-time speed of the vehicle to be detected and the speed range of the vehicle in the current road. The speed range includes a first speed range, a second speed range and a third speed range.

[0117] When the real-time vehicle speed is within the first speed range, the selection order is based on the time sequence of the real-time vehicle video, and the first image after the image to be detected is selected as the following vehicle detection image.

[0118] When the real-time vehicle speed falls within the second speed range, the selection order is based on the time sequence of the real-time vehicle video, and the third image after the image to be detected is selected as the image to follow the vehicle for detection.

[0119] When the real-time vehicle speed is in the third speed range, the selection order is based on the time sequence of the real-time vehicle video, and the fifth image after the image to be detected is selected as the following vehicle detection image.

[0120] The real-time speed of the vehicle to be detected can be detected by speed detectors deployed on both sides of the road. For example, the value range of the first speed range is (0, 20), the value range of the second speed range is (20, 40), and the value range of the third speed range is (40, 60). If the number of images after the image to be detected is less than five, the last image is selected as the image to be detected for the following vehicle.

[0121] Step S27, repeat step S16 to obtain the number of roof pixel sequences and the corresponding number of pixels in the rear detection area of ​​the following vehicle detection image;

[0122] Specifically, in step S27, if multiple vehicles exist simultaneously, the number of pixels corresponding to the roof pixel sequence of all vehicles in the following vehicle detection image is detected.

[0123] Step S28: If the number of pixel sequences on the roof of the vehicle in the vehicle detection image is one, and the total number of suspected vehicle pixels is less than the number threshold, then it is determined that there is no following vehicle in the rear detection area.

[0124] If the number of pixel sequences on the roof of the vehicle in the vehicle detection image is greater than or equal to two, or the total number of suspected vehicle pixels is greater than or equal to the number threshold, then it is determined that there is a following vehicle in the rear detection area, and the process proceeds to step S3.

[0125] Step S3: Segment the vehicle border of the following vehicle in the following vehicle detection image, and determine whether the vehicle to be detected occludes the following vehicle based on the recognition results of the front border area and the windshield border area.

[0126] In this embodiment, step S3 includes the following sub-steps:

[0127] Step S31, repeat step S16, and identify the vehicle bounding box of the following vehicle in the following vehicle detection image;

[0128] If the vehicle frame of the following vehicle contains a front frame area and a windshield frame area, then the front frame area and the windshield frame area are cropped.

[0129] If there is no front frame area or windshield frame area in the vehicle frame of the following vehicle, it is determined that the vehicle to be detected is obstructing the following vehicle, and proceed to step S4.

[0130] The front bezel area may contain the license plate of the following vehicle; the windshield bezel area may contain the driver's head.

[0131] Step S32: Obtain the color corresponding to the license plate type for all license plates, and convert the color of all license plates to the standard license plate grayscale value; where all license plates referred to here are the license plates corresponding to all vehicles traveling on the road;

[0132] Specifically, license plate types include blue background with white characters, yellow background with black characters, green background with white characters, white background with black characters, and black background with white characters, etc., which will not be listed one by one here; the colors corresponding to different license plate types can be obtained from the database; since all license plates have two colors, the standard license plate grayscale value includes the standard text grayscale value and the standard background grayscale value.

[0133] Step S33: Convert the front border region to a grayscale image, and then extract all rectangular regions of the front border region.

[0134] If two grayscale values ​​exist within any rectangular area, proceed to step S34;

[0135] If the number of gray values ​​in all rectangular areas is not two, it is determined that the vehicle to be detected is occluding the following vehicle, and proceed to step S4;

[0136] Step S34: Identify the rectangular area of ​​the following vehicle using a license plate recognition algorithm;

[0137] When the rectangular area represents the license plate of the following vehicle, the license plate number of the following vehicle is identified.

[0138] No action is taken when the rectangular area is not the license plate of the following vehicle.

[0139] Step S4: Based on the real-time vehicle video of all vehicles, determine whether the following vehicle has passed the camera on the previous road, and obtain the license plate number of the following vehicle based on the determination result.

[0140] In this embodiment, step S4 includes the following sub-steps:

[0141] Step S41: Obtain the outline of the upper border area corresponding to the following vehicle, and convert the pixel values ​​of all pixels in the upper border area to grayscale values.

[0142] Step S42: Obtain the real-time vehicle video corresponding to all vehicles captured by the previous road camera, and repeat step S16 to obtain the upper border area of ​​all vehicles in the real-time vehicle video. At the same time, obtain the region outline corresponding to the upper border area of ​​all vehicles and the grayscale value of all pixels.

[0143] It should be noted that the method of identifying following vehicles by the camera on the next road of the current road is the same as the method of identifying following vehicles by the camera on the previous road of the current road, and will not be described in detail in this embodiment.

[0144] Step S43: Compare the outline of the upper border area of ​​all vehicles with the outline of the upper border area of ​​the following vehicles.

[0145] If the outline of the upper border area of ​​any vehicle completely overlaps with the outline of the upper border area of ​​the following vehicle, proceed to step S44.

[0146] If the outline of the upper border area of ​​all vehicles does not coincide with the outline of the upper border area of ​​the following vehicle, no operation will be performed.

[0147] Step S44: Number the pixels in the upper border area of ​​all vehicles. Similarly, number the pixels in the upper border area of ​​the following vehicles. Subtract the gray values ​​of the pixels with the same number and take the absolute value to calculate the pixel gray value difference between all pixels. Count the number of pixels whose pixel gray value difference is greater than the difference threshold.

[0148] If the number of pixels exceeds the threshold, no operation is performed.

[0149] If the number of pixels is less than or equal to the number threshold, the corresponding vehicle is determined to be a suspected following vehicle, and the process proceeds to step S45.

[0150] Step S45: Obtain the interval distance between the camera on the previous road and the camera on the current road, the lower limit and upper limit of the speed range corresponding to the current road, the first time point when the camera on the previous road captures the suspected following vehicle, and the second time point when the camera on the current road captures the suspected following vehicle.

[0151] Step S46: Divide the interval distance by the upper limit of the current road interval to calculate the minimum travel time for the vehicle to travel from the position of the previous road camera to the position of the current road camera. Similarly, divide the interval distance by the lower limit of the current road interval to calculate the maximum travel time for the vehicle to travel from the position of the previous road camera to the position of the current road camera.

[0152] Step S47: Add the minimum travel time to the first time node and use the calculated time node as the minimum endpoint value of the arrival time interval. Similarly, add the maximum travel time to the first time node and use the calculated time node as the maximum endpoint value of the arrival time interval. Construct the arrival time interval of the suspected following vehicle based on the maximum endpoint value and the minimum endpoint value.

[0153] Step S48: When the second time node belongs to the arrival time interval, determine that the suspected following vehicle is the following vehicle, and proceed to step S49.

[0154] When the second time point does not fall within the arrival time interval, the suspected following vehicle is determined not to be a following vehicle.

[0155] Step S49: Convert the real-time vehicle video corresponding to the following vehicle captured by the previous road camera into video frame images, and repeat steps S31 to S34 to obtain the license plate number of the following vehicle.

[0156] Example 2: As Figure 4 As shown, based on another concept of the same invention, an image recognition system based on multi-target detection is proposed, including a data acquisition module, a video segmentation module, a height detection module, an image division module, a vehicle determination module, an information investigation module, an occlusion determination module, a vehicle detection module, and a database module;

[0157] The data acquisition module is used to acquire real-time vehicle videos from the current road camera and the previous road camera in the road.

[0158] The video segmentation module is used to segment real-time vehicle video into video frame images.

[0159] The height detection module is used to determine whether the height of the vehicle to be detected is greater than a height threshold.

[0160] The image segmentation module is used to segment the rear detection region in the image to be detected;

[0161] The vehicle determination module is used to determine whether there are following vehicles in the rear detection area;

[0162] The information investigation module is used to obtain the license plate number of the following vehicle based on the border area;

[0163] The occlusion determination module is used to determine whether the vehicle to be detected is obscuring the following vehicle.

[0164] The vehicle detection module is used to determine whether a suspected following vehicle is a following vehicle based on upstream and downstream road cameras.

[0165] The database module is used to store the license plate type and corresponding color of all license plates, the current speed range of the road, the distance between adjacent road cameras, and the vertical height of the road cameras.

[0166] Example 3: This embodiment of the invention also provides an electronic device for running the aforementioned image recognition method based on multi-target detection; see [link to previous example]. Figure 5 The diagram shown is a structural schematic of an electronic device provided by an embodiment of the present invention. The electronic device includes a memory and a processor. The memory is used to store one or more computer instructions, which are executed by the processor to implement the above-mentioned image recognition method based on multi-target detection.

[0167] Furthermore, Figure 5 The electronic device shown also includes a system bus and a communication interface, with the processor, communication interface, and memory connected via the communication bus;

[0168] The memory may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc. The system bus can be an ISA bus, PCI bus, or EISA bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5The symbol is represented by only one bidirectional arrow, but this does not mean that there is only one communication bus or one type of system bus.

[0169] The processor may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above methods can be completed by integrated logic circuits in the processor's hardware or by software instructions. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the method described in the foregoing embodiments.

[0170] Example 4: This embodiment of the invention also provides a computer storage medium storing computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions cause the processor to implement the above-described image recognition method based on multi-target detection. For specific implementation, please refer to the method embodiment, which will not be repeated here.

[0171] The computer program product of the image recognition method based on multi-target detection provided in the embodiments of the present invention includes a computer storage medium storing program code. The instructions included in the program code can be used to execute the methods in the preceding method embodiments. For specific implementation, please refer to the method embodiments, which will not be repeated here.

[0172] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system and / or device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0173] Furthermore, in the description of the embodiments of the present invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances.

[0174] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0175] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image recognition method based on multi-target detection, characterized in that, The methods include: Step S1: Construct an image of the vehicle to be detected based on real-time vehicle video, and then calculate the vehicle height based on the image. Step S2: Determine whether there is a following vehicle in the rear detection area of ​​the image to be detected by the total number of suspected vehicle pixels and the number of pixels on the roof. Step S3: Segment the vehicle border of the following vehicle in the following vehicle detection image, and determine whether the vehicle to be detected occludes the following vehicle based on the recognition results of the front border area and the windshield border area. Step S4: Based on the real-time vehicle video of all vehicles, determine whether the following vehicle has passed the camera on the previous road, and obtain the license plate number of the following vehicle based on the determination result. Step S4 includes the following sub-steps: Step S41: Obtain the outline of the upper border area corresponding to the following vehicle, and convert the pixel values ​​of all pixels in the upper border area to grayscale values. Step S42: Obtain the real-time vehicle video corresponding to all vehicles captured by the previous road camera, and repeat step S16 to obtain the upper border area of ​​all vehicles in the real-time vehicle video. At the same time, obtain the region outline corresponding to the upper border area of ​​all vehicles and the grayscale value of all pixels. Step S43: Compare the outline of the upper border area of ​​all vehicles with the outline of the upper border area of ​​the following vehicles. If the outline of the upper border area of ​​any vehicle completely overlaps with the outline of the upper border area of ​​the following vehicle, proceed to step S44. If the outline of the upper border area of ​​all vehicles does not coincide with the outline of the upper border area of ​​the following vehicle, no operation will be performed. Step S44: Number the pixels in the upper border area of ​​all vehicles. Similarly, number the pixels in the upper border area of ​​the following vehicles. Subtract the gray values ​​of the pixels with the same number and take the absolute value to calculate the pixel gray value difference between all pixels. Count the number of pixels whose pixel gray value difference is greater than the difference threshold. If the number of pixels exceeds the threshold, no operation is performed. If the number of pixels is less than or equal to the number threshold, the corresponding vehicle is determined to be a suspected following vehicle, and the process proceeds to step S45. Step S45: Obtain the interval distance between the camera on the previous road and the camera on the current road, the lower limit of the current road interval, the first time point when the camera on the previous road captures the suspected following vehicle, and the second time point when the camera on the current road captures the suspected following vehicle. Step S46: Divide the interval distance by the upper limit of the current road interval to calculate the minimum travel time for the vehicle to travel from the position of the previous road camera to the position of the current road camera. Similarly, divide the interval distance by the lower limit of the current road interval to calculate the maximum travel time for the vehicle to travel from the position of the previous road camera to the position of the current road camera. Step S47: Add the minimum travel time to the first time node and use the calculated time node as the minimum endpoint value of the arrival time interval. Similarly, add the maximum travel time to the first time node and use the calculated time node as the maximum endpoint value of the arrival time interval. Construct the arrival time interval of the suspected following vehicle based on the maximum endpoint value and the minimum endpoint value. Step S48: When the second time node belongs to the arrival time interval, determine that the suspected following vehicle is the following vehicle, and proceed to step S49. When the second time point does not fall within the arrival time interval, the suspected following vehicle is determined not to be a following vehicle. Step S49: Convert the real-time vehicle video corresponding to the following vehicle captured by the previous road camera into video frame images, and repeat steps S31 to S34 to obtain the license plate number of the following vehicle.

2. The image recognition method based on multi-target detection according to claim 1, characterized in that, Step S1 includes the following sub-steps: Step S11: Set up a laser vehicle detector within the shooting area of ​​the road camera. When the vehicle to be detected in the road triggers the laser vehicle detector, the road camera captures a real-time video of the vehicle to be detected. Step S12: Obtain the output frame rate corresponding to the road camera; If the output frame rate is greater than or equal to the upper limit of the frame rate, a fixed number of images will be randomly selected as video frame images of the road within a fixed time interval. If the output frame rate is less than the upper limit of the frame rate, the output frame rate is multiplied by a fixed duration, and the calculated result is used as the video frame image of the road. Step S13: Draw a perpendicular line from half the length of the video frame image to the long side of the video frame, and draw a perpendicular line from half the width of the video frame image to the wide side of the video frame. Record the pixel point corresponding to the intersection of the two perpendicular lines as the center pixel point of the video frame image. Step S14: When there are no vehicles driving on the road, the road camera takes a picture of the road. Step S13 is repeated to obtain the center pixel of the road image, and an initial marker point that coincides with the center pixel is set in the road.

3. The image recognition method based on multi-target detection according to claim 2, characterized in that, Step S1 further includes the following sub-steps: Step S15: Starting from the initial marker point, set positioning marker points at fixed intervals and number all positioning marker points. Step S16: Identify the vehicle bounding box of the vehicle to be detected in all video frame images, and connect the pixels with the maximum vertical coordinate in the vehicle bounding box to obtain the roof pixel sequence of the vehicle to be detected in the video frame images. The video frame image corresponding to the intersection of the center pixel and the roof pixel sequence is taken as the detection image of the vehicle to be detected. Step S17: When the center pixel point intersects with the roof pixel sequence, obtain the positioning marker point with the smallest distance to the rear of the vehicle, and then obtain the distance between the positioning marker point and the initial marker point, and record it as the actual displacement distance of the vehicle to be detected. Step S18: Obtain the horizontal distance between the road camera and the initial marker point and the vertical height of the road camera, and calculate the vehicle height of the vehicle to be detected; Step S19: When the vehicle height is greater than or equal to the height threshold, proceed to step S2; No action is taken when the vehicle height is below the height threshold.

4. The image recognition method based on multi-target detection according to claim 3, characterized in that, Step S2 includes the following sub-steps: Step S21: Mark the vehicle border in the image to be detected, and extend it by a fixed pixel length in the positive Y-axis direction with the vehicle top pixel sequence as the baseline. Use the extended area as the rear detection area of ​​the image to be detected. Step S22: Obtain the pixel values ​​of all pixels in the image to be detected, then convert the image to be detected into a grayscale image to be detected and obtain the pixel grayscale values ​​of all pixels in the detection area behind the grayscale image to be detected. Step S23: Convert the road image captured by the road camera when there are no vehicles on the road into a grayscale road image, obtain the grayscale value of all pixels in the grayscale road image, sum the grayscale values ​​and take the average value to calculate the average grayscale value of the grayscale road image. Step S24: Subtract the average gray value from the gray values ​​of all pixels in the rear detection area and take the absolute value to calculate the gray difference of all pixels.

5. The image recognition method based on multi-target detection according to claim 4, characterized in that, Step S2 further includes the following sub-steps: Step S25: If the grayscale difference is greater than the minimum grayscale difference, then the corresponding pixel in the rear detection area is determined to be a suspected vehicle pixel, and the total number of suspected vehicle pixels is counted, and then proceed to step S26. If the grayscale difference is less than or equal to the minimum grayscale difference, it is determined that there is no following vehicle in the rear detection area; Step S26: Obtain the real-time speed of the vehicle to be detected and the speed range of the vehicle in the current road; wherein, the speed range includes a first speed range, a second speed range and a third speed range. When the real-time vehicle speed falls within the first speed range, the first image following the image to be detected is selected as the following vehicle detection image, based on the chronological order of the real-time vehicle video. When the real-time vehicle speed falls within the second speed range, the third image following the image to be detected is selected as the following vehicle detection image, based on the chronological order of the real-time vehicle video. When the real-time vehicle speed falls within the third speed range, the fifth image following the image to be detected is selected as the following vehicle detection image, based on the chronological order of the real-time vehicle video. Step S27, repeat step S16 to obtain the number of roof pixel sequences and the corresponding number of pixels in the rear detection area of ​​the following vehicle detection image; Step S28: If the number of roof pixel sequences in the vehicle detection image is one, and the total number of suspected vehicle pixels is less than the number threshold, then it is determined that there is no following vehicle in the rear detection area; if the number of roof pixel sequences in the vehicle detection image is greater than or equal to two, or the total number of suspected vehicle pixels is greater than or equal to the number threshold, then it is determined that there is a following vehicle in the rear detection area, and proceed to step S3.

6. The image recognition method based on multi-target detection according to claim 5, characterized in that, Step S3 includes the following sub-steps: Step S31: Repeat step S16 to identify the vehicle border of the following vehicle in the vehicle detection image; if there is a front border area and a windshield border area in the vehicle border of the following vehicle, then the front border area and the windshield border area are cropped. If there is no front frame area or windshield frame area in the vehicle frame of the following vehicle, it is determined that the vehicle to be detected is obstructing the following vehicle, and proceed to step S4. Step S32: Obtain the color corresponding to the license plate type for all license plates, and convert the color of all license plates to the standard license plate grayscale value; Step S33: Convert the front border region into a grayscale image, and then extract all rectangular regions of the front border region; if there are two grayscale values ​​in any rectangular region, proceed to step S34; if there are more than two grayscale values ​​in all rectangular regions, it is determined that the vehicle to be detected is occluding the following vehicle, and proceed to step S4. Step S34: The rectangular area of ​​the following vehicle is identified using a license plate recognition algorithm; when the rectangular area is the license plate of the following vehicle, the license plate number of the following vehicle is identified; when the rectangular area is not the license plate of the following vehicle, no operation is performed.

7. An image recognition system based on multi-target detection, characterized in that, The image recognition method based on multi-target detection according to any one of claims 1-6 includes: The data acquisition module is used to acquire real-time vehicle videos from the current road camera and the previous road camera in the road. The video segmentation module is used to segment real-time vehicle video into video frame images; The height detection module is used to determine whether the height of the vehicle to be detected is greater than a height threshold. The image segmentation module is used to segment the rear detection region in the image to be detected; The vehicle detection module is used to determine whether there are following vehicles in the rear detection area; The information investigation module is used to obtain the license plate number of the following vehicle based on the border area; The occlusion detection module is used to determine whether the vehicle under test occludes the following vehicle; The vehicle detection module is used to determine whether a suspected following vehicle is a true following vehicle based on upstream and downstream road cameras; The database module stores the license plate type and corresponding color for all license plates, the current speed range of the road, the distance between adjacent road cameras, and the vertical height of the road cameras.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the image recognition method according to any one of claims 1 to 6.

Images