Vehicle identification methods, devices, terminal equipment and computer-readable storage media

Abstract

This application relates to the field of transportation technology and provides a vehicle identification method, device, terminal equipment, and computer-readable storage medium, applied to a vehicle identification system. The vehicle identification system includes radar and a binocular camera. The method includes: identifying first vehicle information of a target vehicle using the radar; when the target vehicle reaches a first preset position, tracking and photographing the target vehicle using the binocular camera until the target vehicle leaves the camera's field of view; identifying second vehicle information of the target vehicle based on the image set obtained by the binocular camera's tracking and photographing; and identifying the vehicle type of the target vehicle based on the first and second vehicle information. This method can effectively improve the identification accuracy and reliability of free-flow systems.

Description

Technical Field

[0001] This application belongs to the field of transportation technology, and in particular relates to a vehicle identification method, device, terminal equipment and computer-readable storage medium. Background Technology

[0002] In recent years, free-flow systems have been widely used in the field of intelligent transportation. These systems can monitor vehicle status and enable non-stop toll collection. For example, in non-stop toll collection scenarios, free-flow systems can identify the type of vehicle during transit and feed the identification results back to the toll collection system, which then executes the toll collection operation.

[0003] Most existing free-flow systems use cameras to capture images of vehicles in motion, then perform image recognition processing on the captured images to determine vehicle type, driving status, and other vehicle information. However, because the camera's shooting frequency is fixed, it's easy to miss images at high speeds; additionally, in adverse weather conditions such as low visibility, blurry or unclear images are easily captured. All of these factors reduce the accuracy and reliability of the free-flow system's recognition capabilities. Summary of the Invention

[0004] This application provides a vehicle identification method, apparatus, terminal device, and computer-readable storage medium, which can improve the identification accuracy and reliability of free-flow systems.

[0005] In a first aspect, embodiments of this application provide a vehicle identification method applied to a vehicle identification system, the vehicle identification system including radar and a binocular camera, the method comprising:

[0006] The radar identifies the first vehicle information of the target vehicle.

[0007] When the target vehicle reaches the first preset position, the binocular camera tracks and photographs the target vehicle until the target vehicle leaves the shooting range of the binocular camera;

[0008] Second vehicle information of the target vehicle is identified based on the image set obtained by the binocular camera tracking and capturing.

[0009] The vehicle type of the target vehicle is identified based on the first vehicle information and the second vehicle information.

[0010] In this embodiment, the first vehicle information of the target vehicle is identified by radar, and the second vehicle information is identified by tracking and photographing the target vehicle using a binocular camera. This is equivalent to using multiple sensors to acquire vehicle information. Finally, the vehicle type of the target vehicle is identified by combining the first and second vehicle information, which is equivalent to considering the vehicle information acquired by each sensor during the identification process. This method avoids the low identification accuracy caused by the limited accuracy of a single sensor and the randomness of the identification results, effectively improving the identification accuracy and reliability of the vehicle identification system.

[0011] In one possible implementation of the first aspect, the binocular camera includes a first camera and a second camera;

[0012] The step of tracking and photographing the target vehicle using the binocular camera when the target vehicle reaches a first preset position until the target vehicle leaves the field of view of the binocular camera includes:

[0013] When the target vehicle reaches the first preset position, the first camera tracks and films the target vehicle until the target vehicle reaches the second preset position;

[0014] When the target vehicle reaches the second preset position, the second camera tracks and photographs the target vehicle until the target vehicle leaves the shooting area of ​​the second camera.

[0015] In one possible implementation of the first aspect, the second vehicle information includes third vehicle information and fourth vehicle information;

[0016] The image set includes M first images captured by the first camera and N second images captured by the second camera, where M and N are positive integers;

[0017] The step of identifying the second vehicle information of the target vehicle based on the image set obtained by the binocular camera includes:

[0018] The third vehicle information of the target vehicle is identified based on the M first captured images;

[0019] The fourth vehicle information of the target vehicle is identified based on the N second captured images.

[0020] In one possible implementation of the first aspect, the third vehicle information includes the number of axles;

[0021] The step of identifying the third vehicle information of the target vehicle based on the M first captured images includes:

[0022] Identify the axle in each of the first captured images to obtain the first axle information for each of the first captured images;

[0023] Each of the first captured images is assigned a weight based on a third preset position and the first axle information, wherein the third preset position is located between the first preset position and the second preset position;

[0024] Based on the first axle information and the weight in each of the first captured images, the detection score of the identified axle is calculated;

[0025] The number of axles of the target vehicle is determined based on the detection score.

[0026] In one possible implementation of the first aspect, identifying the fourth vehicle information of the target vehicle based on the N second captured images includes:

[0027] The N second-captured images are stitched together to obtain a stitched image.

[0028] The fourth vehicle information of the target vehicle is identified based on the stitched image.

[0029] In one possible implementation of the first aspect, the step of performing image stitching processing on the N second captured images to obtain a stitched image includes:

[0030] Calculate the similar region between the third and fourth captured images, wherein the third and fourth captured images are two adjacent frames of the second captured images;

[0031] If the image similarity corresponding to the similar region is greater than or equal to the first preset threshold, then the third captured image and the fourth captured image are stitched together according to the similar region to obtain a stitched image of the third captured image and the fourth captured image.

[0032] If the image similarity corresponding to the similar region is less than a first preset threshold, then the distance difference between the first distance corresponding to the third captured image and the second distance corresponding to the fourth captured image is calculated, wherein the first distance is the distance between the front of the target vehicle detected by the radar and the third preset position when the third captured image is acquired, and the second distance is the distance between the front of the target vehicle detected by the radar and the third preset position when the fourth captured image is acquired.

[0033] Find the pixel positions corresponding to the distance difference in the third and fourth captured images respectively;

[0034] Based on the pixel positions, the third and fourth captured images are stitched together to obtain a stitched image of the third and fourth captured images.

[0035] In one possible implementation of the first aspect, identifying the vehicle type of the target vehicle based on the first vehicle information and the second vehicle information includes:

[0036] If the current weather type is type 1, then the vehicle type of the target vehicle is identified based on the first vehicle information and the fourth vehicle information to obtain a first identification result; the first identification result is corrected based on the second vehicle information to obtain a final identification result.

[0037] If the current weather type is type two, the vehicle type of the target vehicle is identified based on the third vehicle information to obtain a second identification result; the second identification result is then corrected based on the first vehicle information and the third vehicle information to obtain the final identification result.

[0038] In one possible implementation of the first aspect, the detection range of the radar is greater than the shooting range of the binocular camera, and there is a target cross section in the detection range of the radar, wherein the target cross section is a detection cross section perpendicular to the driving direction of the target vehicle;

[0039] The second preset position is the intersection of the target cross section and the road surface.

[0040] In one possible implementation of the first aspect, the first camera is a white light camera, and the angle between the lens axis of the first camera and the driving direction of the target vehicle is less than 90 degrees.

[0041] The second camera is a red light camera, and the angle between the lens axis of the second camera and the driving direction of the target vehicle is equal to 90 degrees.

[0042] Secondly, embodiments of this application provide a vehicle identification device applied to a vehicle identification system, the vehicle identification system including radar and a binocular camera, the device comprising:

[0043] The first identification unit is used to identify first vehicle information of the target vehicle through the radar.

[0044] The first shooting unit is used to track and shoot the target vehicle through the binocular camera when the target vehicle reaches the first preset position, until the target vehicle leaves the shooting range of the binocular camera;

[0045] The second identification unit is used to identify second vehicle information of the target vehicle based on the image set obtained by the binocular camera tracking and capturing.

[0046] The third identification unit is used to identify the vehicle type of the target vehicle based on the first vehicle information and the second vehicle information.

[0047] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the vehicle identification method as described in any one of the first aspects above.

[0048] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the vehicle identification method as described in any one of the first aspects above.

[0049] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute the vehicle identification method described in any one of the first aspects.

[0050] In a sixth aspect, embodiments of this application provide a vehicle identification system, which includes a radar, a binocular camera, and a data processing device. The data processing device is communicatively connected to the radar and the binocular camera, respectively, and is used to implement the vehicle identification method as described in any one of the first aspects above.

[0051] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a schematic diagram of the vehicle identification system provided in the embodiments of this application;

[0054] Figure 2 This is a schematic diagram of the system installation provided in the embodiments of this application;

[0055] Figure 3 This is a schematic diagram of system installation provided in another embodiment of this application;

[0056] Figure 4 This is a flowchart illustrating the vehicle identification method provided in an embodiment of this application;

[0057] Figure 5 This is a structural block diagram of the vehicle identification device provided in the embodiments of this application;

[0058] Figure 6 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. Detailed Implementation

[0059] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0060] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0061] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0062] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection."

[0063] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0064] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0065] See Figure 1 This is a schematic diagram of the vehicle recognition system provided in an embodiment of this application. Figure 1 As shown, the vehicle recognition system may include a radar 11, a binocular camera 12, and a data processing device 13. The data processing device is communicatively connected to both the radar and the binocular camera. The binocular camera includes a first camera and a second camera. During vehicle recognition, after the radar detects a target vehicle, it feeds the detection data back to the data processing device, which identifies the first vehicle information of the target vehicle based on the detection data. After the binocular camera captures an image of the target vehicle, it feeds the image back to the data processing device, which performs image recognition processing on the captured image to obtain the second vehicle information of the target vehicle. Then, the data processing device identifies the vehicle type of the target vehicle based on the first and second vehicle information.

[0066] LiDAR has advantages such as high resolution, strong anti-interference ability, small size and light weight. Therefore, optionally, in the embodiments of this application, the radar can be a lidar.

[0067] Furthermore, the radar's detection range is larger than that of a binocular camera. Therefore, when the binocular camera is tracking and filming a vehicle, the radar's assistance in detection can effectively improve the effectiveness of the detection. The radar's detection range includes a target cross-section, which is perpendicular to the target vehicle's direction of travel. Since perpendicular detection has high accuracy, ensuring the presence of a target cross-section within the radar's detection range during installation can further improve the radar's detection precision.

[0068] Optionally, the first camera is a white light camera with a white light fill light; the second camera is a red light camera with a red light fill light. This configuration allows the two cameras to complement each other, further improving the shooting accuracy of the stereo camera.

[0069] Preferably, the angle between the lens axis of the first camera and the vehicle's direction of travel is less than 90 degrees; the angle between the lens axis of the second camera and the vehicle's direction of travel is equal to 90 degrees. The first camera captures the vehicle at an angle, allowing for the acquisition of richer vehicle feature information (such as license plate, front of the vehicle, body, and axles). The second camera captures the vehicle at a vertical angle, making vehicle identification using red light clearer at night or in low light conditions, and ensuring that infrared light is not affected by white light, thus increasing the stability and accuracy of vehicle identification at night or in low light environments.

[0070] In one application scenario, see Figure 2 This is a schematic diagram of the system installation provided in an embodiment of this application. Figure 2As shown, a detection pole is installed on one side of the road, and a lidar is mounted on the pole. The lidar can be installed at a height of 6-8 meters, and at an angle of 30-90 degrees to the road surface. During installation, at least one detection section of the lidar should be 10-20 meters away from the intersection of its detection section and the road surface, and there should be a detection section perpendicular to the direction of travel. A binocular camera is also installed on the detection pole, at a height of 6-8 meters. The first camera is at an angle of 40-80 degrees to the direction of travel, and the second camera is at an angle of 90 degrees to the direction of travel. The data processing device can be installed on the detection pole or it can be a terminal device such as a cloud server, remotely communicating with the lidar and the binocular camera.

[0071] In another application scenario, see Figure 3 This is a schematic diagram of system installation provided in another embodiment of this application. For example... Figure 3 As shown, detection poles can be installed on both sides of the road, each equipped with radar and a binocular camera. This installation method can solve the problems of missed detections or obstruction by merging vehicles when traffic is heavy. Of course, this installation method is also more expensive.

[0072] It should be noted that the above is merely a hardware installation example for a vehicle recognition system and is not intended to impose specific limitations. In practical applications, the radar and binocular cameras can be installed separately on different detection poles, and multiple radars and / or multiple binocular cameras can be installed on each detection pole. The specific installation method can be configured according to different application scenarios.

[0073] Based on the vehicle recognition system described above, the vehicle recognition method provided in this application is described below. This method is executed by the data processing device in the vehicle recognition system described above. See also Figure 4 This is a schematic flowchart of the vehicle recognition method provided in the embodiments of this application. It is intended as an example and not a limitation. The method may include the following steps:

[0074] S401 is the first vehicle information used to identify target vehicles via radar.

[0075] Taking lidar as an example, lidar emits laser light at a certain frequency. When a vehicle passes through its detection range, the laser light hits the vehicle and returns. Based on the emitted and reflected beams, lidar can detect the vehicle's length, width, height, number of axles, and driving data such as speed and acceleration. This data can then be sent as initial vehicle information to a data processing device.

[0076] Whenever a vehicle is detected, the radar or data processing device can assign a number to that vehicle to distinguish it from other vehicles.

[0077] S402: When the target vehicle reaches the first preset position, the binocular camera tracks and photographs the target vehicle until the target vehicle leaves the shooting range of the binocular camera.

[0078] Optionally, the first preset position can be the first edge of the binocular camera's shooting range in the direction of oncoming traffic. Alternatively, a position can be manually set within the binocular camera's shooting range, for example, the first preset position can be set to a location within the binocular camera's shooting range that is 12-15 meters away from the radar.

[0079] Whether a vehicle has reached the first preset position can be monitored by radar. For example, after setting the first preset position, the relative distance between that position and the radar is also determined accordingly; when the radar detects that the relative distance between the vehicle and the radar reaches the preset value, it is determined that the vehicle has reached the first preset position.

[0080] When a vehicle reaches a first preset position, the radar can send trigger information to the binocular camera, instructing the camera to track and photograph the target vehicle. Alternatively, the radar can send a trigger signal (indicating the vehicle has reached the first preset position) to the data processing device, which then sends trigger information to the binocular camera. For example, when the radar sends trigger information to the binocular camera, the trigger information may include the vehicle's number and first vehicle information identified by the radar. The binocular camera uses this trigger information to determine which vehicle is the target vehicle and tracks and photographs it. When the data processing device sends trigger information, the radar can send the trigger signal to the data processing device. The data processing device packages the first vehicle information corresponding to the trigger signal and the trigger signal into trigger information and sends it to the binocular camera. The binocular camera uses this trigger information to determine which vehicle is the target vehicle and tracks and photographs it.

[0081] It should be noted that once the vehicle reaches the first preset position, the radar can continue to detect the vehicle during its subsequent movement, meaning that both the radar and the binocular camera are tracking the vehicle.

[0082] In one embodiment, to improve detection accuracy, the binocular camera can perform self-detection in addition to passively waiting for trigger information. Specifically, when the binocular camera detects that a vehicle has reached a first preset position, it can continue to track and photograph the vehicle. This method effectively avoids missed detections caused by missed triggers from the radar or data processing device.

[0083] S403 identifies second vehicle information of the target vehicle based on a set of images captured by a binocular camera.

[0084] A binocular camera can send batches of captured images to a data processing device at a certain frequency, or it can send each captured image to the data processing device as soon as it is acquired.

[0085] When processing each image in the image set, the data processing device first performs target recognition to distinguish different vehicles; then it performs target recognition a second time to identify second vehicle information such as the number of axles, length, width, height, license plate number, and license plate color. The specific process for identifying the second vehicle information can be found in the description of the following embodiments.

[0086] S404, Identify the vehicle type of the target vehicle based on the first vehicle information and the second vehicle information.

[0087] The data processing unit combines the first vehicle information and the second vehicle information to identify the vehicle type of the target vehicle, which is equivalent to considering the vehicle information acquired by each sensor during the identification process. This method avoids the low identification accuracy caused by the limited accuracy of a single sensor and the randomness of the identification results, effectively improving the identification accuracy, robustness, and reliability of the vehicle identification system.

[0088] In free-flow tolling applications, the data processing unit sends the identified vehicle type to the toll collection device, which then determines the toll based on the vehicle type and automatically deducts the corresponding amount from the vehicle's account to achieve non-stop toll collection.

[0089] like Figure 1 As described in the embodiments, the binocular camera has two cameras; correspondingly, in one embodiment, S402 may include:

[0090] When the target vehicle reaches the first preset position, the first camera tracks and films the target vehicle until it reaches the second preset position; when the target vehicle reaches the second preset position, the second camera tracks and films the target vehicle until it leaves the filming area of ​​the second camera.

[0091] Optionally, the second preset position is the intersection of the target cross section (the detection cross section within the radar detection range that is perpendicular to the driving direction of the target vehicle) and the road surface.

[0092] Whether a vehicle has reached the second preset position can be determined by recognizing the captured image. For example, a pixel position corresponding to the second preset position can be set in the captured image. When the vehicle in the captured image reaches that pixel position, it is determined that the vehicle has reached the second preset position. Of course, the second preset position can also be marked with a line on the road surface, but this method is more costly and inconvenient to operate in some application scenarios.

[0093] Accordingly, the second vehicle information includes the third vehicle information and the fourth vehicle information; the image set obtained by the binocular camera tracking includes M first images captured by the first camera and N second images captured by the second camera, where M and N are positive integers. S403 may include:

[0094] I. Identify the third vehicle information of the target vehicle based on M first-shot images.

[0095] II. Identify the fourth vehicle information of the target vehicle based on N second-shot images.

[0096] Optionally, in step I, the step of identifying the number of axles of the target vehicle based on the M first captured images includes: identifying the axles in each first captured image to obtain the number of axles in each first captured image; counting the number of axles that appear most frequently in the M first captured images, and determining this number of axles as the number of axles of the target vehicle.

[0097] For example, assuming M = 100, where 80 of the first images show a vehicle with 6 axles and 20 of the first images show a vehicle with 4 axles, then the target vehicle is determined to have 6 axles.

[0098] The above method uses only quantity as the judgment criterion, which often results in low reliability. Since cameras have relatively high accuracy when capturing close-up objects but relatively low accuracy when capturing distant objects, this characteristic can be utilized to optionally include the following steps for identifying the number of axles of a target vehicle based on M first captured images:

[0099] Identify the axle in each first captured image to obtain the first axle information for each first captured image; assign weights to each first captured image based on a third preset position and the first axle information, wherein the third preset position is located between the first preset position and the second preset position; calculate the detection score of the identified axle based on the first axle information and weights of each first captured image; determine the number of axles of the target vehicle based on the detection scores.

[0100] Preferably, the third preset position is the intersection of the target cross-section and the road surface. The method for determining the third preset position can refer to the method for determining the second preset position, and will not be repeated here. The third preset position can be regarded as the optimal shooting position of the first camera.

[0101] The first axle information may include the number of axles, the detection frame for each axle, and the detection frame number.

[0102] Optionally, the weights can be assigned based on the distance between the target vehicle in the first captured image and the third preset position. In other words, if the target vehicle in the first captured image is farther from the third preset position, the first captured image is assigned a smaller weight; if the target vehicle in the first captured image is closer to the third preset position, the first captured image is assigned a larger weight.

[0103] For example, assuming M = 5, where the target vehicle is closest to the third preset position in the fourth first image, then the weights assigned to the five first images are 0.1, 0.15, 0.25, 0.4, and 0.1 respectively. It should be noted that the above is merely an example of weight allocation and does not impose specific limitations on the value of M or the assigned weights.

[0104] Because the lens axis of the first camera is at a certain angle to the vehicle's direction of travel, the detection effect of the vehicle at different positions in the image may vary depending on how far away it is viewed from the camera. For example, the axle may not be detected in frame t, but it may appear in frame t+1. This axle needs to be marked as a new axle and tracked and detected again. Conversely, the axle may be detected in frame t, but not in frame t+1. This axle needs to be marked as a false axle and tracked and detected again.

[0105] Optionally, one method for calculating the detection score is as follows: For each axle, calculate the first score for that axle in each first image based on the weight of each first captured image and the axle information; sum the first scores corresponding to each of the M first captured images to obtain the detection score for that axle. Specifically, if the axle is not detected in the first captured image, then the first score for that axle in the first captured image is 0; if the axle is detected in the first captured image, then the first score for that axle in the first captured image is the weight of that first captured image.

[0106] For example, assuming M = 5, the weights assigned to the 5 first-captured images are 0.1, 0.15, 0.25, 0.4, and 0.1 respectively. Based on the axle information in each first-captured image, axle A is not detected in images 1-2, but is detected in images 3-5. Therefore, the detection score for axle A is 0 + 0 + 0.25 + 0.4 + 0.1 = 0.75.

[0107] Accordingly, one way to determine the number of axles of the target vehicle based on the detection score is as follows: for each axle, if the detection score of the axle is greater than or equal to a first preset score, then the detection result of the axle is that the target vehicle has the axle; if the detection score of the axle is less than the first preset score, then the detection result of the axle is that the target vehicle does not have the axle; after determining the detection result of each axle, the number of axles of the target vehicle is determined based on the detection result of each vehicle.

[0108] The above method requires calculating the sum of weights for M first-shot images, which is computationally intensive. To reduce computation, an alternative method for determining the number of axles of the target vehicle based on the detection score is as follows:

[0109] For each axle, a first score for that axle in the first captured image is calculated based on the weight of the first captured image and the axle information. If the first score is greater than or equal to a second preset score, the axle is detected as a target vehicle with that axle. If the second score is less than the second preset score, the first score for that axle in the second captured image is calculated based on the weight of the second captured image and the axle information. The first score for that axle in the first captured image is then added to the first score for that axle in the second captured image to obtain a cumulative score. If the cumulative score is greater than or equal to the second preset score, the axle is detected as a target vehicle with that axle. If the cumulative score is less than the second preset score, the first score for that axle in the third captured image is calculated based on the weight of the third captured image and the axle information. The first score for that axle in the third captured image is then added to the current cumulative score to obtain an updated cumulative score. This process continues in the same manner.

[0110] For example, assuming M=5, the target vehicle is closest to the third preset position in the fourth first image, and the weights assigned to the five first images are 0.1, 0.15, 0.25, 0.4, and 0.1 respectively. Specifically, axle A is not detected in images 1-3, but is detected in images 4-5; axle B is detected in images 1-2, and is detected in images 3-5. Assume the second preset score is 0.5.

[0111] For axle A, the sum of the first scores of the first three images is 0. Since the cumulative score is less than 0.5, the first score of the fourth image is added to the cumulative score, resulting in an updated cumulative score of 0 + 0.4 = 0.4. Since the cumulative score is still less than 0.5, the first score of the fifth image is calculated and added to the cumulative score, resulting in a cumulative score of 0.4 + 0.1 = 0.5. At this point, the cumulative score equals 0.5, confirming the presence of axle A in the target vehicle.

[0112] For axle B, the first score in the first captured image is 0.1, which is less than 0.5. The first score in the second captured image is calculated and accumulated, resulting in a total score of 0.1 + 0.15 = 0.25, which is still less than 0.5. Since axle B is not detected in the third to fifth captured images, the first score in each of the third to fifth captured images is 0. Therefore, the first scores in the third to fifth captured images are accumulated, and the final total score is still 0.25 + 0 = 0.25, which is less than 0.5. Thus, it is determined that axle B does not exist in the target vehicle.

[0113] This implementation method allows for the determination of the axle's presence when the sum of the first scores from the first few captured images reaches a preset score, eliminating the need to calculate the first scores from subsequent images and effectively reducing computational load. Furthermore, this method allows the data processing device to obtain a detection result for each captured first image, meaning the detection result is updated in real-time. In contrast, the first implementation method requires acquiring all first captured images to determine the detection result. Compared to the first implementation method, this method offers significantly better real-time performance.

[0114] In one embodiment, the step of identifying fourth vehicle information of the target vehicle based on N second captured images may include:

[0115] The N second-shot images are stitched together to obtain a stitched image; the fourth vehicle information of the target vehicle is identified based on the stitched image.

[0116] like Figure 1 As described in the embodiments, when the lens axis of the second camera is perpendicular to the vehicle's driving direction, the field of view of the second camera is limited. When the vehicle body is long or the driving speed is high, the second image captured by the second camera may not show a complete vehicle image. To address this situation, this embodiment of the application performs image stitching processing on the second captured image to ensure the integrity of the vehicle image in the second captured image, thereby improving the detection accuracy of vehicle information.

[0117] Alternatively, one implementation of image stitching is as follows:

[0118] Calculate the similar regions between the third and fourth captured images, where the third and fourth captured images are two adjacent frames of the second captured images;

[0119] If the image similarity corresponding to the similar region is greater than or equal to the first preset threshold, then the third and fourth captured images are stitched together according to the similar regions to obtain a stitched image of the third and fourth captured images.

[0120] If the image similarity corresponding to the similar region is less than the first preset threshold, then the distance difference between the first distance corresponding to the third captured image and the second distance corresponding to the fourth captured image is calculated. The first distance is the distance between the front of the target vehicle detected by the radar and the third preset position when the third captured image is acquired, and the second distance is the distance between the front of the target vehicle detected by the radar and the third preset position when the fourth captured image is acquired. The pixel positions corresponding to the distance difference in the third captured image and the fourth captured image are found respectively. The third captured image and the fourth captured image are stitched together according to the pixel positions to obtain the stitched image of the third captured image and the fourth captured image.

[0121] In the above implementation, similar regions between two adjacent images can be determined using existing template matching methods, which will not be elaborated further here. Template matching methods can calculate the similarity between different regions of two images, and the region with the highest similarity can be identified as the similar region between the two images.

[0122] Although theoretically, adjacent frames will always have overlapping parts within the stitching area, existing algorithms may not be able to accurately identify similar parts between adjacent frames due to the susceptibility of video to ambient light. When the maximum similarity is less than or equal to a first threshold, this embodiment uses radar-assisted detection of the displacement of the vehicle's front position in adjacent frames as the basis for frame stitching displacement. Because radar's detection characteristics are not easily affected by ambient light, radar-assisted detection can effectively improve detection accuracy.

[0123] In practical applications, when performing image stitching, the width of the stitching region is greater than the maximum displacement of the vehicle in the two adjacent frames. This ensures that there is overlap between the vehicle and the stitching region in the two adjacent frames.

[0124] Alternatively, another way to implement image stitching is:

[0125] Set the frame splicing area window;

[0126] If the image similarity between the framed region windows in the third and fourth images is greater than or equal to the first preset threshold, then the third and fourth images are stitched together based on the image corresponding to the current framed region window to obtain a stitched image of the third and fourth images:

[0127] If the image similarity between the stitching region windows in the third and fourth captured images is less than a first preset threshold, the stitching region window of the fourth captured image is moved forward by a preset pixel position, and the similarity between the image corresponding to the current stitching region window of the fourth captured image and the image corresponding to the stitching region window of the third captured image is calculated. If the similarity is greater than the first preset threshold, the third and fourth captured images are stitched together based on the image corresponding to the current stitching region window. If the similarity is still less than the first preset threshold, the stitching region window of the fourth captured image is moved forward by a preset pixel position until it is moved H times (preset number of times) or a stitching region with a similarity greater than the first preset threshold is found.

[0128] If, after moving H times, no stitching region with a similarity greater than the first preset threshold is found, the distance difference between the first distance corresponding to the third captured image and the second distance corresponding to the fourth captured image is calculated; the pixel positions corresponding to the distance difference in the third captured image and the fourth captured image are found respectively; and the third captured image and the fourth captured image are stitched together according to the pixel positions to obtain a stitched image of the third captured image and the fourth captured image.

[0129] Due to the influence of vehicles or gantries in the image, the previous frame may have shadows while the next frame does not, which will affect the model matching effect. Therefore, in this embodiment, a frame stitching region window is set, and the above problem can be solved by sliding the frame stitching region window.

[0130] based on Figure 1 In the described embodiment, the two cameras of the binocular camera are a white light camera and a red light camera, respectively, which can acquire information about a third vehicle and information about a fourth vehicle. In one embodiment, S404 may include:

[0131] If the current weather type is type 1, the vehicle type of the target vehicle is identified based on the first vehicle information and the fourth vehicle information to obtain the first identification result; the first identification result is corrected based on the second vehicle information to obtain the final identification result.

[0132] If the current weather type is type 2, the vehicle type of the target vehicle is identified based on the third vehicle information to obtain the second identification result; the second identification result is then corrected based on the first and third vehicle information to obtain the final identification result.

[0133] For example, the first type includes clear days and nighttime. Taking nighttime as an example, since radar detection is not easily affected by light and the second camera (red light camera) is not easily affected by white light, more stable and accurate detection results can be obtained at night through radar and the second camera (red light camera). In addition, the third vehicle information from the first camera can further improve the detection accuracy.

[0134] The second type includes rainy / snowy weather and fog / haze. In these weather conditions, particulate matter from rain, snow, or fog / haze may affect radar detection. Since white light cameras are equipped with white light supplementary lighting, they perform better than red light cameras in these conditions. Therefore, in these weather conditions, the first camera can obtain more accurate detection results, which, combined with the radar's first vehicle information and the second camera's fourth vehicle information, can further improve detection accuracy.

[0135] It should be noted that in practical applications, the choice can be made regarding which vehicle information is primary and which is secondary, depending on the specific application scenario. No specific limitations are made here.

[0136] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0137] Corresponding to the vehicle recognition method described in the above embodiments, Figure 5 This is a structural block diagram of the vehicle identification device provided in the embodiments of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0138] Reference Figure 5 The device includes:

[0139] The first identification unit 51 is used to identify the first vehicle information of the target vehicle through the radar.

[0140] The first shooting unit 52 is used to track and shoot the target vehicle through the binocular camera when the target vehicle reaches the first preset position, until the target vehicle leaves the shooting range of the binocular camera.

[0141] The second identification unit 53 is used to identify the second vehicle information of the target vehicle based on the image set obtained by the binocular camera tracking.

[0142] The third identification unit 54 is used to identify the vehicle type of the target vehicle based on the first vehicle information and the second vehicle information.

[0143] Optionally, the binocular camera includes a first camera and a second camera.

[0144] Correspondingly, the first imaging unit 52 is also used for:

[0145] When the target vehicle reaches the first preset position, the first camera tracks and films the target vehicle until the target vehicle reaches the second preset position; when the target vehicle reaches the second preset position, the second camera tracks and films the target vehicle until the target vehicle leaves the filming area of ​​the second camera.

[0146] Optionally, the second vehicle information includes third vehicle information and fourth vehicle information; the image set includes M first images captured by the first camera and N second images captured by the second camera, where M and N are positive integers.

[0147] Correspondingly, the second identification unit 53 is also used for:

[0148] The third vehicle information of the target vehicle is identified based on the M first images; the fourth vehicle information of the target vehicle is identified based on the N second images.

[0149] Optionally, the third vehicle information includes the number of axles.

[0150] Correspondingly, the second identification unit 53 is also used for:

[0151] Identify the axle in each of the first captured images to obtain the first axle information for each of the first captured images; assign a weight to each of the first captured images based on a third preset position and the first axle information, wherein the third preset position is located between the first preset position and the second preset position; calculate the detection score of the identified axle based on the first axle information and the weight for each of the first captured images; determine the number of axles of the target vehicle based on the detection score.

[0152] Optionally, the second identification unit 53 is also used for:

[0153] The N second captured images are stitched together to obtain a stitched image; the fourth vehicle information of the target vehicle is identified based on the stitched image.

[0154] Optionally, the second identification unit 53 is also used for:

[0155] Calculate the similar region between the third and fourth captured images, wherein the third and fourth captured images are two adjacent frames of the second captured images;

[0156] If the image similarity corresponding to the similar region is greater than or equal to the first preset threshold, then the third captured image and the fourth captured image are stitched together according to the similar region to obtain a stitched image of the third captured image and the fourth captured image.

[0157] If the image similarity corresponding to the similar region is less than a first preset threshold, then the distance difference between the first distance corresponding to the third captured image and the second distance corresponding to the fourth captured image is calculated, wherein the first distance is the distance between the front of the target vehicle detected by the radar and the third preset position when the third captured image is acquired, and the second distance is the distance between the front of the target vehicle detected by the radar and the third preset position when the fourth captured image is acquired.

[0158] Find the pixel positions corresponding to the distance difference in the third and fourth captured images respectively;

[0159] Based on the pixel positions, the third and fourth captured images are stitched together to obtain a stitched image of the third and fourth captured images.

[0160] Optionally, the third identification unit 54 is also used for:

[0161] If the current weather type is type 1, then the vehicle type of the target vehicle is identified based on the first vehicle information and the fourth vehicle information to obtain a first identification result; the first identification result is corrected based on the second vehicle information to obtain a final identification result.

[0162] If the current weather type is type two, the vehicle type of the target vehicle is identified based on the third vehicle information to obtain a second identification result; the second identification result is then corrected based on the first vehicle information and the third vehicle information to obtain the final identification result.

[0163] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0164] in addition, Figure 5 The vehicle recognition device shown can be a software unit, a hardware unit, or a combination of software and hardware built into an existing terminal device, or it can be integrated into the terminal device as an independent accessory, or it can exist as a standalone terminal device.

[0165] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0166] Figure 6 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. For example... Figure 6 As shown, the terminal device 6 in this embodiment includes: at least one processor 60 ( Figure 6 (Only one is shown) a processor, a memory 61, and a computer program 62 stored in the memory 61 and executable on the at least one processor 60, which, when executing the computer program 62, implements the steps in any of the above vehicle identification method embodiments.

[0167] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. This terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 6 This is merely an example of terminal device 6 and does not constitute a limitation on terminal device 6. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0168] The processor 60 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0169] In some embodiments, the memory 61 may be an internal storage unit of the terminal device 6, such as a hard disk or memory of the terminal device 6. In other embodiments, the memory 61 may be an external storage device of the terminal device 6, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal device 6. Furthermore, the memory 61 may include both internal and external storage units of the terminal device 6. The memory 61 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0170] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0171] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments above.

[0172] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / terminal equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0173] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0174] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0175] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0176] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0177] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A vehicle identification method, characterized in that, An application is made in a vehicle recognition system, the vehicle recognition system including radar and a binocular camera, the binocular camera including a first camera and a second camera; The method includes: The radar identifies the first vehicle information of the target vehicle. When the target vehicle reaches the first preset position, the binocular camera tracks and photographs the target vehicle until the target vehicle leaves the shooting range of the binocular camera; The target vehicle's second vehicle information is identified based on the image set captured by the binocular camera; wherein the second vehicle information includes third vehicle information, which includes the number of axles; the image set includes M first images captured by the first camera, where M is a positive integer; The vehicle type of the target vehicle is identified based on the first vehicle information and the second vehicle information; The step of identifying the second vehicle information of the target vehicle based on the image set obtained by the binocular camera includes: Identify the axle in each of the first captured images to obtain the first axle information for each of the first captured images; Weights are assigned to each of the first captured images based on a third preset position and the first axle information. The third preset position is located between the first preset position and the second preset position. The second preset position is a position following the first preset position in the direction of travel of the target vehicle. Based on the first axle information and the weight in each of the first captured images, the detection score of the identified axle is calculated; The number of axles of the target vehicle is determined based on the detection score.

2. The vehicle identification method as described in claim 1, characterized in that, The step of tracking and photographing the target vehicle using the binocular camera when the target vehicle reaches a first preset position until the target vehicle leaves the field of view of the binocular camera includes: When the target vehicle reaches the first preset position, the first camera tracks and films the target vehicle until the target vehicle reaches the second preset position; When the target vehicle reaches the second preset position, the second camera tracks and photographs the target vehicle until the target vehicle leaves the shooting area of ​​the second camera.

3. The vehicle identification method as described in claim 2, characterized in that, The second vehicle information includes the fourth vehicle information; The image set includes N second-captured images obtained by the second camera, where N is a positive integer; The step of identifying the second vehicle information of the target vehicle based on the image set obtained by the binocular camera includes: The fourth vehicle information of the target vehicle is identified based on the N second captured images.

4. The vehicle identification method as described in claim 3, characterized in that, The step of identifying the fourth vehicle information of the target vehicle based on the N second captured images includes: The N second-captured images are stitched together to obtain a stitched image. The fourth vehicle information of the target vehicle is identified based on the stitched image.

5. The vehicle identification method as described in claim 4, characterized in that, The step of stitching the N second captured images together to obtain a stitched image includes: Calculate the similar region between the third and fourth captured images, wherein the third and fourth captured images are two adjacent frames of the second captured images; If the image similarity corresponding to the similar region is greater than or equal to the first preset threshold, then the third captured image and the fourth captured image are stitched together according to the similar region to obtain a stitched image of the third captured image and the fourth captured image. If the image similarity corresponding to the similar region is less than a first preset threshold, then the distance difference between the first distance corresponding to the third captured image and the second distance corresponding to the fourth captured image is calculated, wherein the first distance is the distance between the front of the target vehicle detected by the radar and the third preset position when the third captured image is acquired, and the second distance is the distance between the front of the target vehicle detected by the radar and the third preset position when the fourth captured image is acquired. Find the pixel positions corresponding to the distance difference in the third and fourth captured images respectively; Based on the pixel positions, the third and fourth captured images are stitched together to obtain a stitched image of the third and fourth captured images.

6. The vehicle identification method as described in claim 3, characterized in that, The step of identifying the vehicle type of the target vehicle based on the first vehicle information and the second vehicle information includes: If the current weather type is type 1, then the vehicle type of the target vehicle is identified based on the first vehicle information and the fourth vehicle information to obtain a first identification result; the first identification result is corrected based on the third vehicle information to obtain a final identification result. If the current weather type is type two, the vehicle type of the target vehicle is identified based on the third vehicle information to obtain a second identification result; the second identification result is then corrected based on the first vehicle information and the fourth vehicle information to obtain the final identification result.

7. The vehicle identification method as described in claim 2, characterized in that, The detection range of the radar is greater than the shooting range of the binocular camera, and there is a target cross section in the detection range of the radar. The target cross section is a detection cross section perpendicular to the driving direction of the target vehicle. The second preset position is the intersection of the target cross section and the road surface.

8. The vehicle identification method as described in claim 3, characterized in that, The first camera is a white light camera, and the angle between the lens axis of the first camera and the driving direction of the target vehicle is less than 90 degrees. The second camera is a red light camera, and the angle between the lens axis of the second camera and the driving direction of the target vehicle is equal to 90 degrees.

9. A vehicle identification device, characterized in that, An application is made in a vehicle recognition system, the vehicle recognition system including radar and a binocular camera, the binocular camera including a first camera and a second camera; The device includes: The first identification unit is used to identify first vehicle information of the target vehicle through the radar. The first shooting unit is used to track and shoot the target vehicle through the binocular camera when the target vehicle reaches the first preset position, until the target vehicle leaves the shooting range of the binocular camera; The second identification unit is used to identify second vehicle information of the target vehicle based on an image set captured by the binocular camera; wherein the second vehicle information includes third vehicle information, the third vehicle information including the number of axles; the image set includes M first images captured by the first camera, where M is a positive integer; The third identification unit is used to identify the vehicle type of the target vehicle based on the first vehicle information and the second vehicle information; The second identification unit is further configured to: identify the axle in each of the first captured images to obtain first axle information for each of the first captured images; assign weights to each of the first captured images according to a third preset position and the first axle information, wherein the third preset position is located between the first preset position and the second preset position; wherein the second preset position is a position after the first preset position in the driving direction of the target vehicle; calculate the detection score of the identified axle according to the first axle information and the weights in each of the first captured images; and determine the number of axles of the target vehicle according to the detection score.

10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 8.

12. A vehicle identification system, characterized in that, The vehicle identification system includes a radar, a binocular camera, and a data processing device. The data processing device is communicatively connected to the radar and the binocular camera, respectively, and is used to implement the method as described in any one of claims 1 to 8.

Images