Vehicle identification method and device, electronic equipment and readable storage medium

By using images overlapping with the detection area in the port environment to recognize the appearance and driving behavior characteristics of the vehicle, the problem of insufficient identification of violation vehicles in the prior art is solved, and high-accurate identification and safety guarantees are achieved without the need for renovating the vehicle.

CN118918540APending Publication Date: 2024-11-08HUAWEI TECH CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202310516649.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-08
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

In a port environment, it is difficult for the prior art to accurately identify illegal vehicles without remodeling them, especially in vehicles with similar appearances, resulting in the impact of identification accuracy and safety.

Method used

By using a first device (such as a camera or radar) to detect violation events and obtain images overlapping with the detection area, identifying the appearance and driving behavior characteristics of the vehicle to determine the identification information of the violation vehicle, and achieving accurate identification without the need for a positioning device.

Benefits of technology

It improves the accuracy of identification of illegal vehicles, avoids misidentification, enhances port traffic safety, and reduces the need for vehicle transformation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918540A_ABST
    Figure CN118918540A_ABST
Patent Text Reader

Abstract

The invention provides a vehicle identification method. The method comprises the following steps: determining that a first target vehicle has a violation event through first detection data of first equipment; acquiring a plurality of first images, wherein a first detection area of the first equipment is overlapped with a shooting area of the first images; according to a first feature of the first target vehicle in the first detection data, identification information of the first target vehicle in the first image is identified, and the first feature is used for representing the appearance and the driving behavior of the first target vehicle. Therefore, the violation vehicle identified by the first device and the violation vehicle captured in the image are the same vehicle, and the violation vehicle can be accurately identified from vehicles with similar appearances without a positioning device to avoid the transformation of the vehicles.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of intelligent terminal technology, and in particular to a vehicle identification method, device, electronic device and readable storage medium. Background Art

[0002] With the rapid development of the global economy, maritime trade has become increasingly frequent, and a large number of ships frequently travel between major ports. After the ship berths, trucks are needed to transport the cargo to the yard, usually manually driven trucks. Even if there are container terminals that use self-driving trucks, external container trucks are still required to manually transport the cargo from outside the port area to the yard, which puts huge pressure on port traffic.

[0003] When driving in the port area, drivers are prone to speeding and driving against traffic rules, which affects the order and efficiency of port traffic dispatching and even brings the risk of traffic accidents. Therefore, it is necessary to effectively identify illegal vehicles at the port, warn them, and report them, so as to ensure the safety of all personnel in the port during cargo transportation.

[0004] Since port transport vehicles are often of the same model, carrying the same cargo, for example, they are all container trucks, carrying containers with similar appearances; or similar non-operating vehicles, for example, they are all black sedans, making it difficult to distinguish different vehicles from their appearance and models, so vehicles need to be uniquely identified. In some vehicle identification schemes, positioning devices can be deployed on all vehicles entering the port as identification devices to achieve location tracking and reporting of a single vehicle, and the unique identification information of the vehicle can be associated with the reported location information. For example, the license plate number and the vehicle's location information can be associated to accurately track and identify the identification information of the illegal vehicle.

[0005] The installation and deployment of the positioning device in the car requires the cooperation of the car owner, which makes it difficult to widely promote this solution in the above vehicles. Therefore, how to improve the accuracy of identifying illegal vehicles without modifying the vehicle side has become an urgent problem to be solved. Summary of the invention

[0006] The present application provides a vehicle identification method, which determines that a first target vehicle has a traffic violation through first detection data of a first device; obtains multiple first images, and the first detection area of ​​the first device overlaps with the shooting area of ​​the first image; and identifies the identification information of the first target vehicle in the first image according to the first feature of the first target vehicle in the first detection data, and the first feature is used to characterize the appearance and driving behavior of the first target vehicle. This ensures that the traffic violation vehicle identified by the first device is the same as the traffic violation vehicle captured in the image, and achieves accurate identification of the traffic violation vehicle from vehicles with similar appearances without the need for a positioning device to avoid modification of the vehicle.

[0007] In a first aspect, an embodiment of the present application provides a vehicle identification method, the method comprising: determining that a first target vehicle has a traffic violation through first detection data of a first device; acquiring multiple first images, wherein a first detection area of ​​the first device overlaps with a shooting area of ​​the first image; and identifying identification information of the first target vehicle in the first image based on a first feature of the first target vehicle in the first detection data, wherein the first feature is used to characterize the appearance and driving behavior of the first target vehicle.

[0008] That is, the first device can detect the traffic violation event, and then determine the identification information of the illegal vehicle (i.e., the first target vehicle) in the traffic violation event detected by the first device through the image obtained in the shooting area overlapping with the detection range of the first device. For example, the first device can be a radar or a first camera, and the speeding violation event can be monitored by the radar, or the speeding violation event can be identified through the video stream shot by the first camera, and then the appearance and driving behavior of the illegal vehicle corresponding to the traffic violation event can be determined.

[0009] It can be understood that since the first device can detect the traffic violation event but it is difficult to obtain the details of the illegal vehicle, such as the identification information such as the license plate number. Therefore, a first image corresponding to a shooting range that at least partially overlaps with the detection area (i.e., the second lane image hereinafter) can be obtained. The first image is clear and convenient for determining the identification information of the illegal vehicle. Since the shooting range and the detection area at least partially overlap, the same illegal vehicle exists in the overlapping area. Furthermore, the illegal vehicle can be re-determined in the first image based on the appearance features and driving behavior features of the detected illegal vehicle. Therefore, there is no need to modify the vehicle, for example, there is no need to install a positioning device on the vehicle, so as to ensure that the illegal vehicle corresponding to the traffic violation event and the illegal vehicle whose identification information is photographed are the same vehicle, thereby avoiding the misidentification of illegal vehicles and effectively improving the recognition accuracy of illegal vehicles.

[0010] In a possible implementation of the first aspect above, the first device includes a first camera, and determining that a traffic violation has occurred on the first target vehicle through first detection data of the first device includes: capturing a second image through the first camera, and determining that a traffic violation has occurred on the first target vehicle based on the second image.

[0011] It can be understood that the first device can be a first camera, which has a larger shooting range and a more blurred image and can be used to identify traffic violations.

[0012] For example, a continuous multi-frame second image (i.e., the first lane image hereinafter) may be captured by the first camera, and the driving speed of each vehicle may be determined by analyzing the multi-frame second image, thereby monitoring speeding, illegal parking and other violations.

[0013] In some embodiments, multiple frames of the second image can be analyzed in combination with map data and a neural network model. For example, the real-time location of the vehicle and the real-time location of the traffic sign can be determined in combination with the map data, and the content of the traffic sign can be identified in combination with the neural network model, so as to determine whether each vehicle is in violation of traffic rules. Based on the second image, it can be determined that the first target vehicle has violated traffic rules, and the first target vehicle is the violating vehicle. This facilitates the subsequent call of the first image to re-identify the first target vehicle, determine the identification information of the first target vehicle, and improve the accuracy of identifying the violating vehicle.

[0014] In a possible implementation of the first aspect, a shooting focal length of the second image is smaller than a shooting focal length of the first image.

[0015] It can be understood that since a larger area needs to be captured to determine a traffic violation, a first camera with a larger shooting angle captures multiple frames of continuous first lane images to determine the vehicle's traffic violation, thereby triggering tracking of the violating vehicle.

[0016] For example, the appearance features of the first target vehicle, such as the color and model of the vehicle, can be identified from the second image (i.e., the first lane image) captured by the first camera. Then, a second camera with a smaller shooting angle is called to capture multiple frames of the second lane image. The second camera has a clearer image and can identify the identification information of the illegal vehicle, such as the license plate number, the vehicle unique identification code, etc.

[0017] It can be understood that by coordinating the second image with different focal lengths with the first image, it is easier to cover the entire road with the shooting range, which can not only avoid missing the identification of traffic violations, but also avoid missing the identification information of the illegal vehicles.

[0018] In a possible implementation of the first aspect above, the second image is captured by a first camera set at a first position, and the first image is captured by a second camera set at a second position, and the first shooting area of ​​the first camera overlaps with the second shooting area of ​​the second camera; and the first position and the second position have a first corresponding relationship, wherein the first corresponding relationship includes: the vertical distances between the first position and the second position and the first horizontal plane are equal; or, the vertical distance between the first position and the first horizontal plane is greater than the vertical distance between the second position and the first horizontal plane.

[0019] It can be understood that the first camera has a larger shooting angle and a more blurred image, while the second camera has a smaller shooting angle and a clearer image. Therefore, the first camera and the second camera can be set on the same horizontal plane, or the first camera can be set higher than the second camera, that is, the vertical distance between the first position and the first horizontal plane is greater than the vertical distance between the second position and the first horizontal plane. In this way, in the environment where goods are stacked and stored in the port yard, the obstruction of the stacking of goods on the monitoring shooting can be avoided.

[0020] For example, in a container yard, the stacking height can reach 8 layers. Based on the height of 2.6 meters corresponding to a standard 12-foot container, the total height of the 8-layer stacking is about 21 meters. At this time, the first camera can be deployed at a position more than 30 meters above the horizontal plane of the road, thereby preventing the cargo from blocking the first camera from shooting the road. When the second camera is set at a lower position of the pole, it can still shoot a small range of the road at a close distance. Therefore, the second camera will not be blocked by the cargo when shooting the vehicle identification information.

[0021] In a possible implementation of the first aspect, the first corresponding relationship further includes: the first position and the second position are located in the same fixing device; or the first position and the second position are located in different fixing devices.

[0022] It is understood that the first camera and the second camera can be fixed on the same immovable device, such as a single pole fixed on the ground. The first camera and the second camera can also be fixed on two immovable devices, such as two different poles fixed on the ground. As long as the first camera and the second camera have overlapping shooting areas, it can be ensured that the illegal vehicle captured by the first camera can be re-identified in the second road image captured by the second camera, thereby avoiding misidentification of the illegal vehicle.

[0023] In a possible implementation of the first aspect, the first corresponding relationship further includes: the first position and the second position are located on the same mobile working device.

[0024] It is understandable that the first camera and the second camera can be set on the same movable device, for example, they can be set in pairs on each bridge crane in the port yard. Since the first camera and the second camera have at least a partial overlap in the shooting area, during the movement of the device, although the overlapping area of ​​the two cameras is constantly moving, the same illegal vehicle can still be identified.

[0025] In some embodiments, the first camera and the second camera may be arranged in pairs on different movable devices and fixed devices, thereby reducing the number of fixed devices and avoiding the increase in construction costs due to the addition of new fixed devices.

[0026] In a possible implementation of the first aspect, a third camera having a positional relationship with the first camera is located at a third position, and the first camera and the second camera have a first shooting overlapping area, and the first camera and the third camera have a second shooting overlapping area.

[0027] It can be understood that a camera with a large shooting angle area and a camera with a small shooting angle area can be arranged in pairs, so the camera with a large shooting angle area and the camera with a small shooting angle area can be regarded as a camera combination. The first camera with a large shooting angle area and the second camera with a small shooting angle area are the first camera combination, and the camera combination adjacent to the first camera combination is the second camera combination, and the camera with a large shooting angle area in the second camera combination is the third camera (i.e., the first camera in the second camera combination hereinafter). The third camera and the first camera have a second shooting overlapping area, so that the first camera in the multiple camera combinations can monitor all roads to avoid missed recognition of traffic violations.

[0028] In a possible implementation of the first aspect, the acquiring of the first image includes: determining that the first target vehicle enters a first shooting overlapping area according to the plurality of second images; and acquiring a plurality of first images captured by a second camera.

[0029] It can be understood that after determining that the first target vehicle enters the first shooting overlapping area according to the multiple second images, the multiple first images shot by the second camera are triggered to be retrieved. Then, after identifying the violation event, the first image (i.e., the second road image) shot by the second camera is triggered to be acquired only when the first target vehicle enters the first shooting overlapping area, thereby reducing the occupation of system resources.

[0030] In a possible implementation of the first aspect above, obtaining multiple first images taken by the second camera also includes: determining that the first target vehicle enters the first shooting overlapping area along the first driving direction, and obtaining multiple first images taken by the second camera in real time, wherein the first driving direction is used to indicate that the first target vehicle drives from the first shooting area of ​​the first camera to the second shooting area of ​​the second camera; determining that the first target vehicle enters the first shooting overlapping area along the second driving direction, and obtaining multiple first images taken by the second camera historically, wherein the second driving direction is used to indicate that the first target vehicle drives from the second shooting area of ​​the second camera to the first shooting area of ​​the first camera.

[0031] It is understandable that, since the first camera can be used to identify the violation event, the second camera can be used to identify the identification information of the violation vehicle. Therefore, the driving direction of the first target vehicle can be divided into a first driving direction (i.e., forward driving) and a second driving direction (i.e., reverse driving).

[0032] If it is traveling in the forward direction, that is, the first target vehicle drives from the first shooting area of ​​the first camera to the second shooting area of ​​the second camera, the illegal vehicle captured by the second camera can be identified based on the appearance and driving behavior of the first target vehicle captured by the first camera, and then the identification information of the first target vehicle can be determined.

[0033] If the vehicle is traveling in the opposite direction, that is, the first target vehicle is traveling from the second shooting area of ​​the second camera to the first shooting area of ​​the first camera, the second road image (i.e., the first image) captured by the second camera can be retrieved retroactively based on the illegal vehicle captured by the first camera. The identification information of the first target vehicle can be determined through the multiple first images captured by the second camera in history.

[0034] In a possible implementation of the first aspect, a fourth camera that has a positional relationship with the third camera is located at a fourth position, and the third camera and the fourth camera have a third overlapping shooting area.

[0035] It can be understood that the camera with a large shooting angle area and the camera with a small shooting angle area can be arranged in pairs, so the camera with a large shooting angle area and the camera with a small shooting angle area can be regarded as a camera combination. The first camera with a large shooting angle area and the second camera with a small shooting angle area are the first camera combination, and the second camera combination adjacent to the first camera combination, the camera with a large shooting angle area is the third camera (i.e., the first camera in the second camera combination hereinafter), and the camera with a small shooting angle area is the fourth camera (i.e., the second camera in the second camera combination hereinafter).

[0036] It can be understood that in a single camera combination, the shooting area of ​​the camera with a large shooting angle area overlaps at least partially with the shooting area of ​​the camera with a small shooting angle area, so that the shooting image of the camera with a small shooting angle area can be retrieved to re-identify the same illegal vehicle. Therefore, the third camera and the fourth camera have a third shooting overlapping area.

[0037] In a possible implementation of the first aspect above, the method also includes: determining that a traffic violation has occurred for the second target vehicle in the third image based on the multiple third images taken by the first camera; determining that the second target vehicle has entered the second shooting overlapping area based on the multiple third images taken by the first camera; acquiring multiple fourth images taken by the third camera; identifying the second target vehicle in the fourth image based on the second feature of the second target vehicle in the third image; determining that the second target vehicle has entered the third shooting overlapping area based on the fourth image; acquiring multiple fifth images taken by the fourth camera; identifying the identification information of the second target vehicle in the fifth image based on the second feature of the second target vehicle in the third image, wherein the second feature is used to characterize the appearance and driving behavior of the second target vehicle.

[0038] It can be understood that since the third camera can be used to identify the violation event, the fourth camera can be used to identify the identification information of the violation vehicle. Therefore, when the violation vehicle drives from the shooting area of ​​the first camera combination to the shooting area of ​​the adjacent second camera combination, and the violation vehicle does not pass through the shooting area of ​​the second camera in the first camera combination, the driving direction of the violation vehicle can be defined as its direction.

[0039] For example, if the second target vehicle (i.e., the illegal vehicle) enters the above-mentioned second shooting overlapping area (i.e., the shooting overlapping area of ​​the two first cameras in the two adjacent camera combinations) in the multiple third images (i.e., the third road images) taken by the first camera, then the multiple fourth images taken by the third camera (i.e., the multiple first road images taken by the first camera in the second camera combination) are retrieved. After determining that the second target vehicle enters the third shooting overlapping area (i.e., the shooting overlapping area of ​​the first camera and the second camera in the second camera combination hereinafter), the retrieval of the multiple fifth images taken by the fourth camera (i.e., the multiple fourth road images taken by the second camera in the second camera combination) is triggered. At this time, the second target vehicle can be re-identified in the fourth image taken by the third camera based on the appearance and driving behavior of the second target vehicle identified by the illegal event, and then the second target vehicle can be re-identified again in the fifth image taken by the fourth camera.

[0040] Therefore, after a traffic violation is identified, multiple cameras are triggered to track and identify the illegal vehicle until it passes a camera with a smaller shooting angle and identifies the identification information of the illegal vehicle. The features of the illegal vehicle are tracked and transmitted between multiple cameras, so that the illegal vehicle can be tracked and identified, and the same illegal vehicle can be identified.

[0041] In some implementations, the complete violation event can be obtained by tracking and identifying the illegal vehicle, and the video stream data with its identification information can be identified, thereby facilitating the proof of the illegal vehicle and the reporting of the violation event.

[0042] In a possible implementation of the first aspect above, identification information of the first target vehicle in the first image is identified based on a first feature of the first target vehicle in the second image, including: determining a first vehicle to be compared in the first image; identifying the first target vehicle from the first vehicles to be compared based on the first feature of the first target vehicle in the second image; and determining the identification information of the first target vehicle.

[0043] In some embodiments, all vehicles in multiple first images (i.e., multiple frames of second road images) can be used as first vehicles to be compared. Then, based on the first features of the first target vehicle in the second image, such as appearance features and driving behavior features, the first target vehicle is identified from the first vehicles to be compared, and then the identification information of the first target vehicle, such as the license plate number, is identified. Thus, the features of the illegal vehicle are identified based on the image corresponding to the violation event, and then the illegal vehicle is re-identified in multiple first images based on the features of the illegal vehicle, and then the identification information of the illegal vehicle is identified based on the clear first image, thereby avoiding misidentification of the illegal vehicle.

[0044] In a possible implementation of the first aspect, the second target vehicle in the fourth image is identified based on the second feature of the second target vehicle in the third image, including: determining the second vehicle to be compared in the fourth image; and identifying the second target vehicle from the second vehicle to be compared based on the second feature.

[0045] It can be understood that all vehicles in the multiple fourth images taken by the third camera (i.e., the multiple frames of third road images taken by the first camera in the second camera combination) can be used as the second vehicles to be compared. Furthermore, based on the second features of the second target vehicle in the fourth image, such as appearance features and driving behavior features, the second target vehicle is identified from the second vehicles to be compared, and continuous tracking and identification of the second vehicles to be compared is achieved. Thus, the features of the illegal vehicle are identified based on the image corresponding to the violation event, and the illegal vehicle is re-identified in the image taken by the camera with overlapping shooting areas based on the features of the illegal vehicle, thereby avoiding misidentification of the illegal vehicle.

[0046] In a possible implementation of the first aspect above, according to the second feature of the second target vehicle in the fifth image, the identification information of the second target vehicle in the fifth image is identified, including: determining the third vehicle to be compared in the fifth image; identifying the second target vehicle from the third vehicle to be compared according to the second feature; and determining the identification information of the second target vehicle.

[0047] It can be understood that all vehicles in the multiple fifth images taken by the fourth camera (i.e., the multiple frames of fourth road images taken by the second camera in the second camera combination) can be used as the third vehicles to be compared. Furthermore, based on the second features of the second target vehicle in the fourth image, such as appearance features and driving behavior features, the second target vehicle is identified from the third vehicles to be compared, and then the identification information of the second target vehicle, such as the license plate number, is identified. Thus, the features of the illegal vehicle are identified based on the image corresponding to the violation event, and the illegal vehicle is re-identified in the image taken by the camera with overlapping shooting areas based on the features of the illegal vehicle, thereby avoiding misidentification of the illegal vehicle.

[0048] In a possible implementation of the first aspect above, a method for acquiring a first feature of a first target vehicle in a second image includes: determining a first appearance parameter and a first driving behavior parameter of the first target vehicle based on the multiple second images, wherein the first appearance parameter includes a color of the first target vehicle and a model of the first target vehicle, and the first driving behavior parameter includes a lane change information sequence of the first target vehicle and a speed of the first target vehicle; and clustering the first appearance parameter and the first driving behavior parameter into the first feature of the first target vehicle.

[0049] It can be understood that based on the second image, multiple parameters for characterizing vehicle characteristics of the first target vehicle, such as a first appearance parameter and a first driving behavior parameter, can be identified, and then multiple feature parameters can be clustered in a high-dimensional space to obtain clustered features. For example, the first feature of the first target vehicle (i.e., the reference feature below) can be clustered in the high-dimensional space to facilitate identification of the first target vehicle among the vehicles to be compared.

[0050] In a possible implementation of the first aspect, the method for acquiring the lane change information sequence of the first target vehicle includes: recording the lanes occupied by the first target vehicle in time sequence according to the multiple second images to obtain the lane change information sequence of the first target vehicle.

[0051] That is, the continuous multiple frames of the first lane image taken by the first camera can identify the lane information of the vehicle. Therefore, the electronic device can determine the continuous lane change information of the target vehicle corresponding to the violation event in the multiple frames of the first lane image, and then can record its lane position according to the lane change sequence of the vehicle to obtain the lane change information sequence of the first target vehicle.

[0052] It should be noted that the lane change information sequence only records the lane information of the vehicle in sequence according to the lane change order, thereby representing the lane change information of the vehicle during the time period corresponding to the multiple frames of the first lane image.

[0053] It is understandable that, since the shooting frequencies of the first camera and the second camera may be different, it is possible that when the first camera captures a single frame of the first lane image, the second camera does not capture a new image at the same time, making it difficult for the two cameras to accurately align their time. Here, a lane change information sequence that does not record specific time information is proposed. It is not necessary to record the specific moment when the vehicle enters the new lane. It is only necessary to record the lane information in which the vehicle is located in sequence during the captured time period. It can also effectively characterize the driving characteristics of the illegal vehicles in the overlapping shooting area, making it easier to re-identify the first target vehicle in the second lane image.

[0054] When the lane change information sequence is combined with the vehicle speed information, the vehicle's driving behavior in time and space dimensions is more recognizable, making it easy to identify the same illegal vehicle.

[0055] In a possible implementation of the first aspect above, a method for acquiring the second feature of the second target vehicle in the third image includes: determining second appearance parameters and second driving behavior parameters of the second target vehicle based on multiple third images, wherein the second appearance parameters include the color of the second target vehicle and the model of the second target vehicle, and the second driving behavior parameters include the lane change information sequence of the second target vehicle and the speed of the second target vehicle; and clustering the second appearance parameters and the second driving behavior parameters into the second feature of the second target vehicle.

[0056] It can be understood that based on the third image, multiple parameters of the second target vehicle used to characterize vehicle characteristics, such as the second appearance parameter and the second driving behavior parameter, can be identified, and then the multiple feature parameters can be clustered in the high-dimensional space to obtain cluster features. For example, the second feature of the second target vehicle (i.e., the reference feature below) can be clustered in the high-dimensional space to facilitate identification of the second target vehicle among the vehicles to be compared.

[0057] In a possible implementation of the first aspect, the method for acquiring the lane change information sequence of the second target vehicle includes: recording the lanes occupied by the second target vehicle in time sequence according to the multiple third images to obtain the lane change information sequence of the second target vehicle.

[0058] That is, the continuous multiple frames of the first lane image (i.e., the third image) taken by the first camera can identify the lane information of the second target vehicle. Therefore, the electronic device can determine the continuous lane change information of the second target vehicle corresponding to the violation event in the multiple frames of the first lane image, and then can record its lane position according to the lane change sequence of the vehicle to obtain the lane change information sequence of the second target vehicle.

[0059] It should be noted that the lane change information sequence only records the lane information of the vehicle in sequence according to the lane change order, and then represents the lane change information of the vehicle in the time period corresponding to the multiple frames of the first lane image. When the lane change information sequence is combined with the vehicle speed information, the driving behavior of the vehicle in the time and space dimensions has greater identifiability, and it is easy to identify the same illegal vehicle.

[0060] In a possible implementation of the first aspect, obtaining multiple first images captured by the first camera includes: performing image registration on the multiple first images according to an image correspondence between the second image and the first image to obtain multiple first images after image registration.

[0061] That is, after performing image registration on multiple first images, multiple first images with the same viewing angle as the second image can be obtained, thereby facilitating image stitching of the first image and the second image, so as to obtain a complete traffic violation evidence video.

[0062] In a possible implementation of the first aspect, a method for acquiring the image correspondence between the second image and the first image includes: determining the first background image based on the multiple second images, wherein the first background image does not contain dynamic information; determining the second background image based on the multiple first images, wherein the second background image does not contain dynamic information; determining the image correspondence between the second image and the first image based on the first background image and the second background image, wherein the image correspondence includes a perspective mapping coefficient.

[0063] It can be understood that the image correspondence between the second image (i.e., the first lane image) and the first image (i.e., the second lane image) can be a perspective correspondence. Therefore, the above image correspondence can be characterized by perspective conversion parameters. After removing the dynamic information, multiple first background images and multiple second background images are obtained, which facilitates the determination of perspective mapping coefficients.

[0064] For example, the electronic device can perform a series of image processing such as rotation, translation, and scaling on the first lane image, so that the first lane image can have the same shooting angle as the second lane image. This facilitates the determination of the features of each vehicle to be compared in the second lane image, such as driving behavior features such as lane change information, driving direction and speed, and appearance information such as vehicle model and color, so as to improve the feature comparison accuracy with the target vehicle violating the traffic rules in the first lane image.

[0065] In a possible implementation of the first aspect, the step of acquiring the multiple first images taken by the second camera further includes: obtaining multiple first images that are time-aligned with each second image based on a time correspondence between the second images and the first images.

[0066] It is understandable that due to hardware differences, shooting frequency errors, transmission link delay errors and other reasons between cameras, the actual shooting time of the first lane image and the second lane image with the same timestamp does not correspond, making it difficult to ensure that the relative position information of the compared vehicles in the overlapping shooting area is consistent.

[0067] Therefore, according to the time correspondence between the second image and the first image, multiple first images that are time-aligned with each second image can be obtained, thereby improving the accuracy of vehicle re-identification between different cameras.

[0068] In a possible implementation of the first aspect, the time correspondence between the second image and the first image includes a first imaging error time between the second image and the first image, and a method for acquiring the first imaging error time includes: determining the first imaging error time between the first camera and the second camera based on an imaging timestamp of the first camera and an imaging timestamp of the second camera.

[0069] That is, the electronic device can pre-store the preset imaging error time between the first camera and the second camera, for example, the preset imaging error time can be written into the configuration file as a configuration parameter, so that a single-frame first lane image and multiple frames of second lane images within the imaging error time can be used for vehicle comparison, thereby ensuring that there is a vehicle to be compared in the second lane image that is most similar to the target vehicle in the first lane image in relative position.

[0070] In a possible implementation of the first aspect above, it also includes: stitching the above-mentioned multiple second images and the multiple first images after the above-mentioned image registration to obtain multiple first stitched images; stitching the video stream captured by the first camera and the video stream captured by the second camera according to the above-mentioned multiple first stitched images to obtain the violation video of the first target vehicle.

[0071] That is, by splicing multiple frames of first lane images where the violation was captured but the vehicle logo is difficult to identify into multiple frames of second lane images where the violation was difficult to capture but the vehicle logo can be accurately identified, a complete violation video stream can be obtained for reporting the violation, which can make the evidence of the violation more effective.

[0072] In a possible implementation of the first aspect, obtaining multiple fourth images taken by a third camera includes: performing image registration on the multiple fourth images according to an image correspondence between the third image and the fourth image to obtain multiple fourth images after image registration.

[0073] That is, after performing image registration on the plurality of fourth images, a plurality of fourth images with the same viewing angle as the third image can be obtained, thereby facilitating image stitching of the third image and the fourth image, so as to obtain a complete video of traffic violation evidence.

[0074] In a possible implementation of the first aspect, obtaining multiple fifth images taken by a fourth camera includes: performing image registration on the multiple fifth images according to an image correspondence between the fourth image and the fifth image to obtain multiple fifth images after image registration.

[0075] That is, after performing image registration on multiple fifth images, multiple fifth images with the same viewing angle as the fourth image can be obtained, so as to facilitate image stitching of the fourth image and the fifth image, so as to obtain a complete traffic violation evidence video.

[0076] In a possible implementation of the first aspect above, it also includes: stitching the above-mentioned multiple third images, the above-mentioned multiple fourth images after image registration, and the above-mentioned multiple fifth images after image registration to obtain multiple second stitched images; stitching the video stream captured by the first camera, the video stream captured by the third camera, and the video stream captured by the fourth camera according to the above-mentioned multiple second stitched images to obtain the violation video of the second target vehicle.

[0077] That is, by splicing multiple third images and multiple fourth images that capture the violation but the vehicle logo is difficult to identify, and then splicing them into multiple fifth images that are difficult to capture but the vehicle logo can be accurately identified, a complete violation video stream can be obtained for reporting the violation, which can make the evidence of the violation more effective.

[0078] In a possible implementation of the first aspect above, acquiring multiple fourth images taken by a third camera further includes: obtaining multiple fourth images that are time-aligned with each third image according to a time correspondence between the third image and the fourth image.

[0079] It is understandable that due to hardware differences, shooting frequency errors, transmission link delay errors and other reasons between cameras, the actual shooting times of the third image and the fourth image with the same timestamp do not correspond, making it difficult to ensure that the relative position information of the compared vehicles in the overlapping shooting areas is consistent.

[0080] Therefore, according to the time correspondence between the third image and the fourth image, multiple fourth images that are time-aligned with each third image can be obtained, thereby improving the accuracy of vehicle re-identification between different cameras.

[0081] In a possible implementation of the first aspect, the time correspondence between the third image and the fourth image includes a second imaging error time between the third image and the fourth image, and a method for acquiring the second imaging error time includes: determining the imaging error time between the first camera and the third camera based on an imaging timestamp of the first camera and an imaging timestamp of the third camera.

[0082] That is, the electronic device can pre-store the preset imaging error time between the first camera and the third camera, for example, the preset imaging error time can be written into the configuration file as a configuration parameter, so that the single-frame third image and the multiple-frame fourth image within the imaging error time are compared for the vehicle, thereby ensuring that the fourth image contains a vehicle to be compared that is most similar to the target vehicle in the third image in relative position.

[0083] In a possible implementation of the first aspect, the step of acquiring the multiple fifth images taken by the fourth camera further includes: obtaining multiple fifth images that are time-aligned with each fourth image based on a time correspondence between the fourth image and the fifth image.

[0084] It is understandable that due to hardware differences, shooting frequency errors, transmission link delay errors and other reasons between cameras, the actual shooting times of the fourth image and the fifth image with the same timestamp do not correspond, making it difficult to ensure that the relative position information of the compared vehicles in the overlapping shooting areas is consistent.

[0085] Therefore, according to the time correspondence between the fourth image and the fifth image, multiple fifth images that are time-aligned with each fourth image can be obtained, thereby improving the accuracy of vehicle re-identification between different cameras.

[0086] In a possible implementation of the first aspect, the time correspondence between the fourth image and the fifth image includes a third imaging error time between the fourth image and the fifth image, and a method for acquiring the third imaging error time includes: determining the imaging error time between the third camera and the fourth camera according to an imaging timestamp of the third camera and an imaging timestamp of the fourth camera.

[0087] That is, the electronic device can pre-store the preset imaging error time between the third camera and the fourth camera, for example, the preset imaging error time can be written into the configuration file as a configuration parameter, so that the single-frame fourth image and the multiple-frame fifth image within the imaging error time are compared for the vehicle, thereby ensuring that the fifth image contains a vehicle to be compared that is most similar to the target vehicle in the fourth image in relative position.

[0088] In a possible implementation of the first aspect, the first device also includes a first radar, and the determining that the first target vehicle has a traffic violation through the first detection data of the first device includes: determining that the first target vehicle has a traffic violation through the first point cloud data detected by the first radar.

[0089] That is, the first camera in the above solution can be replaced by a radar. For example, the point cloud data can be obtained by the radar to determine the violation of the vehicle, thereby triggering the tracking of the illegal vehicle.

[0090] In some embodiments, the radar may be a millimeter wave radar or a laser radar. The advantage of radar is that it has strong penetration of rain and fog. Therefore, using radar to replace the first camera can prevent bad weather from affecting the identification of traffic violation events.

[0091] In a possible implementation of the first aspect above, the first point cloud data is detected by a first radar set at a fifth position, the first image is captured by a second camera set at a second position, and the first detection area of ​​the first radar overlaps with the second shooting area of ​​the second camera; and the fifth position and the second position have a second corresponding relationship, wherein the second corresponding relationship includes: the vertical distances between the fifth position and the second position and the second horizontal plane are equal; or, the vertical distance between the fifth position and the second horizontal plane is greater than the vertical distance between the second position and the second horizontal plane.

[0092] That is, the first camera in the above solution can be replaced by a radar, and the radar can be set at the same height as the second camera, or at a higher height than the second camera.

[0093] In the second aspect, an embodiment of the present application also provides a vehicle identification system, the system comprising a first device and a second camera, wherein the first device comprises a first camera, and the first camera is arranged at a first position, the second camera is arranged at a second position, and a first shooting area of ​​the first camera overlaps with a second shooting area of ​​the second camera; and the first position and the second position have a first corresponding relationship, wherein the first corresponding relationship comprises: the vertical distances between the first position and the second position and a first horizontal plane are equal; or, the vertical distance between the first position and the first horizontal plane is greater than the vertical distance between the second position and the first horizontal plane.

[0094] It can be understood that the first camera has a larger shooting angle and a more blurred image, while the second camera has a smaller shooting angle and a clearer image. Therefore, the first camera and the second camera can be set on the same horizontal plane, or the first camera can be set higher than the second camera, that is, the vertical distance between the first position and the first horizontal plane is greater than the vertical distance between the second position and the first horizontal plane. In this way, in the environment where goods are stacked and stored in the port yard, the obstruction of the stacking of goods on the monitoring shooting can be avoided.

[0095] In a possible implementation of the second aspect above, the system also includes a third camera and a fourth camera, wherein the third camera is set at a third position, the fourth camera is set at a fourth position, the third shooting area of ​​the third camera overlaps with the fourth shooting area of ​​the fourth camera; and the third position and the fourth position have a second corresponding relationship, wherein the second corresponding relationship includes: the vertical distances between the third position and the fourth position and the second horizontal plane are equal; or, the vertical distance between the third position and the second horizontal plane is greater than the vertical distance between the fourth position and the second horizontal plane.

[0096] It can be understood that the third camera has a larger shooting angle and a more blurred image, while the fourth camera has a smaller shooting angle and a clearer image. Therefore, the third camera and the fourth camera can be set on the same horizontal plane, or the third camera can be set higher than the fourth camera, that is, the vertical distance between the third position and the second horizontal plane is greater than the vertical distance between the fourth position and the second horizontal plane. In this way, in the environment where goods are stacked and stored in the port yard, the obstruction of the monitoring shooting by the stacking of goods can be avoided.

[0097] In a possible implementation of the second aspect, the third shooting area of ​​the third camera overlaps with the first shooting area of ​​the first camera.

[0098] That is, the third camera and the first camera have overlapping shooting areas, so that the first camera in the combination of multiple cameras can monitor all roads, thereby avoiding missed recognition of traffic violation events.

[0099] In a possible implementation of the second aspect, the third shooting area and the fourth shooting area overlap to the first shooting area along a first direction; or, the third shooting area and the fourth shooting area overlap to the first shooting area along a second direction different from the first direction.

[0100] It can be understood that the camera with a large shooting angle area and the camera with a small shooting angle area can be arranged in pairs, so the camera with a large shooting angle area and the camera with a small shooting angle area can be regarded as a camera combination. The first camera with a large shooting angle area and the second camera with a small shooting angle area are the first camera combination, and the second camera combination adjacent to the first camera combination, the camera with a large shooting angle area is the third camera (i.e., the first camera in the second camera combination hereinafter), and the camera with a small shooting angle area is the fourth camera (i.e., the second camera in the second camera combination hereinafter).

[0101] In a possible implementation of the second aspect, the first device also includes a first radar, wherein the first device includes the first radar, and the first radar is set at a fifth position, the second camera is set at a second position, and the first detection area of ​​the first radar overlaps with the second shooting area of ​​the second camera; and the fifth position and the second position have a second corresponding relationship, wherein the second corresponding relationship includes: the vertical distances between the fifth position and the second position and the second horizontal plane are equal; or, the vertical distance between the fifth position and the second horizontal plane is greater than the vertical distance between the second position and the second horizontal plane.

[0102] That is, the first camera in the above solution can be replaced by a radar. For example, the point cloud data can be obtained by the radar to determine the violation of the vehicle, thereby triggering the tracking of the illegal vehicle.

[0103] In some embodiments, the radar may be a millimeter wave radar or a laser radar. The advantage of radar is that it has strong penetration of rain and fog. Therefore, using radar to replace the first camera can prevent bad weather from affecting the identification of traffic violation events.

[0104] In the third aspect, an embodiment of the present application also provides an electronic device, comprising: one or more processors; one or more memories; the one or more memories store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device executes the vehicle identification method provided by the first aspect and various possible implementations.

[0105] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, characterized in that instructions are stored on the storage medium, and when the instructions are executed on a computer, the computer executes the vehicle identification method provided by the first aspect and various possible implementations.

[0106] In a fifth aspect, an embodiment of the present application further provides a computer program product, characterized in that it includes a computer program / instruction, which, when executed by a processor, implements the vehicle identification method provided by the above-mentioned first aspect and various possible implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0107] Figure 1A A schematic diagram of the principle of vehicle identification is shown;

[0108] Figure 1B Another schematic diagram of the principle of vehicle identification is shown;

[0109] Figure 2A According to some embodiments of the present application, a schematic diagram of a vehicle identification scenario is shown;

[0110] Figure 2B According to some embodiments of the present application, another schematic diagram of a vehicle identification scenario is shown;

[0111] Figure 3 According to some embodiments of the present application, a schematic flow chart of a vehicle identification method is shown;

[0112] Figure 4 According to some embodiments of the present application, a schematic diagram of a scenario for identifying a traffic violation vehicle based on a marking line is shown;

[0113] Figure 5A A schematic diagram of a vehicle recognition scenario with no overlapping shooting areas is shown;

[0114] Figure 5B According to an embodiment of the present application, a schematic diagram of an implementation process of determining a target vehicle based on multiple frames of second images is shown;

[0115] Figure 6 According to some embodiments of the present application, a schematic diagram of a scene of image registration by target feature recognition is shown;

[0116] Figure 7 According to an embodiment of the present application, a schematic diagram of an implementation process of determining a target vehicle based on a multi-frame spliced ​​image is shown;

[0117] Figure 8 According to some embodiments of the present application, a schematic diagram of a vehicle lane change scenario is shown;

[0118] Fig. 9 According to some embodiments of the present application, a schematic diagram of a process based on a traffic violation vehicle and a vehicle to be compared is shown;

[0119] Fig.10 According to an embodiment of the present application, a schematic diagram of an implementation process of determining a vehicle to be compared based on a timestamp is shown;

[0120] Fig.11A According to some embodiments of the present application, a schematic flow chart of a vehicle identification method is shown;

[0121] Fig. 11B According to some embodiments of the present application, a schematic diagram of a scenario for identifying a vehicle according to its driving direction is shown;

[0122] Fig.12 According to some embodiments of the present application, a specific data processing flow for identifying a vehicle according to its driving direction is shown;

[0123] Fig.13 According to some embodiments of the present application, a schematic diagram of a deployment scenario of a camera combination is shown;

[0124] Fig.14Ais a schematic structural block diagram of an electronic device provided according to an embodiment of the present application;

[0125] Fig. 14B is a schematic structural block diagram of a processing module in an electronic device provided according to an embodiment of the present application;

[0126] Fig.15 is a schematic structural block diagram of a camera provided according to an embodiment of the present application;

[0127] Fig.16 is a hardware structure block diagram of an electronic device provided according to an embodiment of the present application;

[0128] Fig.17 It is a schematic hardware structure block diagram of a camera provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0129] The illustrative embodiments of the present application include, but are not limited to, a vehicle identification method, an electronic device, and a computer-readable storage medium.

[0130] In order to facilitate those skilled in the art to understand the solutions in the embodiments of the present application, some concepts and terms involved in the embodiments of the present application are explained below.

[0131] (1) Traffic gantry: A door frame-like frame used for road traffic, including crossbeams, columns, chassis, guide beams and fulcrums and corresponding traffic facilities, such as road signs, road restriction signs, surveillance cameras and LED induction screens.

[0132] (2) Image registration: It is the process of matching and superimposing two or more images acquired at different times, using different sensors or under different conditions. For example, two or more images can be matched and superimposed using imaging devices under different weather, illumination, camera positions and angles to obtain the target image. If different imaging devices acquire multiple frames of continuous images under different conditions, multiple frames of continuous images can be matched and superimposed through image registration to obtain a complete continuous video stream. Therefore, image registration can be widely used in remote sensing data analysis, computer vision, image processing and other fields.

[0133] (3) Time registration: synchronize the asynchronous measurement information of various sensors on the same target to the same time to achieve information data synchronization. In some embodiments, the above-mentioned sensor can also be an image acquisition device, such as a camera, so that the images captured by different cameras on the same target are synchronized to the same shooting time.

[0134] (4) Traffic signs: The traffic signs referred to in the embodiments of the present application may include traffic lights, traffic signs and traffic markings. Traffic lights may include motor vehicle lights, non-motor vehicle lights, pedestrian crossing lights, direction indicator lights, etc.

[0135] Traffic signs (also called road traffic signs or signs) are facilities used to manage traffic by conveying information to road users in the form of colors, shapes, characters, graphics, etc. Traffic signs are usually set on the right side of the road or above the lane in the form of traffic signs; they can also be set on the left side of the road or on both sides according to the specific situation.

[0136] Traffic signs are classified into two categories according to their functions: primary signs and auxiliary signs. Primary signs mainly include warning signs, prohibition signs, instruction signs, and direction signs. Auxiliary signs are signs attached to primary signs to provide auxiliary explanations for primary signs. Common traffic signs include speed limit signs, no parking signs, no U-turn signs, etc.

[0137] Traffic markings, also known as road traffic markings or markings, are traffic settings consisting of various lines, arrows, words, patterns, facade marks, physical marks, raised road signs and contour marks drawn or installed on the road. The function of traffic markings is to convey information such as rules, warnings, and guidance about road traffic to road users. They can be used in conjunction with signs or alone.

[0138] Traffic markings can be divided into instruction markings, prohibition markings and warning markings according to their functions. Common traffic markings include white dotted lines, white solid lines, double yellow solid lines, pedestrian crossings, guide arrows, road text markings (such as road speed limit markings, etc.), road graphic markings (such as motor vehicle road markings, non-motor vehicle road markings, etc.), etc. The cameras referred to in the embodiments of the present application include all cameras with traffic violation recognition and shooting functions, including smart illegal parking capture balls, electronic police, checkpoint monitoring cameras, interval speed measurement cameras, etc. The camera can be a fixed camera (such as a camera fixed on a pole) or a movable camera. The camera can take continuous photos or videos and identify whether the photos or videos taken include traffic violations. The camera can monitor violations on one road or monitor violations on multiple roads at the same time.

[0139] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0140] Figure 1A A schematic diagram of the principle of vehicle identification is shown.

[0141] refer to Figure 1A , vehicle 00 travels on the road along a first direction X, a pole 10 is arranged on the side of the road, the pole 10 may be, for example, a lamp pole, a telephone pole or a gantry pole, etc., and a monitoring device, such as a camera 11, may be arranged. Camera 11 may be used to obtain video data on road area A. If vehicle 00 travels through road area A, the appearance feature data of vehicle 00 may be determined based on the video data, and whether vehicle 00 is in violation of traffic rules may be identified in combination with relevant traffic signs. For example, multiple frames of driving images of vehicle 00 in road area A may be obtained based on video data A1, and the signal light identification may be determined based on video data A1. Then, the violation event of vehicle 00, such as running a red light, may be identified based on the signal light identification, and the vehicle characteristics such as the vehicle speed, vehicle shape, and vehicle color may be determined based on the above multiple frames of driving images, and the illegal vehicle 00 may be identified from the monitoring video data.

[0142] Since there are many other traffic violations besides running a red light, such as speeding, driving against the flow and illegal parking, it is difficult to accurately identify speeding, driving against the flow and illegal parking only through the limited field of view of the camera 11. Therefore, in order to accurately identify traffic violations besides running a red light, a positioning device can be set on the vehicle.

[0143] Combine the following Figure 1B The above process involving the positioning device identifying the vehicle is described in detail.

[0144] Figure 1B Another schematic diagram of the principle of vehicle identification is shown.

[0145] refer to Figure 1B , a vehicle 01 is traveling on a road along a first direction X, a pole 12 is provided on the side of the road, and a traffic light 13 is provided on the pole 12. A positioning device 02 is provided on the vehicle 01, and the positioning device 02 can be used to obtain the position data and time data of the vehicle 01, so as to determine the driving parameters of the vehicle 01 in real time. For example, the positioning device 02 can determine the current position, driving speed, current lane and driving direction of the vehicle 01, and can realize synchronous reporting of the driving parameters and the vehicle identification.

[0146] It can be understood that based on the above driving parameters combined with the location information of relevant traffic signs, such as the location information of traffic lights, lane location information, etc., it is possible to accurately identify violations such as vehicles running red lights, speeding, driving in the wrong direction, and illegal parking. In addition, the positioning device 02 can also track and report the identification information of the illegal vehicle in real time, thereby accurately determining the illegal vehicle in the monitoring video.

[0147] However, since the installation and deployment of the positioning device in the car requires the cooperation of the car owner, it is difficult to widely promote this solution in vehicles. Therefore, how to improve the accuracy of identifying illegal vehicles without modifying the vehicle side has become an urgent problem to be solved.

[0148] In order to solve the problem that it is difficult to accurately identify illegal vehicles due to the lack of a positioning device, an embodiment of the present application provides a vehicle identification method, which uses a first camera with a larger shooting field angle to capture images of a larger area to confirm the vehicle's violation event, and then uses a second camera with a smaller shooting field angle to capture the vehicle's license plate and other identification information. At the same time, in order to ensure that the vehicle captured by the second camera is the illegal vehicle captured by the first camera, the shooting area of ​​the second camera is set to partially overlap with the shooting area of ​​the first camera.

[0149] Specifically, since it is necessary to shoot a larger area to determine the traffic violation, the first camera with a larger shooting angle shoots multiple frames of continuous first lane images to determine the vehicle's traffic violation, thereby triggering the tracking of the illegal vehicle. For example, the appearance features of the illegal vehicle, such as the color and model of the vehicle, are identified from the first lane image shot by the first camera. Then, the second camera with a smaller shooting angle is controlled to shoot multiple frames of the second lane image. The second camera has a clearer image and can identify the identification information of the illegal vehicle, such as the license plate number, the vehicle's unique identification code, etc. Moreover, the shooting area of ​​the second camera overlaps at least partially with the shooting area of ​​the first camera. By comparing the vehicle appearance features obtained by the above two cameras and the driving behavior features of the overlapping area, it is ensured that the vehicle in the second lane image is the same illegal vehicle as the vehicle in the first lane image, thereby accurately determining the traffic violation and the illegal vehicle.

[0150] In some embodiments, the first camera may be a short-focus camera, the second camera may be a long-focus camera, and the long-focus camera and the short-focus camera may have the same resolution.

[0151] It can be understood that the above short-focus camera has a larger shooting range and a more blurred image, and can be used to identify traffic violations, such as obtaining the vehicle's driving speed by shooting a continuous multi-frame image analysis to monitor speeding and illegal parking. The long-focus camera has a smaller shooting range and a clearer image, and can be used to identify text information and other identification information.

[0152] In some implementations, all telephoto cameras are used to monitor traffic violations. However, due to the small shooting range of telephoto cameras, it is easy to miss traffic violations. Therefore, a large number of telephoto cameras need to be densely arranged to cover all roads with the small shooting range of telephoto cameras, resulting in excessively high total cost of monitoring equipment.

[0153] It should be noted that the long focus and short focus in the embodiment of the present application are the relative focal lengths of the second camera with respect to the first camera, so that the shooting area of ​​the first camera is larger than the shooting area of ​​the second camera. The focal length parameters used by the above-mentioned cameras are not specifically limited here.

[0154] In some embodiments, a telephoto camera and a short-focus camera may be arranged in pairs at a pole, so that the shooting range of the multiple short-focus cameras covers the road, and each short-focus camera is matched with a telephoto camera whose shooting area at least partially overlaps, so that the entire road can be monitored with a smaller number of short-focus cameras and a smaller number of telephoto cameras. After the short-focus camera identifies a traffic violation, the identification information of the violating vehicle, such as the license plate number, can be identified by the paired telephoto cameras. This ensures that the violating vehicle identified by the short-focus camera is consistent with the violating vehicle in the traffic violation captured by the telephoto camera. In addition, without the need for a positioning device, the violating vehicle can be accurately identified from vehicles with similar appearances.

[0155] In some embodiments, the first camera and the second camera may be arranged at different positions at the same distance from the horizontal plane of the road, for example Figure 2A On the pole 210 shown, a first camera 211 and a second camera 212 with different shooting angles are arranged at the same distance from the horizontal plane where the road is located. By setting the first camera 211 as a short-focus camera and the second camera 212 as a long-focus camera, the shooting area B of the first camera 211 can be larger than the shooting area C of the second camera 212. In addition, it is also possible to achieve at least partial overlap between the shooting area B of the first camera 211 and the shooting area C of the second camera 212, for example Figure 2A Area D is shown.

[0156] In other embodiments, the first camera and the second camera are arranged at different positions at different distances from the horizontal plane where the road is located. For example, the first camera can be arranged at a higher position of the pole, and the second camera can be arranged at a lower position of the pole. In the environment where goods are stacked and stored in the port yard, the obstruction of the monitoring shooting by the stacking of goods can be avoided. For example, taking the container yard as an example, the stacking height can reach 8 layers. Calculated based on the height of 2.6 meters corresponding to the standard 12-foot container, the total height of the 8-layer stacking is about 21 meters. Therefore, the first camera can be deployed at a position more than 30 meters away from the horizontal plane where the road is located, thereby avoiding the goods blocking the first camera from shooting the road. When the second camera is arranged at a lower position of the pole, it can still shoot a small range of roads at a close distance. Therefore, the second camera will not be blocked by the goods when shooting the identification information of the vehicle.

[0157] Combine the following Figure 2BThe specific process of identifying illegal vehicles by two cameras set at different positions at different distances from the horizontal plane of the road as exemplified in the above embodiment is described in detail.

[0158] Figure 2B According to some embodiments of the present application, another schematic diagram of a vehicle identification scenario is shown.

[0159] like Figure 2B As shown, a pole 210 is arranged beside the road, on which a first camera 211 is arranged at position 1, and a second camera 212 is arranged at position 2. The shooting angle of the first camera 211 is angle J1, and the shooting area corresponding to the shooting angle J1 is road area B. The shooting angle of the second camera 212 is angle J2, and the shooting area corresponding to the shooting angle J2 is road area C. In some embodiments, the first camera 211 and the second camera 212 have different camera resolutions, and the resolution of the second camera 212 is greater than the resolution of the first camera 211. Therefore, along the Y direction perpendicular to the horizontal plane, the second camera 212 is closer to the road than the first camera 211, so that the second camera 212 can obtain a clearer image than the first camera 211, which is convenient for obtaining detailed features of the vehicle, such as the license plate number, etc. The first camera 211 can shoot a larger road area than the second camera 212, which can avoid missing the identification of vehicle violations.

[0160] It can be understood that the above positions 1 and 2 are both on the pole 210 for example only. In other embodiments, the position 1 where the first camera 211 is set and the position 2 where the second camera 212 is set can also be set on different devices / apparatuses respectively.

[0161] It can be understood that, since the shooting angles of the first camera 211 and the second camera 212 overlap at least partially, the road areas shot by them also overlap at least partially, for example Figure 2B The road area D shown may be captured by the first camera 211 and the second camera 212 at the same time.

[0162] It can be understood that the focal length of the first camera 211 may be shorter than that of the second camera 212 , so that the shooting area B of the first camera 211 is larger than the shooting area C of the second camera 212 .

[0163] In some embodiments, the vehicle 200 travels on the road along the first direction X. When the vehicle 200 travels into the road area B, the first camera 211 can capture the driving video of the vehicle 200, and send the continuous multiple-frame first lane images in the driving video of the vehicle 200 to the electronic device 230 through the network 220. The electronic device 230 determines the driving parameters of the vehicle and the traffic signs of the road according to the continuous multiple-frame first lane images of the vehicle 200, and identifies the violation of the vehicle 200 based on the driving parameters of the vehicle and the traffic signs.

[0164] Exemplarily, the above-mentioned continuous multiple frames of first lane images may include images from the first frame to the fifth frame, such as images A1, A2, A3, A4 and A5. The displacement distance of vehicle 200 is determined according to the position of vehicle 200 in the first frame image A1 and the position of vehicle 200 in the fifth frame image A5 in combination with map parameters, and then the real-time driving speed of vehicle 200 can be determined in combination with the length of the time interval corresponding to the shooting of the above five frames of images.

[0165] It can be understood that the above-mentioned violations may include running red lights, crossing the line, driving in the wrong direction, illegal lane changes, occupying non-motor vehicle lanes, illegal parking (hereinafter referred to as illegal parking), speeding and driving against the guidance (hereinafter referred to as driving in the wrong direction), etc.

[0166] After determining the violation, the electronic device 230 triggers the tracking and identification of the vehicle 200 according to the violation. For example, the image corresponding to the video shot by the first camera 211 can be continuously retrieved to identify the vehicle 200 corresponding to the violation in real time in the image, and to determine the appearance features of the vehicle 200, such as vehicle color, vehicle model, vehicle size, etc. When it is determined that the illegal vehicle 200 enters the overlapping shooting area D between the first camera 211 and the second camera 212, the driving video of the vehicle 200 shot by the second camera 212 is retrieved, and multiple frames of second lane images are obtained therefrom. Since the second lane image in the second video stream acquired by the second camera 212 is relatively clear, the electronic device 230 can recognize the identification information of the vehicle 200, such as the license plate information, based on the multiple frames of the second lane image.

[0167] It can be understood that by triggering the identification process of the violating vehicle through the violation event, the tracking process of the non-violating vehicle can be avoided, and the power consumption overhead of the electronic device 230 can be effectively reduced.

[0168] It can be understood that the electronic device 230 applicable to the above-mentioned vehicle identification method provided in the embodiment of the present application may include but is not limited to mobile phones, foldable screen mobile phones, tablet computers, desktops, laptops, handheld computers, netbooks, and augmented reality (AR) / virtual reality (VR) devices, smart TVs, smart watches and other electronic devices that obtain user operations through input devices. The above-mentioned input devices can be screens, keyboards, mice, etc., and no specific restrictions are made on the above-mentioned electronic devices.

[0169] In some embodiments, the first camera 211 and the second camera 212 may be disposed in the electronic device 230 and may be regarded as a single complete product.

[0170] It can be understood that the data interaction between the first camera 211 and the second camera 212 and the electronic device 230 can be wired or wireless, which is not limited here.

[0171] In order to better identify illegal vehicles, based on the above Figure 2B An example vehicle identification scenario is given below, in conjunction with relevant drawings, a detailed description of the method flow of vehicle identification in some embodiments of the present application is provided.

[0172] Example 1

[0173] Figure 3 According to some embodiments of the present application, a schematic flow chart of a vehicle identification method is shown. It can be understood that Figure 3 The execution entity of each step of the illustrated process may be the electronic device 230 described above, or other electronic devices, and the execution entity of a single step will not be described in detail.

[0174] like Figure 3 As shown, the process may include the following steps:

[0175] 301, obtaining a plurality of frames of continuous first lane images captured by a first camera.

[0176] Exemplarily, the electronic device 230 can obtain the first video stream shot by the first camera 211 through the electronic device 230, and the shooting range of the first video stream is Figure 2B Road area B in the example.

[0177] It can be understood that the first video stream may include multiple frames of continuous first lane images, which makes it easier for the electronic device 230 to determine the driving parameters and appearance features of all vehicles, so as to identify traffic violations in real time.

[0178] It can be understood that the first lane image in the example of the embodiment of the present application can be a photo taken by the first camera 211, or the first lane image can be a frame of image captured from the first video stream captured by the first camera 211. The first lane image can also be a two-dimensional or three-dimensional picture generated based on the first lane image captured by the first camera 211. For example, after acquiring the first lane image captured by the first camera 211, the number of lanes, traffic markings and other information included in the first lane image can be identified. Then, a two-dimensional or three-dimensional road image corresponding to the image is generated based on the identified number of lanes and traffic markings and other information. The road image generated in this way can exclude content in the image that is not related to traffic information, such as trees on both sides of the road, buildings on both sides of the road, etc.

[0179] 302 , determining a single vehicle driving parameter according to a plurality of frames of continuous first lane images and map data.

[0180] Exemplarily, the electronic device 230 can determine the driving parameters of a single vehicle based on the above-mentioned multiple frames of continuous images, for example, by determining the displacement of a single vehicle and the length of time taken by the vehicle to travel the above-mentioned displacement through multiple frames of continuous images combined with map data, and then determine the vehicle's driving speed and direction.

[0181] It can be understood that the above driving parameters include but are not limited to the driving speed, current lane and driving direction of each vehicle.

[0182] In some embodiments, the electronic device 230 can call the application programming interface (API) of the map application to obtain the map data of the current road. The map data can be a vector model electronic map or a raster model electronic map, which can include multiple layers. For example, layer 1 can include the most basic terrain shape data in the map and certain related additional information, such as road names, river names, etc.; and layer 2 can include road congestion status, etc. Among them, the environmental information can be determined by the terrain shape data in the layer, and then the environmental information in the map can be used as a point of interest (POI). The POI is used to characterize the distribution of different environmental features. For example, a container stack can be used as a POI to form an environmental feature. Based on the above map data and multiple frames of continuous first lane images, the position data of a single vehicle in each frame of the image can be determined. For example, the field of view distance between a single vehicle and the container stack POI in each frame of the first lane image is combined with the configuration parameters such as the shooting angle preset in the first camera to identify the specific position information of the single vehicle.

[0183] In some implementations, the specific location information includes lane information, longitude and latitude information, and displacement vector information. In combination with the longitude and latitude information and the displacement vector information, the electronic device 230 can determine the displacement of the vehicle, thereby facilitating the determination of the vehicle's travel speed and travel direction.

[0184] 303 : Determine relevant traffic signs according to multiple frames of continuous first lane images.

[0185] For example, the electronic device 230 may determine relevant traffic signs based on the above-mentioned multiple frames of continuous first lane images. It is understood that the relevant traffic signs may be signs related to traffic regulations, such as traffic lights, traffic signs, and traffic markings. Traffic lights may include motor vehicle lights, non-motor vehicle lights, pedestrian crossing lights, direction indicator lights, and the like.

[0186] In some embodiments, the electronic device 230 can use a preset event recognition model to recognize the above-mentioned relevant traffic signs from multiple frames of first lane images to facilitate the determination of traffic violation events. The above-mentioned preset event recognition model includes but is not limited to a neural network model trained with preset traffic violation events.

[0187] It can be understood that the above-mentioned neural network models include but are not limited to region convolutional neural network (R-CNN), fast region convolutional neural network (Faster RCNN) and residual neural network combined with feature pyramid network (residual network + feature pyramid network, ResNet-FPN).

[0188] It can be understood that R-CNN is a neural network model that performs target detection on images to complete feature recognition. In R-CNN, the region of interest (ROI) for target detection can be created in the form of candidate regions, and these regions are converted into images of fixed size and input into the convolutional neural network (CNN) respectively. Then, several fully connected layers are connected after CNN to realize target feature classification and calculate the bounding box. The above bounding box selects the candidate region (region proposal) for final target recognition. In some embodiments, multiple candidate regions for calculating feature vectors can be selected on the first lane image input to R-CNN, for example, each traffic sign on the road is selected into a single candidate region, and the corresponding identification features are calculated for each candidate region. Furthermore, the traffic regulations represented by the traffic sign can be determined based on the above identification features. The driving characteristics of each vehicle determined by combining multiple frames of continuous first lane images can determine whether a single vehicle is in violation of traffic rules. For example, determine whether the driving characteristics of the vehicle match the traffic regulations represented by the traffic sign. If the driving characteristics of the vehicle do not match the traffic regulations represented by the traffic sign, it can be determined that the vehicle has committed a traffic violation, and the event type of the traffic violation can be determined, such as illegal parking, speeding, or driving in the wrong direction.

[0189] 304, determining the violation event based on relevant traffic signs and single vehicle driving parameters.

[0190] It can be understood that based on the driving parameters of each vehicle combined with relevant traffic signs, the electronic device 230 can effectively identify traffic violations.

[0191] For example, when a vehicle stops at the side of the road, if the position of the vehicle does not change in multiple consecutive frames of the first lane image, the displacement is 0, and there is a no-parking sign on the road, it can be determined that the vehicle has violated the traffic rules and the violation is illegal parking.

[0192] For another example, when a vehicle is traveling on a road, multiple frames of continuous first lane images determine that the ratio of the displacement generated by the vehicle to the time taken is 120 km / h, and the road has a speed limit sign of 60 km / h, then it can be determined that the vehicle has committed a traffic violation, and the traffic violation is speeding.

[0193] For example, refer to Figure 4 When a vehicle is traveling on the road, multiple frames of continuous first lane images determine that the vehicle moves from point A1 to point A2, but the driving direction indicated by the road marking R is from point A2 to point A1, then it can be determined that the vehicle has violated the traffic rules, and the violation is wrong-way driving.

[0194] 305, determining the appearance characteristics and driving behavior characteristics of the target vehicle according to the traffic violation event.

[0195] Exemplarily, the electronic device 230 may determine a target vehicle based on the traffic violation event, and may frame a selected image area of ​​the target vehicle from multiple frames of first lane images using a preset feature recognition model.

[0196] It can be understood that the above-mentioned preset feature recognition model includes but is not limited to a neural network model trained with preset vehicle appearance features.

[0197] It can be understood that the above-mentioned neural network models include but are not limited to region convolutional neural network (R-CNN), fast region convolutional neural network (Faster RCNN) and residual neural network combined with feature pyramid network (residual network + feature pyramid network, ResNet-FPN).

[0198] In some embodiments, the target vehicle can be framed and selected as a candidate area on the first lane image in the input preset feature recognition model, and the corresponding appearance features of the target vehicle candidate area can be calculated and extracted, such as vehicle model, color, etc. It can be understood that the above appearance features can be used to distinguish the target vehicle from vehicles with different appearances, thereby improving the efficiency of identifying the target vehicle.

[0199] In addition, the driving behavior characteristics of the target vehicle can also be determined, such as the vehicle's spatiotemporal position change information, including the vehicle speed, the vehicle's driving lane and its lane change information, etc. It can be understood that even if the vehicles look similar, the vehicles' lane change information and real-time driving speeds are different. Therefore, based on the characteristic dimension of the vehicle's spatial information that changes in time, combined with other features such as the corresponding speed, different vehicles can be effectively identified, which can then be used to accurately locate illegal vehicles. The specific implementation process of determining the driving behavior characteristics of the target vehicle will be described in detail below and will not be repeated here.

[0200] 306 , obtaining multiple frames of second lane images captured by a second camera according to the traffic violation event.

[0201] For example, the electronic device 230 can continuously identify the target vehicle violating the traffic rules in the first video stream according to the determined traffic violation event, so as to continuously track the location information of the target vehicle. Furthermore, when the target vehicle violating the traffic rules enters the overlapping shooting range of the first camera and the second camera, the electronic device 230 can simultaneously obtain the first video stream captured by the first camera and the second video stream captured by the second camera, so as to accurately identify the same violating vehicle and its identification information by obtaining video stream data from multiple sources with overlapping shooting ranges.

[0202] 307 , determining the target vehicle from all vehicles in the multiple frames of the second lane image according to the appearance features and driving behavior features of the target vehicle.

[0203] It can be understood that since the appearance characteristics and driving behavior characteristics of each vehicle are specific and easy to identify, such as the spatial position changes of the vehicle in accordance with the time sequence, the different lane change situations of the vehicle, or the different speeds of the vehicles changing lanes to the same lane. In this way, the illegal vehicle can be accurately determined to avoid the vehicle with a similar appearance to the illegal vehicle being mistakenly identified as the target vehicle for the illegal traffic. Therefore, the electronic device 230 can easily find the target vehicle in the second video stream. Since the second camera has a high shooting clarity, the electronic device 230 can determine the identification information of the target vehicle, such as the license plate number, from multiple frames of the second lane image in the second video stream.

[0204] 308 , determining identification information of the target vehicle according to the multiple frames of the second lane image.

[0205] Exemplarily, the electronic device 230 can identify the identification information of the target vehicle from the above-mentioned multiple frames of second lane images in combination with the preset feature recognition model. The identification information can be any identification information different from other vehicles, and can be a text identification or an image identification.

[0206] In some embodiments, the electronic device 230 can select the identification area of ​​the target vehicle as a candidate area on the second lane image in the input preset feature recognition model, and extract corresponding identification features based on the candidate area, such as license plate number, text coding information, etc.

[0207] It can be understood that according to the above steps 301 to 308, the vehicle identification method can identify the violation event through the first video stream captured by the first camera, and identify the appearance characteristics and driving behavior characteristics of the violation vehicle when the violation event is identified, and then track and identify the violation vehicle in real time. When the violation vehicle enters the shooting range of the second camera, multiple frames of the second lane image in the monitoring video of the second camera are obtained, and the appearance characteristics and driving behavior characteristics of the violation vehicle are used to determine the target vehicle from the second lane image, and then determine the identification information of the target vehicle, such as the license plate number. Thereby, the missed identification of the violation event and the misidentification of the violation vehicle are effectively avoided, and the violation vehicle can be accurately reported by means of the determined identification information of the violation vehicle.

[0208] Regarding the specific implementation process of the above steps 307 to 308, Figure 5A , Figure 5B and Figure 6 Provide a detailed introduction.

[0209] Figure 5A A schematic diagram of a vehicle recognition scenario with non-overlapping shooting areas is shown.

[0210] refer to Figure 5A , vehicle 00 travels on a road along the X direction, and poles 20 and 21 are provided on the road. Camera 021 provided on pole 20 photographs area 021R of the road, and camera 022 provided on pole 21 photographs area 022R of the road. Here, camera 021 is used to identify traffic violation events, and camera 022 is used to identify vehicle identification information and report traffic violation events.

[0211] It can be understood that since the shooting areas of the cameras shooting the road do not overlap, the vehicle can change lanes in the non-overlapping areas of the camera shooting ranges.

[0212] Therefore, if the camera shooting areas do not overlap, the following two types of illegal vehicle identification errors may occur:

[0213] (1) The illegal vehicle in zone 021R changes lanes, and a similar-looking vehicle passes through zone 022R in the original lane, causing the similar-looking vehicle to be mistakenly reported as an illegal vehicle;

[0214] (2) The illegal vehicle in area 021R changes lanes, and no other vehicles pass through the original lane. The data used for violation reporting is empty data.

[0215] In addition, reporting of traffic violations requires a complete video stream as evidence of the violation, so if the camera shooting areas do not overlap, it is impossible to obtain complete video evidence of the violation.

[0216] To this end, some embodiments of the present application propose a specific implementation process for image recognition of the target vehicle in the second lane, so that the shooting areas of the first camera and the second camera are at least partially overlapped, and image registration and image stitching are performed on the different videos captured by the two cameras based on the overlapping shooting areas, ensuring that the two cameras can capture the same illegal vehicle, and the illegal vehicle can be accurately re-identified at the second camera, and complete video evidence of the violation can be obtained.

[0217] Combine the following Figure 5B and Figure 6 The specific implementation process of the above-mentioned use of image registration and image stitching to identify the target vehicle in the second lane image is specifically described.

[0218] Figure 5B According to an embodiment of the present application, a schematic diagram of an implementation process of determining a target vehicle based on multiple frames of second lane images is shown. It can be understood that Figure 5B The execution entity of each step of the illustrated process may be the electronic device 230 described above, or other electronic devices, and the execution entity of a single step will not be described in detail.

[0219] like Figure 5B As shown, the process may include the following steps:

[0220] 501, obtaining a plurality of continuous frames of first lane images captured by a first camera, and determining a first background image.

[0221] Exemplarily, the electronic device 230 may obtain a plurality of continuous frames of first lane images captured by the first camera. In some embodiments, the plurality of continuous frames of first lane images capture the overlapping shooting area of ​​the first camera and the second camera to determine the first background image of the overlapping shooting area under the perspective of the first camera.

[0222] It can be understood that the first background image is an image with dynamic information removed, such as a road image obtained after removing moving vehicles, and the road image obtained after removing the moving vehicles may also include static scenes within the shooting range. In some embodiments, multiple consecutive frames of first lane images taken by the first camera can be obtained, and dynamic information can be removed by statistical filtering to obtain a background image for reference. For example, the average pixel value of each pixel of the collected multiple frames can be obtained by accumulating the pixel values ​​of each pixel in the multiple frames and dividing by the number of frames through the multi-frame averaging method. Then, all pixels are combined by the average pixel value of each pixel to obtain a reference background image.

[0223] 502, acquiring a plurality of continuous frames of second lane images captured by a second camera, and determining a second background image.

[0224] Exemplarily, the electronic device 230 may obtain a plurality of continuous frames of the second lane image captured by the second camera after determining that a traffic violation has occurred. In some embodiments, the plurality of continuous frames of the second lane image captures the overlapping shooting area of ​​the first camera and the second camera to determine the second background image of the overlapping shooting area under the perspective of the second camera.

[0225] It can be understood that the second background image is an image with dynamic information removed, such as a road image obtained by removing moving vehicles, and the road image obtained by removing moving vehicles may also include static scenes within the shooting range.

[0226] For example, refer to Figure 6 , the continuous multiple frames of the second lane image can present multiple frames of images in the area C. After removing the dynamic information, the second background image containing the tree 601 and located in the area C can be determined.

[0227] 503 , performing image registration on a plurality of consecutive frames of second lane images and a plurality of consecutive frames of first lane images according to the first background image and the second background image, to obtain a plurality of registered second lane images.

[0228] For example, the electronic device 230 may perform image registration based on the first background image of the overlapping shooting area at the first camera angle of view and the second background image of the second camera angle of view. For example, the electronic device 230 may perform image registration with the continuous multiple frames of the second lane image that captures the overlapping shooting area and the continuous multiple frames of the first lane image. For example, the electronic device 230 may perform image registration with the first lane image and the second lane image by identifying the same target feature in the first background image and the second background image.

[0229] In some embodiments, the electronic device 230 can determine the image mapping relationship between the first lane image and the second lane image based on the first background image and the second background image. The image mapping relationship can be represented by a perspective conversion coefficient, and then the first lane image and the second lane image are spliced ​​according to the perspective conversion coefficient to obtain a multi-frame spliced ​​image.

[0230] In some embodiments, the electronic device 230 can use a trained image stitching model to complete the stitching process of the first lane image and the second lane image. For example, in the pre-registration stage of the image stitching model, the second background image can be perspective transformed to generate a training set, the second background image can be used as a reference image, and the scale invariant feature transform (SIFT) algorithm can be used to extract feature coefficients from the training set, and these feature coefficients are input into the image stitching model for training. Thereafter, the output data of the trained image stitching model is the perspective transformation parameter, which is used to characterize the image mapping relationship between the second background image and the first background image. Furthermore, the electronic device 230 can transform multiple frames of continuous first lane images into the perspective of the second lane image according to the above perspective transformation parameters. For example, the image mapping relationship between the second background image and the first background image is obtained by taking the second background image as a reference image, and the corresponding perspective transformation parameters are determined. The first lane image taken at a high altitude can be changed accordingly according to the above perspective transformation parameters. For example, the electronic device 230 can rotate, translate, scale, and perform a series of image processing on the first lane image, so that the first lane image can have the same shooting perspective as the second lane image. This makes it easier to determine the features of each vehicle to be compared in the second lane image, such as driving behavior features such as lane change information, driving direction and speed, and appearance information such as vehicle model and color, so as to improve the accuracy of feature comparison with the target vehicle violating the traffic rules in the first lane image.

[0231] 504 , determining the target vehicle and identification information of the target vehicle from the second lane image after multiple frames of registration according to the appearance information and driving behavior characteristics of the target vehicle.

[0232] For example, the electronic device 230 can re-identify the target vehicle that violates the traffic rules from the multiple frames in the second lane image after the above registration, and can further identify the identification information of the target vehicle. The identification process of the target vehicle that violates the traffic rules and the identification information of the target vehicle will be described in detail below, and will not be repeated here.

[0233] 505 , stitching a plurality of continuous frames of first lane images and a plurality of continuous frames of registered second lane images into a traffic violation video of the target vehicle.

[0234] Exemplarily, the electronic device 230 can stitch together the first video stream captured by the first camera and the second video stream captured by the second camera based on multiple frames of first lane images corresponding to the overlapping shooting area and multiple frames of second lane images aligned based on the first lane images to obtain a complete violation video of the target vehicle.

[0235] It can be understood that the above-mentioned image stitching process may be stitching the first lane image into the second lane image, or may be stitching the second lane image into the first lane image.

[0236] Combine the following Figure 6 The implementation process of stitching the second lane image into the first lane image in the above example is described in detail.

[0237] Figure 6 According to some embodiments of the present application, a schematic diagram of a scene of image registration through target feature recognition is shown.

[0238] refer to Figure 6 , the continuous multiple frames of the first lane image can present multiple frames of images in area B. After removing the dynamic information, the first background image containing the tree 601 and located in area B can be determined. Furthermore, the tree 601 in the overlapping area D captured by the first camera and the second camera can be used as the target feature. The electronic device 230 can perform image registration on the first background image and the second background image without dynamic information based on the tree 601, and then splice the first lane image presenting area B and the second lane image presenting area C into the same image after image deformation based on the tree 601, thereby obtaining the spliced ​​multiple frames of the second lane image.

[0239] It can be understood that the target features used for image registration may also be container stacks, buildings, road signs, etc.

[0240] For example, the electronic device 230 may extract feature points from a background image and a second background image, for example, may extract feature points of a tree 601, and then match the feature points of the tree 601 between the two. After determining the target feature tree 601, the electronic device 230 may perform image registration on the first background image and the second background image, determine the spliceable position of the second background image in the first background image, and splice the second background image into the spliceable position of the first background image.

[0241] Continue to refer Figure 6 When the traffic violation of vehicle 602 is speeding, since the shooting area C of the second camera is small, the image of the second lane may only be one frame due to the excessive speed of vehicle 602, resulting in the inability to output complete video evidence of the traffic violation, which makes the evidence of the traffic violation weak.

[0242] It can be understood that the electronic device 230 can obtain a complete traffic violation video stream for reporting the traffic violation event by splicing multiple frames of first lane images in which the traffic violation event is captured but the vehicle logo is difficult to identify into multiple frames of second lane images in which the traffic violation event is difficult to capture but the vehicle logo can be accurately identified. This can make the evidence of the traffic violation event more effective.

[0243] It can be understood that according to the above steps 501 to 505, this embodiment can achieve image registration of the first lane image and the second lane image according to the common target features in the background, so that multiple frames of the first lane image and multiple frames of the second lane image can be spliced ​​into multiple frames of spliced ​​images. Based on the multiple frames of spliced ​​images, the target vehicle that violates the traffic rules can be accurately identified, and a complete violation video stream can be obtained, which is convenient for reporting the violation incident.

[0244] Regarding the specific implementation process of step 504 above, Figure 7 Provide a detailed introduction.

[0245] Figure 7 According to an embodiment of the present application, a schematic diagram of an implementation process of determining a target vehicle based on a multi-frame spliced ​​image is shown. It can be understood that Figure 7 The execution entity of each step of the illustrated process may be the electronic device 230 described above, or other electronic devices, and the execution entity of a single step will not be described in detail.

[0246] 701, determining a lane change information sequence of a target vehicle according to a plurality of consecutive frames of first lane images captured by a first camera, wherein the plurality of consecutive frames of first lane images are a plurality of consecutive frames of first lane images of an overlapping capture area of ​​the first camera and the second camera.

[0247] For example, the continuous multiple frames of the first lane image captured by the first camera can identify the lane information of the vehicle. Therefore, the electronic device 230 can determine the continuous lane change information of the target vehicle corresponding to the violation event in the multiple frames of the first lane image, and then record the lane position of the vehicle according to the lane change sequence to obtain the lane change information sequence.

[0248] Combine the following Figure 8 The lane change information sequence is described in detail.

[0249] Figure 8 According to some embodiments of the present application, a schematic diagram of a vehicle lane changing scenario is shown.

[0250] refer to Figure 8 , the four-lane road is marked as lane 1, lane 2, lane 3 and lane 4. Based on multiple frames of first lane images arranged in time sequence, such as first lane images at time T1, time T2, time T3 and time T4, it can be determined that the vehicle changes lanes from lane 1 to lane 2, then changes lanes from lane 2 to lane 3, and changes lanes from lane 3 to lane 4 at time T4. At this time, the lane change information sequence of the vehicle can be recorded as 1234.

[0251] It should be noted that the lane change information sequence mentioned above only records the lane information of the vehicle in sequence according to the lane change order, and further characterizes the lane change information of the vehicle during the time period corresponding to the multiple frames of the first lane image. It is understandable that since the shooting frequencies of the first camera and the second camera may be different, it is possible that when the first camera captures a single frame of the first lane image, the second camera does not capture a new image at the same time, so it is difficult for the two cameras to perform accurate time alignment. Therefore, a lane change information sequence that does not record specific time information is proposed. There is no need to record the specific moment when the vehicle enters the new lane. It is only necessary to record the lane information of the vehicle in sequence during the captured time period. It can also effectively characterize the driving characteristics of the illegal vehicles in the overlapping shooting area, so as to facilitate the re-identification of the target vehicle in the second lane image.

[0252] 702 , clustering the driving behavior features of the lane change information sequence of the target vehicle and the appearance features of the target vehicle to obtain reference cluster features.

[0253] Exemplarily, the appearance features include but are not limited to the color, pattern and model of the vehicle, while the driving behavior features reflect the temporal and spatial position changes of the vehicle, including but not limited to the vehicle speed and lane change information. The electronic device 230 can cluster the lane change information sequence of the target vehicle in combination with the features such as vehicle speed, vehicle color, model, and vehicle pattern, and obtain a high-dimensional reference cluster feature.

[0254] It can be understood that the above clustering is the process of dividing a collection of physical or abstract objects into multiple classes consisting of similar objects. The cluster generated by clustering is a collection of data objects that are similar to objects in the same cluster and different from objects in other clusters.

[0255] For example, the dimension of the reference cluster feature refers to the attribute of the feature. For example, vehicle speed is a feature attribute, and vehicle color is also a feature attribute. Then, vehicle speed, vehicle color, vehicle model, and vehicle pattern can be used to obtain a four-dimensional reference cluster feature. For example, in a high-dimensional space, the reference cluster feature can be regarded as a spherical four-dimensional feature cluster.

[0256] Furthermore, the reference cluster features can be used to analyze the target vehicles that violate the traffic rules. For example, the electronic device 230 can compare the obtained reference cluster features with the features of each vehicle in the second lane image, that is, the electronic device 230 can determine the vehicle with the highest feature similarity in the second lane image as the target vehicle that violates the traffic rules.

[0257] 703, determining a lane change information sequence of each to-be-compared vehicle based on the continuous multiple frames of the second lane image captured by the second camera, wherein the continuous multiple frames of the second lane image are continuous multiple frames of the second lane image of the overlapping capture area of ​​the first camera and the second camera.

[0258] Exemplarily, the continuous multiple frames of the second lane images captured by the second camera may be continuous multiple frames of the second lane images after registration. For example, the electronic device 230 may identify the lane information of each vehicle to be compared based on the continuous multiple frames of the second lane images after registration. Therefore, the electronic device 230 may determine the continuous lane change information of each vehicle to be compared in the multiple frames of the second lane images, and then the electronic device 230 may record the lane position of the vehicle according to the lane change sequence of the vehicle to obtain the lane change information sequence.

[0259] 704 , cluster the driving behavior features including the lane change information sequence of each vehicle to be compared and the appearance features of each vehicle to be compared to obtain cluster features to be compared.

[0260] It is understandable that the electronic device 230 can cluster the driving behavior characteristics and appearance characteristics of the vehicle to be compared to obtain high-dimensional clustered characteristics to be compared. The specific implementation of clustering can refer to the implementation of clustering the driving behavior characteristics and appearance characteristics of the target vehicle in step 703 above, and for the sake of simplicity, it will not be repeated here.

[0261] 705 , determining the similarity between the reference cluster feature and each cluster feature to be compared.

[0262] Exemplarily, the reference clustering feature can be regarded as a high-dimensional spherical cluster in the high-dimensional space, and the single clustering feature to be compared can also be regarded as a high-dimensional spherical cluster in the high-dimensional space. Therefore, the electronic device 230 can determine the similarity between the reference clustering feature and each clustering feature to be compared by calculating the distance value between the reference clustering feature and each clustering feature to be compared. It can be understood that the smaller the distance value between different feature clustering clusters, the greater the similarity.

[0263] 706 , determining the vehicle to be compared corresponding to the cluster feature to be compared with the greatest similarity as the target vehicle.

[0264] It can be understood that the most similar vehicle to be compared is the target vehicle, and the electronic device 230 can perform a second accurate identification of the target vehicle in the first camera in the video stream of the second camera based on multiple features such as lane change sequence information, vehicle speed, color, etc., so as to identify the same illegal vehicle based on the first lane image and the second lane image.

[0265] Combine the following Fig. 9 The process of identifying target vehicles based on high-dimensional features is described in detail.

[0266] Fig. 9 According to some embodiments of the present application, a flow chart based on illegal vehicles and vehicles to be compared is shown.

[0267] refer to Fig. 9 , the illegal vehicle is the target vehicle corresponding to the illegal event identified in the overlapping shooting area of ​​the first lane image, and the vehicle to be compared is each vehicle in the overlapping shooting area of ​​the second lane image. In order to identify the target vehicle among the vehicles to be compared, the electronic device 230 can fuse the multi-parameter features of the illegal vehicle to obtain the reference feature, and fuse the multi-parameter features of the vehicle to be compared to obtain the feature to be compared, and then compare the similarity between the reference feature and the feature to be compared, and determine the feature to be compared with the highest similarity. Therefore, the vehicle to be compared corresponding to the most similar feature to be compared is the illegal vehicle in the second lane image, thereby realizing the tracking and identification of the illegal vehicle from the first camera to the second camera. Among them, the above-mentioned multi-parameter features include the lane change information sequence, the vehicle speed in the overlapping shooting area, the vehicle color and the vehicle signal.

[0268] It is understood that the lane change information sequence and vehicle speed can be determined based on the corresponding multiple frames of first lane images of the violating vehicle in the overlapping shooting area. For example, the electronic device 230 can record the lane change information sequence based on time sequence, and can determine the vehicle speed based on the first lane images of different frames.

[0269] It can be understood that the specific implementation method for determining the vehicle speed has been illustrated above and will not be described in detail here for the sake of brevity.

[0270] It can be understood that the lane change information sequence, the vehicle speed in the overlapping shooting area, the vehicle color and the vehicle model can be integrated into features of multiple dimensions, so the electronic device 230 can cluster the features of the multiple dimensions in the high-dimensional space. The multi-parameter features of the illegal vehicles can be clustered to obtain the reference cluster features, and the multi-parameter features of the vehicles to be compared can be clustered to obtain the cluster features to be compared. The reference cluster features and the cluster features to be compared are compared by clustering distance to determine the similarity between the two. Thus, the vehicle to be compared with the smallest distance can be determined as the most similar vehicle to be compared, and it can be determined as the target vehicle and the recognition result of the target vehicle is output.

[0271] It can be understood that according to the above steps 701 to 706, this implementation can characterize the multiple dimensions of features of a single vehicle based on the vehicle lane change information in the overlapping shooting area combined with the vehicle speed, vehicle color and vehicle model clustering, so that the target vehicle can be accurately identified again in the second lane image corresponding to the overlapping shooting area, ensuring that the first camera and the second camera can identify the same illegal vehicle.

[0272] In order to improve the accuracy and speed of feature comparison, the features of the vehicles in the images taken by the first camera and the second camera at the same time can be compared. Fig.10 The specific implementation process of step 703 after time registration is described in detail.

[0273] Fig.10 According to an embodiment of the present application, a schematic diagram of an implementation process of determining a vehicle to be compared based on a timestamp is shown. It can be understood that Fig.10 The execution entity of each step of the illustrated process may be the electronic device 230 described above, or other electronic devices, and the execution entity of a single step will not be described in detail.

[0274] like Fig.10 As shown, the process may include the following steps:

[0275] 703a, determining an imaging error time period according to an imaging timestamp of the first lane image and a preset imaging error time between the first camera and the second camera.

[0276] For example, when the first camera captures the first lane image, an imaging timestamp may be marked for the first lane image according to the internal time of the first camera itself. Similarly, when the second camera captures the second lane image, an imaging timestamp may be marked for the second lane image according to the internal time of the second camera itself.

[0277] It is understandable that due to hardware differences, shooting frequency errors, transmission link delay errors and other reasons between cameras, the actual shooting time of the first lane image and the second lane image with the same timestamp does not correspond, making it difficult to ensure that the relative position information of the compared vehicles in the overlapping shooting area is consistent.

[0278] In order to improve the recognition accuracy of the target vehicle, in some embodiments, the electronic device 230 may store a preset imaging error time between the first camera and the second camera. For example, the preset imaging error time may be written into a configuration file as a configuration parameter so that a single-frame first lane image and multiple-frame second lane images within the imaging error time are used for vehicle comparison. When the electronic device 230 re-identifies the target vehicle violating the traffic rules in the second lane image based on the first lane image, multiple-frame second lane images within the imaging error time period may be retrieved based on the single-frame first lane image to determine all vehicles to be compared, thereby ensuring that there is a vehicle to be compared in the second lane image that is most similar in relative position to the target vehicle in the first lane image.

[0279] 703b, acquiring multiple frames of second lane images within the imaging error time period.

[0280] In some embodiments, the target vehicle is shown to enter the overlapping shooting area in the first lane image at time T, and the electronic device 230 retrieves the second lane image based on the timestamp at time T. The electronic device 230 obtains the preset imaging error time as ΔT, and then retrieves multiple frames of second lane images with timestamps corresponding to the time period from time T to time (T+ΔT), thereby ensuring that there is a vehicle to be compared in the second lane image that is closest to the relative position of the target vehicle in the first lane image. This can improve the recognition accuracy of the target vehicle. It can be understood that only retrieving the second lane image data of a time period, for example, only retrieving the second lane image within the preset imaging error time period, greatly reduces the number of vehicles to be compared compared to retrieving the image data of the entire time length captured by the second camera, thereby improving the speed of target vehicle recognition.

[0281] 703c, determining all vehicles to be compared according to multiple frames of second lane images within the imaging error time period.

[0282] For example, the electronic device 230 may identify all vehicles in multiple frames of second lane images within the imaging error time period and use them as vehicles to be compared.

[0283] 703d, determining the lane change information sequence of each vehicle to be compared based on all the vehicles to be compared.

[0284] For example, the electronic device 230 can identify the lane information of each vehicle to be compared based on the second lane image after multiple consecutive frames of registration. Therefore, the electronic device 230 can determine the continuous lane change information of each vehicle to be compared in the multiple frames of the second lane image, and then the electronic device 230 can record the lane position of the vehicle according to the lane change sequence of the vehicle to obtain the lane change information sequence.

[0285] According to the specific implementation process of the above steps 703a to 703d, when the electronic device 230 re-identifies the target vehicle violating the traffic rules in the second lane image based on the first lane image, multiple frames of the second lane image within the imaging error time period can be retrieved according to the timestamp of the single frame of the first lane image to determine all the vehicles to be compared, and the target vehicle identification processing is performed on the above vehicles to be compared. In this way, it is ensured that there is a vehicle to be compared in the second lane image that is most similar to the target vehicle in the first lane image in relative position, thereby improving the recognition accuracy of the target vehicle.

[0286] In order to avoid missing illegal vehicles, based on the above Figure 2B The vehicle identification scenario is an example. The method flow of vehicle identification in other embodiments of the present application is described in detail below in conjunction with relevant drawings.

[0287] Example 2

[0288] Fig.11AAccording to some embodiments of the present application, a flow chart of a vehicle identification method is shown. The difference from Embodiment 1 is that Embodiment 2 may include multiple camera combinations, each of which may include a first camera and a second camera, wherein the lane image taken by the first camera is used to identify violations, and the lane image taken by the second camera is used to identify identification information, such as license plate numbers. Then, the illegal vehicle is tracked and identified based on the overlapping shooting areas between adjacent camera combinations, and the illegal vehicle is identified through the overlapping shooting areas of the first camera and the second camera in the same camera combination, and then the specific license plate information is identified through any clear image frame of the illegal vehicle taken by the second camera. Therefore, while accurately tracking and identifying the illegal vehicle, a complete evidence video containing the illegal behavior and the license plate number can be obtained.

[0289] Understandably, Fig.11A The execution entity of each step of the illustrated process may be the electronic device 230 described above, or other electronic devices, and the execution entity of a single step will not be described in detail.

[0290] like Fig.11A As shown, the process may include the following steps:

[0291] 1101, obtain multiple frames of continuous first lane images taken by the first camera in all camera combinations.

[0292] Exemplarily, the first camera and the second camera may be recorded in pairs as a camera combination, for example, a short-focus camera may be used as the first camera, and a long-focus camera may be used as the second camera. Furthermore, the camera parameters in the camera combination may be recorded in the electronic device 230, for example, the setting position of a single camera combination may be recorded in the electronic device 230. At this time, the electronic device 230 may determine the deployment position of the camera combination, and then determine the adjacent camera combination.

[0293] It is understandable that the two cameras are recorded as a camera combination in pairs for example only. Here, it is sufficient that the cameras in the camera combination have at least some overlapping shooting areas, and there is no limit on the specific number of cameras. For example, a short-focus camera can be combined with multiple long-focus cameras to form a camera combination.

[0294] Exemplarily, the shooting areas of two first cameras in adjacent camera combinations may be set to at least partially overlap, so that the shooting areas of multiple first cameras in multiple camera combinations can cover the entire road, facilitating tracking and identifying the target vehicle.

[0295] In some embodiments, different tags can also be added to each camera, for example, the camera can be marked with a unique identification code, so that when the video stream shot by a single camera needs to be called, the electronic device 230 can quickly query or retrieve the video stream shot by the camera according to the unique identification code.

[0296] Therefore, illustratively, the electronic device 230 can obtain multiple frames of continuous first lane images taken by the first camera in all camera combinations through the unique identification code to obtain real-time images of all roads to avoid missing identification of violations.

[0297] 1102, determining a single vehicle driving parameter according to a plurality of frames of continuous first lane images and map data.

[0298] 1103 , determining relevant traffic signs according to multiple frames of continuous first lane images.

[0299] 1104, determining the violation event based on relevant traffic signs and single vehicle driving parameters.

[0300] It can be understood that the specific implementation of the above steps 1102 to 1104 can refer to the above steps 302 to 304, and for the sake of brevity, they are not repeated here.

[0301] 1105 , determining the driving direction of the target vehicle according to the traffic violation event and the continuous multiple frames of the first lane image.

[0302] Exemplarily, the electronic device 230 can identify the target vehicle that violates the traffic rules from multiple consecutive frames of first lane images based on the traffic violation event, and then determine the real-time position of the target vehicle based on the multiple consecutive frames of first lane images, and then determine the driving direction of the target vehicle based on the shooting area of ​​the camera.

[0303] It should be noted that the driving direction is usually the displacement direction of the target vehicle and the vector orientation of the map data, but in this application, the driving direction of the target vehicle is determined based on the relative driving direction of the vehicle and the shooting area of ​​the camera. For example, the camera combination where the first camera that identifies the illegal vehicle is located is used as the current camera combination, and when the vehicle drives from the first camera shooting area of ​​the current camera combination to the second camera shooting area, it is determined that the vehicle is traveling in the forward direction.

[0304] It can be understood that defining the driving direction of the vehicle by using different camera shooting areas can effectively determine the camera to be called in real time, thereby facilitating real-time tracking and identification of the target vehicle.

[0305] 1106, determine whether the driving direction is forward driving. If the driving direction is forward driving, execute steps 1108 to 1110 below to determine the identification information of the target vehicle based on the multiple frames of the second lane image captured by the second camera in the current camera combination. If the driving direction is not forward driving, further execute step 1107 below to further determine whether the driving direction is reverse driving.

[0306] For example, when the vehicle moves from the first camera shooting area of ​​the current camera combination to the second camera shooting area, it can be determined that the vehicle is moving in the forward direction.

[0307] In some embodiments, the electronic device 230 can determine multiple consecutive frames of first lane images in the overlapping shooting area in the current camera combination based on the violation event, thereby determining the real-time position of the vehicle and the distance change between the center point of the overlapping shooting area in the current camera combination, and then determining the driving direction of the vehicle.

[0308] 1107, determine whether the driving direction is reverse driving. If the driving direction is reverse driving, execute steps 1111 to 1113 below to determine the identification information of the target vehicle based on the multiple frames of historical images captured by the second camera in the current camera combination. If the driving direction is not reverse driving, it can be determined that the driving direction is reverse driving, and then execute steps 1114 to 1116 below to call the adjacent camera combination to continue tracking the target vehicle, and enter step 1106 to re-determine the driving direction until the identification information of the target vehicle is determined.

[0309] For example, when the vehicle drives from the second camera shooting area of ​​the current camera combination to the first camera shooting area, it can be determined that the vehicle is driving in the reverse direction. When the vehicle drives from the first camera shooting area of ​​the current camera combination to the shooting area of ​​the adjacent camera combination and does not pass through the second camera shooting area of ​​the current camera combination, it can be determined that the vehicle is driving in the other direction.

[0310] Combine the following Fig. 11B It is explained in detail about driving.

[0311] Fig. 11B According to some embodiments of the present application, a schematic diagram of a scenario for identifying a vehicle based on its driving direction is shown.

[0312] refer to Fig. 11B, camera 1170A and camera 1180A constitute camera combination A, camera 1170B and camera 1180B constitute camera combination B, wherein the shooting area of ​​camera 1170A and the shooting area of ​​camera 1170B at least partially overlap. Vehicle 1190 travels in direction X2, and travels in direction X1 after turning. If the electronic device 230 identifies that the vehicle 1190 is speeding and turning according to camera 1170A, it is necessary to identify the vehicle 1190 in real time. Since vehicle 1190 does not enter the shooting range of camera 1170B of camera combination A, but turns and enters the shooting range of the adjacent camera 1170B, the electronic device 230 can determine that the vehicle 1190 is traveling in the other direction. The electronic device 230 can identify the vehicle 1190 based on the image captured by the camera 1170B and continue to track the vehicle 1190 until the vehicle is captured by the camera 1180B, then re-identify the vehicle 1190 based on the image captured by the camera 1180B and determine the identification information of the vehicle 1190, such as the license plate number.

[0313] 1108. Acquire multiple frames of second lane images captured by a second camera in the current camera combination according to the traffic violation event.

[0314] 1109 , determining the target vehicle from all vehicles in the multiple frames of the second lane image according to the appearance information and the driving behavior characteristics of the target vehicle.

[0315] 1110, determine identification information of the target vehicle according to multiple frames of second lane images.

[0316] It can be understood that the specific implementation of the above steps 1108 to 1110 can refer to the above steps 306 to 308, and for the sake of brevity, they are not repeated here.

[0317] 1111. Acquire multiple frames of historical second lane images taken by a second camera in the current camera combination according to the time of the traffic violation event.

[0318] For example, since reverse driving means that the vehicle enters the shooting area of ​​the first camera from the shooting area of ​​the second camera, when the first camera captures the vehicle violating traffic rules, the identification information cannot be directly identified. Therefore, the video stream captured by the second camera before the first camera captured the traffic violation event can be traced back to determine the historical images that can be used to identify the identification information. Furthermore, the electronic device 230 can identify the identification information of the target vehicle based on the multiple frames of historical second lane images captured by the second camera.

[0319] 1112, determine the target vehicle from all vehicles in multiple frames of historical second lane images according to the appearance information and driving behavior characteristics of the target vehicle.

[0320] 1113, determine identification information of the target vehicle according to multiple frames of historical second lane images.

[0321] It can be understood that the specific implementation of the above steps 1112 to 1113 can refer to the above steps 307 to 308, and for the sake of brevity, they are not repeated here.

[0322] 1114 , obtain a plurality of continuous frames of images of the third lane taken by the first camera in the adjacent camera combination.

[0323] Exemplarily, the electronic device 230 can determine a camera combination adjacent to the current camera combination based on the preset position of the camera combination. For example, the electronic device 230 recognizes that the vehicle is traveling in the other direction, retrieves the video stream shot by the first camera in the camera combination adjacent to the current camera combination, and determines multiple frames of the third lane image. Since the first camera is set at a position far away from the horizontal plane where the road is located, it is impossible to directly identify the identification information based on the third lane image, so the driving trajectory of the target vehicle can continue to be identified based on the third lane image to achieve tracking of the target vehicle until the target vehicle passes the second camera in any camera combination, and the identification information of the target vehicle is identified.

[0324] In some embodiments, the electronic device 230 can stitch the acquired continuous multiple frames of the third lane image and the multiple frames of the first lane image in which the violation event is identified into a continuous video stream. In addition, the electronic device 230 can also stitch all the continuous images retrieved in the process of tracking and identifying the violation event and identification information of the target vehicle into a complete video stream, so as to provide complete video evidence for reporting the violation vehicle and the violation event.

[0325] 1115, determining the target vehicle from all vehicles in the multiple frames of the third lane image according to the appearance information and driving behavior characteristics of the target vehicle.

[0326] It can be understood that the specific execution process of the above step 1115 can refer to the above step 307, and for the sake of brevity, it is not repeated here.

[0327] 1116. Determine the real-time position of the target vehicle entering the shooting area of ​​the second camera in any camera combination according to the multiple frames of continuous third lane images.

[0328] For example, the real-time position of the target vehicle is determined based on multiple frames of continuous third lane images, thereby achieving real-time tracking of the target vehicle that violates traffic rules. If it is determined that the target vehicle enters the shooting area of ​​the second camera in any camera combination, the electronic device 230 can obtain multiple frames of fourth lane images shot by the second camera that shoots the target vehicle to determine the identification information of the target vehicle.

[0329] 1117, obtain multiple frames of images of the fourth lane captured by the second camera that captures the target vehicle.

[0330] 1118. Determine identification information of the target vehicle based on the multiple frames of fourth lane images.

[0331] It can be understood that the specific execution process of the above steps 1117 to 1118 can refer to the above steps 1109 to 1110, and for the sake of brevity, they are not repeated here.

[0332] According to the above steps 1101 to 1118, the vehicle identification method can group the first camera and the second camera with at least partial overlapping shooting areas into a camera combination, and the shooting areas of two first cameras in adjacent camera combinations also overlap at least partially. In the process of the electronic device 230 tracking and identifying the target vehicle, the vehicle driving direction can be divided into three categories:

[0333] (1) A vehicle moving from the first camera to the second camera in the same group is identified as moving in the forward direction;

[0334] (2) The vehicle is identified as traveling in the opposite direction when it moves from the second camera to the first camera in the same group;

[0335] (3) The target vehicle that enters the combined shooting area of ​​the adjacent camera without passing through the second camera in the same group is identified as traveling in the other direction.

[0336] Thus, in different driving directions, the electronic device 230 retrieves video streams captured by different cameras, so that the electronic device 230 can identify the violation event based on the first lane image captured by the first camera in any camera combination, and track and identify the target vehicle in real time until the target vehicle passes the second camera in any camera combination, and then retrieves the fourth lane image captured by the second camera to determine the identification information of the target vehicle. The tracking process of the target vehicle is realized, and the passage path and identification information of the illegal vehicle can be accurately identified.

[0337] In other embodiments, since the illegal vehicle is tracked in real time after the illegal event is identified until the identification information of the illegal vehicle is identified, a continuous video corresponding to the time of the illegal behavior until the identification information is identified can be obtained. It can be understood that when reporting illegal vehicles and illegal events, the continuous evidence video containing illegal behavior and identification information has a good corroborative ability, so the reporting accuracy of illegal events can be improved.

[0338] Fig.12 According to some embodiments of the present application, the specific data processing flow of the above steps 1101 to 1117 is shown. It can be understood that Fig.12The execution entity of each step of the illustrated process may be the electronic device 230 described above, or other electronic devices, and the execution entity of a single step will not be described in detail.

[0339] like Fig.12 As shown, the process may include the following steps:

[0340] 1201, Identification of traffic violation events.

[0341] It can be understood that the specific implementation process of the above step 1201 can refer to the specific implementation process of steps 1101 to 1104 in the above text. For the sake of brevity, it will not be repeated here.

[0342] 1202, vehicle driving direction identification.

[0343] It can be understood that the specific implementation process of the above step 1202 can refer to the specific implementation process of the above step 1105, and for the sake of brevity, it will not be repeated here.

[0344] 1203, whether the vehicle is traveling in the forward direction. If yes, proceed to step 1205 to step 1206, and identify the license plate of the illegal vehicle with the multiple frames of the second lane image captured by the second camera. If no, proceed to step 1204, and determine whether the vehicle is traveling in the reverse direction.

[0345] It can be understood that the specific implementation process of the above step 1203 can refer to the specific implementation process of the above step 1106, and for the sake of brevity, it will not be repeated here.

[0346] 1204, whether the vehicle is driving in the opposite direction. If yes, proceed to step 1207, and use the multiple frames of images in the historical video captured by the second camera to re-identify the illegal vehicle, thereby facilitating the identification of the license plate of the illegal vehicle. If no, proceed to steps 1208 and 1209, and by locating the first camera in the adjacent camera combination, obtain images from the video sources of multiple cameras for comparison, so as to repeatedly re-identify the illegal vehicle and realize the tracking and identification of the illegal vehicle. Until the vehicle is located at the second camera, the illegal vehicle and its license plate number are re-identified.

[0347] It can be understood that the specific implementation process of the above step 1204 can refer to the specific implementation process of the above step 1107, and for the sake of brevity, it will not be repeated here.

[0348] 1205, the illegal vehicle is located at the second camera.

[0349] 1206, Multi-source comparison to re-identify vehicles violating traffic rules.

[0350] It can be understood that the specific implementation process of the above steps 1205 and 1206 can refer to the specific implementation process of steps 1108 to 1110 in the above text, and for the sake of brevity, they are not repeated here.

[0351] 1207, re-identify the illegal vehicle among the vehicles to be compared in the historical video.

[0352] It can be understood that the specific implementation process of the above step 1207 can refer to the specific implementation process of the above step 1107, and for the sake of brevity, it will not be repeated here.

[0353] 1208, the illegal vehicle is located at the first camera in the adjacent camera combination.

[0354] 1209, Multi-source comparison to re-identify vehicles violating traffic rules.

[0355] It can be understood that the specific implementation process of the above steps 1208 to 1209 can refer to the specific implementation process of steps 1114 to 1115 in the above text, and for the sake of brevity, they are not repeated here.

[0356] 1210, the illegal vehicle is located on a second camera in any camera combination.

[0357] 1211, Multi-source comparison to re-identify vehicles violating traffic rules.

[0358] It can be understood that the specific implementation process of the above steps 1210 to 1211 can refer to the specific implementation process of steps 1116 to 1117 in the above text. For the sake of brevity, they are not repeated here.

[0359] 1212, license plate recognition.

[0360] It can be understood that the specific implementation process of the above step 1212 can refer to the specific implementation process of step 308 above, and for the sake of brevity, it will not be repeated here.

[0361] According to the implementation of the above steps 1210 to 1212, the electronic device 230 retrieves the video streams captured by different cameras to synchronously compare the vehicles, so that the electronic device 230 can identify the violation event based on the first lane image captured by the first camera in any camera combination, and track and identify the target vehicle in real time until the target vehicle passes the second camera in any camera combination, then retrieve the fourth lane image captured by the second camera to determine the identification information of the target vehicle. The tracking process of the target vehicle is realized, and the passage path and identification information of the illegal vehicle can be accurately identified.

[0362] In other embodiments, in order to achieve at least partial overlap of the shooting areas of the two first cameras in the camera combination shown in the above-mentioned Embodiment 1 and Embodiment 2, the camera combination can be arranged on a movable device, such as a movable working equipment in the application scenario.

[0363] Fig.13 According to some embodiments of the present application, a schematic diagram of a deployment scenario of a camera combination is shown.

[0364] refer to Fig.13 The first camera combination can be arranged on a fixed pole station P10, and the first camera P1 in the first camera combination is arranged at a position farther from the road than the second camera P2 in the group.

[0365] The second camera assembly may be arranged on a fixed pole station P11, and the first camera P3 in the second camera assembly is arranged at a position farther from the road than the second camera P4 in the group.

[0366] Since adding pole stations in various operating areas of the port will involve additional civil engineering and modification of communication power supply lines, the cost of adding pole stations is high. Therefore, the camera combination can be set on a movable operating device to control costs and ensure that the first cameras in adjacent camera combinations can still have at least partial overlapping shooting areas.

[0367] Understandably, Fig.13 The illustrated on-site bridge crane G10 can be moved to a designated container stacking position to lift the container onto a container truck, which then transports the container. Therefore, a third camera assembly can be provided on the on-site bridge crane G10 to capture the container truck transporting the container in real time.

[0368] In some embodiments, the first camera G1 in the third camera combination is set at a position farther from the road than the second camera G2 in the group, so that the third camera combination can shoot in real time whether the container truck transporting the container has violated the regulations, and can fill in the shooting area that cannot be shot by the pole station P10 and the pole station P11. When the bridge crane G10 in the field moves, the overlapping shooting area of ​​the camera G2 and the camera G1 changes continuously. At this time, the electronic device 230 can automatically align the shooting angles of the camera G2 and the camera G1 through steps 501 to 503 above to achieve continuous tracking and identification of the target vehicle.

[0369] Furthermore, since the pole stations are all set up along the road, the angles of the cameras installed on them to shoot the container trucks are relatively fixed, usually shooting in the direction of the road extension. The on-site bridge cranes are constantly moving based on the designated container stacking position, so the above-mentioned third camera combination can shoot more angles of the container trucks, including the side angles of the container trucks, which are different from the shooting angles of the first camera combination and the second camera combination. Therefore, through the first camera G1 in the third camera combination set on the on-site bridge crane, more dimensional vehicle appearance features can be obtained, thereby effectively improving the recognition accuracy of violation events.

[0370] The software structure of the electronic device 230 is described in detail below with reference to the relevant drawings.

[0371] Fig.14A It is a schematic structural block diagram of an electronic device provided according to an embodiment of the present application.

[0372] refer to Fig.14A The electronic device 230 includes: an acquisition unit 1401, a display unit 1402, a processing module 1403 and a reporting unit 1404.

[0373] The acquisition unit 1401 is used to acquire the video streams captured by the first camera and the second camera.

[0374] The display unit 1402 is used to present a plurality of frames of continuous first lane images in a first video stream captured by a first camera in a first display area (not shown in the figure) of a display interface;

[0375] The display unit 1402 is further used to present a plurality of frames of second lane images in a second video stream captured by a second camera in a second display area (not shown in the figure) of the display interface.

[0376] The processing module 1403 is used to identify the violation event based on multiple frames of continuous first lane images, determine the violation time and the type of violation event, track the violating vehicle in the second lane image based on the violation event, determine the identification information of the violating vehicle, and obtain the video stream corresponding to the violation event.

[0377] The reporting unit 1404 is used to report the identification information of the illegal vehicle and the video stream corresponding to the illegal event. For example, the identification information, location information, illegal event type, illegal time and the video stream corresponding to the illegal event of the illegal vehicle can be reported to the traffic management application.

[0378] In some embodiments, the processing module 1403 may include a traffic violation vehicle identification unit 031, a vehicle positioning unit 032, a multi-source comparison migration re-identification unit 033, a license plate recognition unit 034 and a traffic violation output unit 035, so as to realize the mutual calling of different camera video streams, accurately identify the identification information of the traffic violation vehicle, and splice the video stream corresponding to the traffic violation event.

[0379] Combine the following Fig. 14B The processing module 1403 is described in detail.

[0380] Fig. 14B It is a schematic structural block diagram of a processing module in an electronic device provided according to an embodiment of the present application.

[0381] refer to Fig. 14B The processing module 1403 includes a traffic violation vehicle identification unit 031, a vehicle positioning unit 032, a multi-source comparison migration re-identification unit 033, and a license plate recognition unit 034.

[0382] The illegal vehicle identification unit 031 is used to identify illegal events and illegal vehicles. It can be understood that the specific implementation process of the illegal vehicle identification unit 031 can refer to the above steps 301 to 304, and for the sake of simplicity, it will not be repeated here.

[0383] The vehicle positioning unit 032 is used to determine the real-time position information of the target vehicle according to the violation event, for example, to determine that the real-time position of the target vehicle enters the overlapping shooting area of ​​the first camera and the second camera, at which time, multiple frames of the second lane image shot by the second camera can be obtained. It can be understood that the specific implementation process of the vehicle positioning unit 032 can refer to step 306 above, and for the sake of simplicity, it is not repeated here.

[0384] The multi-source comparison migration re-identification unit 033 is used to determine the target vehicle from all vehicles in the multiple frames of the second lane image according to the appearance information and driving behavior characteristics of the target vehicle, or to determine the target vehicle according to the multiple frames of the third lane image taken by the first camera in the adjacent camera combination, or to determine the target vehicle according to the multiple frames of the fourth lane image taken by the second camera in the adjacent camera combination. It can be understood that the specific implementation process of the multi-source comparison migration re-identification unit 033 can refer to step 307 or step 1115 above, and for the sake of brevity, it is not repeated here.

[0385] The license plate recognition unit 034 is used to determine the identification information of the target vehicle according to the multiple frames of images. It can be understood that the specific implementation process of the license plate recognition unit 034 can refer to step 308, step 1110 or step 1113 above, and for the sake of simplicity, it is not repeated here.

[0386] Continue to refer Fig. 14B In some embodiments, the processing module 1403 also includes a violation output unit 035, which is used to output violation events, license plate information and violation videos. The above-mentioned violation events may include the type of violation behavior and the time when the violation occurred, and the above-mentioned violation video is continuous time video information of the violation vehicle.

[0387] In addition, in some other embodiments, when solving the problem that it is difficult to accurately identify illegal vehicles due to the lack of a positioning device, the first camera in the above scheme can be replaced by a radar. For example, the point cloud data can be obtained by the radar to determine the illegal event of the vehicle, thereby triggering the tracking of the illegal vehicle. For example, the appearance characteristics of the illegal vehicle, such as the vehicle model, can be identified from the point cloud data detected by the radar, and the driving behavior characteristics of the illegal vehicle, such as the driving speed and lane change information, can be identified from the point cloud data. Then, the second camera with a smaller shooting angle is controlled to shoot multiple frames of the second lane image. The second camera has a clearer image and can identify the identification information of the illegal vehicle, such as the license plate number, the vehicle unique identification code, etc. In addition, the shooting area of ​​the second camera overlaps at least partially with the detection area of ​​the above radar. By comparing the appearance characteristics and driving behavior characteristics of the illegal vehicle identified based on the point cloud data, the illegal vehicle is tracked and identified again from the second camera, which can ensure that the vehicle in the second lane image and the vehicle in the point cloud data are the same illegal vehicle, thereby accurately determining the illegal event and the illegal vehicle.

[0388] In some embodiments, the radar may be a millimeter wave radar or a laser radar. The advantage of radar is that it has strong penetration of rain and fog. Therefore, using radar to replace the first camera can prevent bad weather from affecting the identification of traffic violation events.

[0389] Taking LiDAR as an example, laser point cloud data is obtained by LiDAR emitting laser signals to the surroundings and then collecting the reflected laser signals. The distance information of the target object can be measured by information such as the speed of light and the time from the laser emission to the return, and then the three-dimensional coordinate information and distance information of the target in front can be calculated by combining map information and environmental information. Each point of the point cloud data collected by LiDAR contains three-dimensional coordinate information, and can also contain reflection intensity information, echo number information, etc. Therefore, by collecting point cloud data corresponding to the vehicles on the road in the detection area through LiDAR, the real-time location information, vehicle model information, and height information of the vehicle can be obtained, and the corresponding traffic sign information on the map data can be used to determine the violation. For example, illegal parking, speeding and other violations can be effectively identified.

[0390] It can be understood that the corresponding driving characteristics and appearance characteristics of the illegal vehicle can be collected through point cloud data.

[0391] For example, the point cloud data collected in time sequence can determine the displacement information of the illegal vehicle, and the displacement information can be combined with the map information to determine the lane change information of the vehicle. Then, the lane change information generated by the illegal vehicle in time sequence can be obtained to obtain the lane change information sequence.

[0392] Therefore, multiple appearance parameters and driving behavior parameters of the illegal vehicle contained in the point cloud data can be clustered as reference features, thereby facilitating the identification of the same illegal vehicle in multiple frames of the second lane image taken by the second camera.

[0393] In other embodiments, the first camera and the radar can be set together at the same position, and the detection range of the radar and the shooting range of the second camera can overlap at least partially. In bad weather, the radar can be used to detect point cloud data to determine the violation event, the appearance of the violating vehicle and the driving behavior characteristics. In good weather, the first camera can be used to shoot the first road image to detect the violation event, the appearance of the violating vehicle and the driving behavior characteristics, thereby adapting to the detection of violation behaviors in different weather environments.

[0394] The software structure of the camera mentioned above is described in detail below with reference to the relevant drawings.

[0395] Fig.15 is a schematic structural block diagram of a camera provided according to an embodiment of the present application. Fig.15 The camera 1500 shown includes a camera unit 1501 , a processing unit 1502 , and a communication unit 1503 .

[0396] Understandably, Fig.15 The software structure of the camera 1500 shown may be the schematic software structure of the first camera or the second camera mentioned above.

[0397] The camera unit 1501 is used to capture multiple frames of continuous images of the road.

[0398] The processing unit 1502 is used to remove dynamic information from the multiple frames of continuous images obtained by shooting, and determine the background image.

[0399] The communication unit 1503 is used to send the time of each frame of the image captured to the electronic device 230 mentioned above via wired and / or wireless means, so that the electronic device 230 can identify the violation event and the illegal vehicle based on the multiple frames of continuous images captured.

[0400] In some embodiments, the camera 1500 includes a first camera and a second camera, and the shooting parameters of the first camera are different from the shooting parameters of the second camera, but the shooting area of ​​the first camera and the shooting area of ​​the second camera are at least partially overlapped, so that the electronic device 230 can identify the same illegal vehicle based on the images captured by the first camera and the second camera, avoiding mistaking vehicles with similar appearance for illegal vehicles.

[0401] In other embodiments, part or all of the functions performed by the processing module 1403 in the electronic device 230 may be performed by the processing unit 1502 in the camera 1500 .

[0402] The hardware structure of the electronic device mentioned above is described in detail below with reference to the relevant drawings.

[0403] Fig.16 is a hardware structure block diagram of an electronic device provided according to an embodiment of the present application. Fig.16 The electronic device 230 shown includes a processor 1601 , a memory unit 1602 and a storage medium 1603 .

[0404] The processor 1601 , the memory unit 1602 , and the storage medium 1603 may communicate with each other via a bus 1604 .

[0405] Processor 1601 is the control center of electronic device 230. In some embodiments, processor 1601 includes one or more central processor units (CPUs). Processor 1601 can be a single-core (single CPU) processor or a multi-core (multi-CPU) processor. Unless otherwise specified, components such as processors or memories used to perform tasks can be implemented as general components temporarily configured to perform tasks at a given time or specific components manufactured to perform tasks. The term "processor" as used herein refers to one or more devices or circuits. Processor 1601 can also be other general-purpose processors, such as graphics processing units (GPUs), system on chips (SoCs), digital signal processors (DSPs), or any combination thereof.

[0406] The program code executed by the CPU of the processor 1601 may be stored in the memory unit 1602 or the storage medium 1603. In some embodiments, the program code, such as the kernel, the program to be debugged, etc., may be stored in the storage medium 1603 and copied to the memory unit 1602 for execution by the processor 1601. The processor 1601 controls the execution of other programs or processes, controls the communication with peripheral devices, and controls the use of data processing device resources, thereby controlling the operation of the electronic device 230, thereby implementing the operation steps of the above-mentioned vehicle identification method.

[0407] The lane images such as the first lane image and the second lane image required in the above method may be stored in the storage medium 1603 .

[0408] In addition to the data bus, the bus 1604 may also include a power bus, a control bus, a status signal bus, etc. However, for the sake of clarity, various buses are labeled as the bus 1604 in the figure.

[0409] In some embodiments, the electronic device 230 further includes a communication interface (not shown in the figure), which is used to realize communication between the electronic device 230 and an external device or equipment. For example, the electronic device 230 can receive the original road image taken by the camera through the communication interface and send the determined configuration information to the camera.

[0410] It can be understood that the above and other operations and / or functions of each module in the electronic device 230 are respectively for implementing the corresponding processes of the above method, and for the sake of brevity, they are not described here in detail.

[0411] The hardware structure of the camera mentioned above is described in detail below with reference to the relevant drawings.

[0412] Fig.17 is a schematic hardware structure block diagram of a camera provided according to an embodiment of the present application. Fig.17 The camera 1700 shown includes a camera module 1710 and an image processor 1720 communicatively connected to the camera module 1710 .

[0413] Understandably, Fig.17 The hardware structure of the camera 1700 shown may be the schematic hardware structure of the first camera or the second camera mentioned above.

[0414] The camera module 1710 is used to obtain image data and input the image data into the image processor 1720 so that the image processor 1720 processes the image data. The communication connection between the camera module 1710 and the image processor 1720 may include data transmission through electrical connection methods such as wiring, or data transmission may be achieved through coupling and other methods. It is understandable that the camera module 1710 and the image processor 1720 may also achieve communication connection through other methods that can achieve data transmission.

[0415] The function of the image processor 1720 is to optimize the digital image signal through a series of complex mathematical algorithms, and finally transmit the processed signal to the display. The image processor 1720 can be an image processing chip or a digital signal processing chip (digital signal processor, DSP), which is used to transmit the data obtained by the photosensitive chip to the central processor in a timely and fast manner and refresh the photosensitive chip. Therefore, the quality of the DSP chip directly affects the picture quality, such as color saturation, clarity, etc.

[0416] In some embodiments, the camera 1700 further includes an analog-to-digital converter 1730. The analog-to-digital converter 1730 is connected between the camera module 1710 and the image processor 1720. The analog-to-digital converter 1730 is used to convert the signal generated by the camera module 1710 into a digital image signal and transmit it to the image processor 1720, and then the digital image signal is processed by the image processor 1720, and finally the image or video is displayed on the display screen or monitor.

[0417] In some embodiments, the camera 1700 also includes a memory 1740, which is communicatively connected to the image processor 1720. The image processor 1720 processes the image digital signal and then transmits the image to the memory 1740, so that when the image needs to be viewed later, the image can be retrieved from the storage at any time and displayed on the display screen.

[0418] In some embodiments, the image processor 1720 may further compress the processed image digital signal and store it in the memory 1740 to save space in the memory 1740 .

[0419] The camera module 1710 includes an optical lens 1711, a photosensitive element 1712 and a color filter array 1713. The photosensitive element 1712 is located on the image side of the optical lens 1711, and when the camera module 1710 is working, the scene to be imaged passes through the optical lens 1711 and is imaged on the photosensitive element 1712.

[0420] Specifically, the working principle of the camera module 1710 is as follows: the light L reflected by the photographed scene generates an optical image through the optical lens 1711 and is projected onto the surface of the photosensitive element 1712. The photosensitive element 1712 converts the optical image into an electrical signal, namely an analog image signal S1, and transmits the converted analog image signal S1 to the analog-to-digital converter 1730, so as to be converted into a digital image signal S2 through the analog-to-digital converter 1730 and given to the image processor 1720.

[0421] The photosensitive element 1712 is a semiconductor chip with hundreds of thousands to millions of photodiodes on its surface. When exposed to light, it generates electric charge, which is converted into digital signals through the analog-to-digital converter 1730 chip. The photosensitive element 1712 can be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The charge coupled device photosensitive element 1712 CCD is made of a highly sensitive semiconductor material, which can convert light into electric charge, which is converted into digital signals through the analog-to-digital converter 1730 chip. CCD is composed of many photosensitive units. When the CCD surface is exposed to light, each photosensitive unit will reflect the charge on the component, and the signals generated by all the photosensitive units are added together to form a complete picture. Complementary metal oxide semiconductor CMOS mainly uses semiconductors made of two elements, silicon and germanium, so that semiconductors with N (with negative charge) and P (with positive charge) levels coexist on CMOS. The current generated by these two complementary effects can be recorded and interpreted into images by the processing chip.

[0422] The camera 1700 may also include a communication unit 1750 for sending all images obtained by the image processor 1720 to an electronic device. The communication unit 1750 may also be used to send images of traffic violations to an electronic device or a server. Images of traffic violations may also be stored in the memory 1740. The communication unit 1750 may also receive an image call instruction sent by an electronic device and send the historical images stored in the memory 1740 to the electronic device 230.

[0423] It can be understood that the camera 1700 according to the embodiment of the present application may correspond to the camera 1500 in the embodiment of the present application, and the above and other operations and / or functions of each module in the camera 1700 are respectively for implementing the corresponding processes of the above method. For the sake of brevity, they are not elaborated here.

[0424] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction or program code in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0425] It should be noted that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or an instruction or program code in the form of software. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to be executed, or a combination of hardware and software modules in the decoding processor to be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0426] It is understood that the memory in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memory. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of exemplary but not limiting explanation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous connection dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM). It should be noted that the memory of the system and method described herein is intended to include but is not limited to these and any other suitable types of memory.

[0427] According to the method provided in the embodiments of the present application, the present application also provides a computer program product, which includes: computer program code, when the computer program code is executed on a computer, the computer implements the steps performed by the electronic device in any one of the above embodiments.

[0428] According to the method provided in the embodiment of the present application, the present application also provides a computer program product, which includes: computer program code, when the computer program code is executed on a computer, the computer implements the steps performed by the camera in any one of the above embodiments.

[0429] According to the method provided in the embodiments of the present application, the present application also provides a computer-readable medium, which stores program code. When the program code runs on a computer, the computer implements the steps performed by the electronic device in any one of the above embodiments.

[0430] According to the method provided in the embodiment of the present application, the present application also provides a computer-readable medium, which stores a program code. When the program code runs on a computer, the computer implements the steps performed by the camera in any one of the above embodiments.

[0431] According to the method provided in the embodiment of the present application, the present application also provides a system, which includes the aforementioned electronic device and at least one camera.

[0432] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0433] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0434] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0435] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions or program codes to enable an electronic device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0436] References to "one embodiment" or "an embodiment" in the specification mean that the specific features, structures, or characteristics described in conjunction with the embodiment are included in at least one exemplary implementation or technology disclosed in the present application. The appearance of the phrase "in one embodiment" in various places in the specification does not necessarily all refer to the same embodiment.

[0437] The present application also discloses a device for performing the operations in the text. The device can be constructed specifically for the required purpose or it can include a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program can be stored in a computer-readable medium, such as, but not limited to, any type of disk, including a floppy disk, an optical disk, a CD-ROM, a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), an EPROM, an EEPROM, a magnetic or optical card, an application-specific integrated circuit (ASIC) or any type of medium suitable for storing electronic instructions, and each can be coupled to a computer system bus. In addition, the computer mentioned in the specification may include a single processor or may be an architecture involving multiple processors for increased computing power.

[0438] The processes and displays presented herein are not inherently related to any specific computer or other device. Various general purpose systems may also be used in conjunction with the programs according to the teachings herein, or it may prove convenient to construct more specialized devices to perform one or more method steps. The structures for various of these systems are discussed in the following description. In addition, any specific programming language sufficient to implement the techniques and embodiments disclosed in this application may be used. Various programming languages ​​may be used to implement the present disclosure, such as the vehicle identification method discussed herein.

[0439] Additionally, the language used in this specification has been primarily selected for readability and instructional purposes and may not have been selected to delineate or limit the disclosed subject matter.Accordingly, this disclosure is intended to illustrate but not to limit the scope of the concepts discussed herein.

Claims

1. A vehicle identification method, characterized in that: The method comprises: Determining, through first detection data of a first device, that a first target vehicle has a traffic violation; Acquire a plurality of first images, wherein a first detection area of ​​the first device overlaps with a shooting area of ​​the first image; According to a first feature of the first target vehicle in the first detection data, identification information of the first target vehicle in the first image is identified, wherein the first feature is used to characterize the appearance and driving behavior of the first target vehicle.

2. The method according to claim 1, characterized in that The first device includes a first camera, and determining that the first target vehicle has a traffic violation event through first detection data of the first device includes: A second image is captured by the first camera, and a traffic violation event is determined for the first target vehicle based on the second image.

3. The method according to claim 2, characterized in that The shooting focal length of the second image is smaller than the shooting focal length of the first image.

4. The method according to claim 2, characterized in that: The second image is captured by a first camera disposed at a first position, the first image is captured by a second camera disposed at a second position, and a first capturing area of ​​the first camera overlaps with a second capturing area of ​​the second camera; Furthermore, the first position and the second position have a first corresponding relationship, wherein the first corresponding relationship includes: The first position and the second position are at equal vertical distances from the first horizontal plane; or, A vertical distance between the first position and the first horizontal plane is greater than a vertical distance between the second position and the first horizontal plane.

5. The method according to claim 4, characterized in that The first corresponding relationship also includes: The first position and the second position are located on the same fixture; or, The first position and the second position are located at different fixing devices.

6. The method according to claim 4, characterized in that The first corresponding relationship also includes: The first position and the second position are located on the same mobile working equipment.

7. The method according to claim 4, characterized in that The third camera having a positional relationship with the first camera is located at a third position, and the first camera and the second camera have a first shooting overlapping area, and the first camera and the third camera have a second shooting overlapping area.

8. The method according to claim 7, characterized in that The acquiring of the first image comprises: Determining, based on the plurality of second images, that the first target vehicle enters the first shooting overlapping area; Acquire a plurality of first images captured by the second camera.

9. The method according to claim 8, characterized in that The acquiring of the plurality of first images captured by the second camera further includes: Determine that the first target vehicle enters the first shooting overlapping area along a first driving direction, and obtain a plurality of first images shot in real time by the second camera, wherein the first driving direction is used to indicate that the first target vehicle drives from a first shooting area of ​​the first camera to a second shooting area of ​​the second camera; Determine that the first target vehicle enters the first shooting overlapping area along a second driving direction, and obtain multiple first images historically taken by the second camera, wherein the second driving direction is used to indicate that the first target vehicle drives from the second shooting area of ​​the second camera to the first shooting area of ​​the first camera.

10. The method according to claim 7, characterized in that A fourth camera that has a positional relationship with the third camera is located at a fourth position, and the third camera and the fourth camera have a third shooting overlapping area.

11. The method according to claim 10, characterized in that The method further comprises: Determining, based on the plurality of third images captured by the first camera, that a second target vehicle in the third image has a traffic violation; Determining that a second target vehicle enters the second shooting overlapping area according to the plurality of third images captured by the first camera; Acquire a plurality of fourth images taken by the third camera; identifying the second target vehicle in the fourth image according to the second feature of the second target vehicle in the third image; Determining, according to the fourth image, that the second target vehicle enters the third shooting overlapping area; Acquire a plurality of fifth images taken by the fourth camera; According to a second feature of the second target vehicle in the third image, identification information of the second target vehicle in the fifth image is identified, wherein the second feature is used to characterize the appearance and driving behavior of the second target vehicle.

12. The method according to claim 7, characterized in that The identifying information of the first target vehicle in the first image according to the first feature of the first target vehicle in the second image includes: Determining a first vehicle to be compared in the first image; According to a first feature of the first target vehicle in the second image, identifying a first target vehicle from the first vehicles to be compared; Determine identification information of the first target vehicle.

13. The method according to claim 11, characterized in that The step of identifying the second target vehicle in the fourth image according to the second feature of the second target vehicle in the third image comprises: determining a second vehicle to be compared in the fourth image; A second target vehicle is identified from the second vehicles to be compared according to the second feature.

14. The method according to claim 11, characterized in that The step of identifying identification information of the second target vehicle in the fifth image according to the second feature of the second target vehicle in the fifth image includes: determining a third vehicle to be compared in the fifth image; Identify a second target vehicle from the third vehicles to be compared according to the second feature; Determine identification information of the second target vehicle.

15. The method according to claim 2, characterized in that The method of acquiring the first feature of the first target vehicle in the second image includes: Determining a first appearance parameter and a first driving behavior parameter of the first target vehicle according to the plurality of second images, wherein the first appearance parameter includes a color of the first target vehicle and a model of the first target vehicle, and the first driving behavior parameter includes a lane change information sequence of the first target vehicle and a speed of the first target vehicle; The first appearance parameter and the first driving behavior parameter are clustered as a first feature of the first target vehicle.

16. The method according to claim 15, characterized in that The method for acquiring the lane change information sequence of the first target vehicle includes: The lanes occupied by the first target vehicle are recorded in time sequence according to the plurality of second images to obtain a lane change information sequence of the first target vehicle.

17. The method according to claim 11, characterized in that The method for acquiring the second feature of the second target vehicle in the third image includes: Determining a second appearance parameter and a second driving behavior parameter of the second target vehicle according to the plurality of third images, wherein the second appearance parameter includes a color of the second target vehicle and a model of the second target vehicle, and the second driving behavior parameter includes a lane change information sequence of the second target vehicle and a speed of the second target vehicle; The second appearance parameter and the second driving behavior parameter are clustered into a second feature of the second target vehicle.

18. The method according to claim 17, characterized in that The method for acquiring the lane change information sequence of the second target vehicle includes: The lane occupied by the second target vehicle is recorded in time sequence according to the plurality of third images to obtain a lane change information sequence of the second target vehicle.

19. The method according to claim 7, characterized in that The acquiring of a plurality of first images captured by the first camera comprises: According to the image correspondence between the second image and the first image, the plurality of first images are subjected to image registration to obtain a plurality of first images after image registration.

20. The method according to claim 19, characterized in that The method for acquiring the image correspondence relationship between the second image and the first image includes: Determine a first background image according to the plurality of second images, wherein the first background image does not contain dynamic information; Determine a second background image according to the plurality of first images, wherein the second background image does not contain dynamic information; An image correspondence relationship between the second image and the first image is determined according to the first background image and the second background image, wherein the image correspondence relationship includes a perspective mapping coefficient.

21. The method according to claim 12, characterized in that The acquiring of the plurality of first images captured by the second camera further includes: According to the time correspondence between the second image and the first image, a plurality of first images that are time-aligned with each second image are obtained.

22. The method according to claim 21, characterized in that The time correspondence between the second image and the first image includes a first imaging error time between the second image and the first image, and a method for acquiring the first imaging error time includes: A first imaging error time between the first camera and the second camera is determined according to the imaging timestamp of the first camera and the imaging timestamp of the second camera.

23. The method according to claim 19, characterized in that The method further comprises: Stitching the plurality of second images and the plurality of first images after image registration to obtain a plurality of first stitched images; The video stream captured by the first camera and the video stream captured by the second camera are spliced ​​together according to the multiple first spliced ​​images to obtain a traffic violation video of the first target vehicle.

24. The method according to claim 11, characterized in that The acquiring a plurality of fourth images captured by the third camera comprises: According to the image correspondence between the third image and the fourth image, the plurality of fourth images are subjected to image registration to obtain a plurality of fourth images after image registration.

25. The method according to claim 24, characterized in that The acquiring a plurality of fifth images captured by the fourth camera comprises: According to the image correspondence between the fourth image and the fifth image, the plurality of fifth images are subjected to image registration to obtain a plurality of fifth images after image registration.

26. The method according to claim 25, characterized in that The method further comprises: splicing the plurality of third images, the plurality of fourth images after the image registration, and the plurality of fifth images after the image registration to obtain a plurality of second spliced ​​images; The video stream captured by the first camera, the video stream captured by the third camera, and the video stream captured by the fourth camera are spliced ​​according to the multiple second spliced ​​images to obtain the traffic violation video of the second target vehicle.

27. The method according to claim 11, characterized in that The acquiring of a plurality of fourth images captured by the third camera further includes: According to the time correspondence between the third image and the fourth image, a plurality of fourth images that are time-aligned with each third image are obtained.

28. The method according to claim 27, characterized in that The time correspondence between the third image and the fourth image includes a second imaging error time between the third image and the fourth image, and the method for acquiring the second imaging error time includes: An imaging error time between the first camera and the third camera is determined according to the imaging timestamp of the first camera and the imaging timestamp of the third camera.

29. The method according to claim 11, characterized in that The acquiring of a plurality of fifth images captured by the fourth camera further includes: According to the time correspondence between the fourth image and the fifth image, a plurality of fifth images that are time-aligned with each fourth image are obtained.

30. The method according to claim 29, characterized in that The time correspondence between the fourth image and the fifth image includes a third imaging error time between the fourth image and the fifth image, and the method for acquiring the third imaging error time includes: The imaging error time between the third camera and the fourth camera is determined according to the imaging time stamp of the third camera and the imaging time stamp of the fourth camera.

31. The method according to claim 1, characterized in that The first device further includes a first radar, and determining that the first target vehicle has a traffic violation event through the first detection data of the first device includes: It is determined that the first target vehicle has a traffic violation through the first point cloud data detected by the first radar.

32. The method according to claim 31, characterized in that The first point cloud data is detected by a first radar disposed at a fifth position, the first image is captured by a second camera disposed at a second position, and a first detection area of ​​the first radar overlaps with a second capturing area of ​​the second camera; Furthermore, the fifth position and the second position have a second corresponding relationship, wherein the second corresponding relationship includes: The fifth position and the second position are at an equal vertical distance from the second horizontal plane; or, A vertical distance between the fifth position and the second horizontal plane is greater than a vertical distance between the second position and the second horizontal plane.

33. A vehicle identification system, characterized in that: comprising a first device and a second camera, wherein the first device comprises a first camera, and The first camera is set at a first position, the second camera is set at a second position, and a first shooting area of ​​the first camera overlaps with a second shooting area of ​​the second camera; Furthermore, the first position and the second position have a first corresponding relationship, wherein the first corresponding relationship includes: The first position and the second position are at equal vertical distances from the first horizontal plane; or, A vertical distance between the first position and the first horizontal plane is greater than a vertical distance between the second position and the first horizontal plane.

34. The system according to claim 33, characterized in that The system further comprises a third camera and a fourth camera, wherein: The third camera is set at a third position, the fourth camera is set at a fourth position, and the third shooting area of ​​the third camera overlaps with the fourth shooting area of ​​the fourth camera; Furthermore, the third position and the fourth position have a second corresponding relationship, wherein the second corresponding relationship includes: The third position and the fourth position are at an equal vertical distance from the second horizontal plane; or, A vertical distance between the third position and the second horizontal plane is greater than a vertical distance between the fourth position and the second horizontal plane.

35. The system according to claim 34, characterized in that The third shooting area of ​​the third camera overlaps with the first shooting area of ​​the first camera.

36. The system according to claim 35, characterized in that The third photographing area and the fourth photographing area overlap to the first photographing area along a first direction; or, The third photographing region and the fourth photographing region overlap the first photographing region along a second direction different from the first direction.

37. The system according to claim 33, characterized in that The first device further includes a first radar, wherein the first device includes the first radar, and, The first radar is set at a fifth position, the second camera is set at a second position, and a first detection area of ​​the first radar overlaps with a second shooting area of ​​the second camera; Furthermore, the fifth position and the second position have a second corresponding relationship, wherein the second corresponding relationship includes: The fifth position and the second position are at an equal vertical distance from the second horizontal plane; or, A vertical distance between the fifth position and the second horizontal plane is greater than a vertical distance between the second position and the second horizontal plane.

38. An electronic device, characterized in that: include: one or more processors; One or more memories; the one or more memories store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device executes the vehicle identification method described in any one of claims 1 to 32.

39. A computer-readable storage medium, characterized in that: The storage medium stores instructions, which, when executed on a computer, cause the computer to execute the vehicle identification method according to any one of claims 1 to 32.

Citation Information

Cited By

  • Method and device for determining vehicle attribute information, medium and program product

    CN119851258A