Vehicle image fusion method, device, computer equipment and storage medium
By performing time-related fusion processing on the front, body and rear pictures of the vehicle, the accuracy of vehicle identification is improved, and the problem of not being able to determine whether it is the same car when the vehicle types are similar is solved.
Patent Information
- Application Number
- CN202210438627.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-25
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2042-04-25
AI Technical Summary
In vehicle identification, the fusion results of the same type of vehicle are similar, and it is impossible to determine whether it is the same vehicle, resulting in low recognition accuracy.
By obtaining multiple pictures taken by the first camera on the front of the vehicle and the pictures taken by the second camera on the body, fuse them according to the shooting time to obtain the first fusion result; at the same time, obtaining the pictures taken by the third camera on the rear of the vehicle, and according to the shooting time, Fusion is carried out with the front and rear pictures of the vehicle to obtain the second fusion result to improve the accuracy of vehicle identification.
The vehicle body picture is fused with the front picture corresponding to the car body, and the vehicle rear picture is fused with the body picture of the fused vehicle front picture to improve the accuracy of vehicle recognition, solving the problem that it is impossible to determine whether it is the same car when the vehicle types are similar.
Smart Images

Figure CN114897685B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of electronic technology, and in particular to a vehicle image fusion method, apparatus, computer equipment, storage medium and computer program product. Background Art
[0002] With the development of electronic technology, in the current toll lane application, the entrance overtaking rejection uses independent cameras installed in different positions to collect pictures and videos. After processing the collected pictures and images, they are imported into the established vehicle fusion model to obtain the vehicle fusion results, so as to determine whether it is the same vehicle. However, for many vehicles of the same type, the calculated fusion results are similar, and it is impossible to determine whether they are the same vehicle. Summary of the invention
[0003] Based on this, it is necessary to provide a vehicle image fusion method, device, computer equipment, computer-readable storage medium and computer program product that can improve the accuracy of vehicle recognition in response to the above technical problems.
[0004] In a first aspect, the present application provides a vehicle image fusion method. The method comprises:
[0005] Acquire multiple front images of different vehicles captured by the first camera and a target vehicle body image captured by the second camera;
[0006] According to the shooting time of each picture, the target vehicle body picture and the multiple vehicle front pictures are fused to obtain a first fusion result, wherein the first fusion result is used to represent the target vehicle front picture corresponding to the target vehicle body picture, and the target vehicle front picture is one of the multiple vehicle front pictures;
[0007] Acquire a plurality of rear end images of different vehicles taken by a third camera, wherein the first camera, the second camera and the third camera are sequentially arranged along a traveling direction of the vehicle;
[0008] According to the shooting time of each picture, the first fusion result and multiple rear-end pictures are fused to obtain a second fusion result, wherein the second fusion result is used to represent the target front picture and the target rear picture corresponding to the target body picture, and the target rear picture is one of the multiple rear-end pictures.
[0009] In one embodiment, the step of fusing the target vehicle body image and the plurality of vehicle head images according to the shooting time of each image to obtain a first fusion result includes:
[0010] A vehicle front picture that meets a first condition is selected from the multiple vehicle front pictures as the target vehicle front picture, where the first condition is related to the shooting time of each picture.
[0011] In one embodiment, the first condition includes at least one of the following:
[0012] The shooting time of the target vehicle front image is less than the shooting time of the target vehicle body image;
[0013] a first difference between the shooting time of the target vehicle front image and the shooting time of the target vehicle body image is greater than a second difference between the shooting time of any other vehicle front image and the shooting time of the target vehicle body image, wherein the other vehicle front images are images other than the target vehicle front image among the multiple vehicle front images;
[0014] A third difference between the shooting time of the target vehicle body image and the shooting time of the target vehicle front image is greater than a first preset ratio, the first preset ratio is a ratio of the first camera distance to the vehicle speed corresponding to the target vehicle body image, and the first camera distance is an actual distance of the scene between the first camera and the second camera;
[0015] The shooting time of the target vehicle front image is greater than the shooting time of the previous vehicle body image of the target vehicle body image.
[0016] In one embodiment, before selecting a vehicle front picture satisfying a first condition from the plurality of vehicle front pictures as the target vehicle front picture, the method includes:
[0017] Performing logical filtering on the multiple vehicle front photos to filter out vehicle front photos with logical errors;
[0018] Eliminate the vehicle front picture with logical errors from the multiple vehicle front pictures.
[0019] In one embodiment, the fusing the first fusion result with a plurality of rear-end images according to the shooting time of each image to obtain a second fusion result includes:
[0020] A vehicle rear picture satisfying a second condition is selected from the plurality of vehicle rear pictures as the target vehicle rear picture, where the second condition is related to the shooting time of each picture.
[0021] In one embodiment, the second condition includes at least one of the following:
[0022] The license plate information of the target vehicle rear image is consistent with the license plate information of the target vehicle front image corresponding to the target vehicle body image;
[0023] The shooting time of the target vehicle rear image is greater than the splicing time of the target vehicle body image;
[0024] A fourth difference between the shooting time of the target rear vehicle image and the stitching time of the target vehicle body image is greater than a second preset ratio, the second preset ratio is a ratio of the second camera distance to the vehicle speed corresponding to the target vehicle body image, and the second camera distance is the actual distance of the scene between the second camera and the third camera.
[0025] In a second aspect, the present application also provides a vehicle image fusion device. The device comprises:
[0026] The image acquisition module is used to acquire a plurality of vehicle front images obtained by taking the front images of different vehicles by the first camera and a target vehicle body image obtained by taking the vehicle body of the target vehicle by the second camera;
[0027] a fusion module, used for fusing the target vehicle body image and the multiple vehicle front images according to the shooting time of each image, so as to obtain a first fusion result, wherein the first fusion result is used to represent the target vehicle front image corresponding to the target vehicle body image, and the target vehicle front image is one of the multiple vehicle front images;
[0028] The image acquisition module is also used to acquire a plurality of rear end images of different vehicles taken by a third camera, wherein the first camera, the second camera and the third camera are arranged in sequence along the traveling direction of the vehicle;
[0029] The fusion module is also used to fuse the first fusion result and multiple rear-end pictures according to the shooting time of each picture to obtain a second fusion result, wherein the second fusion result is used to represent the target front picture and the target rear picture corresponding to the target body picture, and the target rear picture is one of the multiple rear-end pictures.
[0030] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the vehicle image fusion method as described in the first aspect is implemented.
[0031] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the vehicle image fusion method as described in the first aspect is implemented.
[0032] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the vehicle image fusion method described in the first aspect is implemented.
[0033] The vehicle image fusion method, device, computer equipment, storage medium and computer program product are as follows: a plurality of vehicle front images obtained by taking the vehicle front of different vehicles by a first camera and a target vehicle body image obtained by taking the vehicle body of the target vehicle by a second camera; the target vehicle body image and the plurality of vehicle front images are fused according to the shooting time of each image to obtain a first fusion result, the first fusion result is used to represent the target vehicle front image corresponding to the target vehicle body image, and the target vehicle front image is one of the plurality of vehicle front images; a plurality of vehicle rear images obtained by taking the vehicle rear of different vehicles by a third camera, the first camera, the second camera and the third camera are sequentially arranged along the vehicle travel direction; the first fusion result and the plurality of vehicle rear images are fused according to the shooting time of each image to obtain a second fusion result, the second fusion result is used to represent the target vehicle front image and the target vehicle rear image corresponding to the target vehicle body image, and the target vehicle rear image is one of the plurality of vehicle rear images. The vehicle front image corresponding to the vehicle body is fused with the vehicle body image, and the corresponding vehicle rear image is fused according to the vehicle body image fused with the vehicle front image, so as to improve the accuracy of vehicle recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 A diagram showing an application environment of a vehicle image fusion method in an embodiment;
[0035] Figure 2 A schematic diagram of a process of a vehicle image fusion method in one embodiment;
[0036] Figure 3 A schematic diagram of a flow chart of a vehicle image fusion method in another embodiment;
[0037] Figure 4 is a structural block diagram of a vehicle image fusion device in one embodiment;
[0038] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0040] The vehicle image fusion method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the first camera 101, the second camera 102 and the third camera 103 are placed in sequence according to the direction of the vehicle, and are used to shoot the front, body and rear of the vehicle respectively. The first camera 101, the second camera 102 and the third camera 103 communicate with the main control unit 104 through the network respectively. The data storage system can store the data that the main control unit 104 needs to process. The data storage system can be integrated on the main control unit 104, or it can be placed on the cloud or other network servers. The main control unit 104 obtains a plurality of front images taken by the first camera 101 from the front of different vehicles and a target body image taken by the second camera 102 from the body of the target vehicle; according to the shooting time of each image, the target body image and the plurality of front images are fused to obtain a first fusion result, the first fusion result is used to indicate the target front image corresponding to the target body image, and the target front image is one of the plurality of front images; obtains a plurality of rear images taken by the third camera 103 from the rear of different vehicles, the first camera 101, the second camera 102 and the third camera 103 are sequentially arranged along the direction of travel of the vehicle; according to the shooting time of each image, the first fusion result and the plurality of rear images are fused to obtain a second fusion result, the second fusion result is used to indicate the target front image and the target rear image corresponding to the target body image, and the target rear image is one of the plurality of rear images. Among them, the terminal 102 can be but is not limited to an image shooting device, and the main control unit 104 can be implemented by an independent server or a server cluster composed of multiple servers.
[0041] In one embodiment, Figure 2 As shown, a vehicle image fusion method is provided, which is applied to Figure 1 Taking the main control unit 104 in FIG. 1 as an example, the method includes the following steps:
[0042] Step 202 , obtaining a plurality of vehicle front images obtained by taking the front images of different vehicles by the first camera and a target vehicle body image obtained by taking the vehicle body of the target vehicle by the second camera.
[0043] Among them, the first camera refers to a front camera used to identify the front part of the vehicle in the area of the vehicle to be detected. The second camera refers to a side camera used to identify the body of the vehicle in the vehicle area to be detected. When the vehicle passes the front camera, the front camera shoots the front of the vehicle to obtain a picture of the front of the vehicle, as well as the time when the picture of the front of the vehicle was taken. Optionally, the front camera can also recognize the license plate of the vehicle at the same time. When the body of the vehicle passes the second camera, the side camera shoots the body of the vehicle to obtain the picture of the body of the vehicle, as well as the time when the picture of the body of the vehicle was taken. Optionally, the side camera can also identify the body part of the vehicle to obtain the vehicle's driving speed, vehicle model, axle type, and number of axles; the side camera takes the body picture and splices it into a body picture, which also includes: the time when the side camera takes the body picture, and the start time and end time of splicing the body picture.
[0044] Specifically, the main control unit obtains multiple front pictures of different vehicles taken by the front camera and the time when the front pictures of the vehicles were taken; the main control unit obtains the target body picture taken by the side camera of the target vehicle and the time when the body picture was taken by the side camera.
[0045] Step 204 , according to the shooting time of each picture, the target vehicle body picture and the plurality of vehicle front pictures are fused to obtain a first fusion result.
[0046] Among them, fusion can be to process the image data through image processing and computer technology, etc., to extract the beneficial information of each image data to the maximum extent, and finally integrate it into a high-quality image to improve the utilization rate of image information, improve the accuracy and reliability of computer interpretation, and enhance the spatial resolution and spectral resolution of the original image, which is conducive to monitoring. In this embodiment, fusion can be to fuse the image data of the same target vehicle collected by the first camera and the second camera to obtain a first fusion result. Specifically, the first fusion result is used to represent the target vehicle head image corresponding to the target vehicle body image, which can be the same image or the target vehicle body and the target.
[0047] Specifically, the main control unit selects a target front picture that meets the fusion conditions from multiple front pictures according to the shooting time of the pictures taken by the front camera and the side camera, and fuses the target body picture with the target front picture to obtain a target front picture corresponding to the target body picture; wherein the target front picture is one of the multiple front pictures.
[0048] Step 206, obtaining a plurality of rear end images of different vehicles photographed by a third camera, wherein the first camera, the second camera and the third camera are sequentially arranged along the traveling direction of the vehicle.
[0049] Among them, the third camera refers to the rear camera of the vehicle to be detected in the area of the vehicle to be detected, which is used to identify the rear part of the vehicle. When passing the rear camera, the rear camera identifies the license plate information of the rear of different vehicles and takes pictures of the rear of the vehicle, which also includes: the time of taking pictures of the rear of the vehicle. The first camera, the second camera and the third camera are arranged in sequence along the direction of travel of the vehicle. When the vehicle is driving, it first passes the first camera, the first camera takes pictures of the front of the vehicle, then passes the second camera, the second camera takes pictures of the body of the vehicle, and finally passes the third camera, the third camera takes pictures of the rear of the vehicle.
[0050] Specifically, the main control unit obtains the rear images of different vehicles taken by the rear camera, the shooting time of the rear images of different vehicles, and the license plate information of the rear of different vehicles recognized by the rear camera. The front camera, the side camera and the rear camera are arranged in sequence along the direction of travel of the vehicle.
[0051] Step 208: According to the shooting time of each picture, the first fusion result and the plurality of rear-end pictures are fused to obtain a second fusion result.
[0052] Specifically, the main control unit selects target vehicle rear photos that meet the fusion conditions from multiple vehicle rear photos according to the time when the rear ends of different vehicles are photographed by the rear camera, and fuses the target vehicle body photo, the target vehicle front photo and the target vehicle rear photo to obtain the target vehicle front photo and the target vehicle rear photo corresponding to the target vehicle body photo; wherein the target vehicle rear photo is one of the multiple vehicle rear photos.
[0053] In the above vehicle fusion method, a plurality of front images obtained by taking the front of different vehicles by a first camera and a target body image obtained by taking the body of the target vehicle by a second camera; according to the shooting time of each image, the target body image and the plurality of front images are fused to obtain a first fusion result, the first fusion result is used to represent the target front image corresponding to the target body image, and the target front image is one of the plurality of front images; a plurality of rear images obtained by taking the rear of different vehicles by a third camera, the first camera, the second camera and the third camera are sequentially arranged along the direction of travel of the vehicle; according to the shooting time of each image, the first fusion result and the plurality of rear images are fused to obtain a second fusion result, the second fusion result is used to represent the target front image and the target rear image corresponding to the target body image, and the target rear image is one of the plurality of rear images. The front image corresponding to the body is fused with the body image, and the corresponding rear image is fused according to the body image fused with the front image, so as to improve the accuracy of vehicle recognition.
[0054] In one embodiment, the target vehicle body image and the multiple vehicle front images are fused according to the shooting time of each image to obtain a first fusion result, including: selecting a vehicle front image that meets a first condition from the multiple vehicle front images as the target vehicle front image, and the first condition is related to the shooting time of each image.
[0055] The first condition refers to a fusion condition that meets the target vehicle body image and the target vehicle head image corresponding to the target vehicle body.
[0056] Specifically, the main control unit selects a vehicle front picture that meets a first condition from a plurality of vehicle front pictures as a target vehicle front picture, and the first condition is related to the shooting time of each picture.
[0057] In this embodiment, the target vehicle front picture is selected according to the first condition, which can improve the accuracy of selecting the target vehicle front picture.
[0058] In one embodiment, the first condition includes at least one of the following: the shooting time of the target vehicle front image is less than the shooting time of the target vehicle body image; the first difference between the shooting time of the target vehicle front image and the shooting time of the target vehicle body image is greater than the second difference between the shooting time of any other vehicle front image and the shooting time of the target vehicle body image, and the other vehicle front images are multiple vehicle front images other than the target vehicle front image; the third difference between the shooting time of the target vehicle body image and the shooting time of the target vehicle front image is greater than a first preset ratio, and the first preset ratio is a ratio of the first camera distance to the vehicle speed corresponding to the target vehicle body image, and the first camera distance is the actual distance of the scene between the first camera and the second camera; the shooting time of the target vehicle front image is greater than the shooting time of the previous vehicle body image of the target vehicle body image.
[0059] Among them, the shooting time of the target vehicle body image includes: the wheel detection time of the target vehicle body; the shooting time of the target vehicle front image includes: the time of the last frame of the photo taken of the front of the same vehicle; the first camera distance refers to the actual distance of the scene between the vehicle front camera and the vehicle side camera.
[0060] Specifically, the target vehicle body image is merged with the target vehicle front image that meets the first condition, and the first condition at least includes: the time of the target vehicle front image taken by the first camera is less than the time of the target vehicle body image taken by the second camera; the first difference between the time of the target vehicle front image taken by the first camera and the time of the target vehicle body image taken by the second camera is greater than the second difference between the time of other vehicle front images taken by the first camera and the time of the target vehicle body image taken by the second camera, and the other vehicle front images taken by the first camera are images other than the target vehicle front image taken by the first camera among the multiple vehicle front images taken by the first camera;
[0061] The third difference between the shooting time of the target vehicle body image and the shooting time of the target vehicle front image is greater than the first preset ratio, the first preset ratio is the ratio of the first camera distance to the vehicle speed corresponding to the target vehicle body image, and the first camera distance is the actual distance of the scene between the vehicle front camera and the vehicle side camera.
[0062] In one possible embodiment, the difference between the wheel detection time corresponding to the target vehicle body and the end time of the target vehicle head video stream recognition is greater than the ratio of the distance from the vehicle side camera splicing point to the vehicle head recognition area to the vehicle driving speed corresponding to the target vehicle body. The specific formula is:
[0063] T2-T1>S / V
[0064] Among them, T1 represents the time of the last frame of the photo taken of the target vehicle front; T2 represents the wheel detection time corresponding to the target vehicle body; S represents the distance from the splicing point of the vehicle side camera to the vehicle front recognition area (unit: meter); V represents the vehicle speed corresponding to the target vehicle body (unit: km / h).
[0065] The shooting time of the target vehicle front image is greater than the shooting time of the previous vehicle body image of the target vehicle body image.
[0066] In a possible embodiment, the target vehicle head video stream recognition end time is greater than the wheel detection time of the previous vehicle corresponding to the target vehicle body image. The specific formula is:
[0067] T1>T0
[0068] Among them, T0 represents the wheel detection time of the vehicle previous to the vehicle corresponding to the target vehicle body image; T1 represents the end time of the target vehicle head video stream recognition.
[0069] In this embodiment, by limiting the first condition, the accuracy of selecting the target vehicle head image can be improved.
[0070] In one embodiment, before selecting a vehicle front picture that meets a first condition from the multiple vehicle front pictures as the target vehicle front picture, the steps include: performing logical filtering on the multiple vehicle front pictures to screen out vehicle front pictures with logical errors; and removing vehicle front pictures with logical errors from the multiple vehicle front pictures.
[0071] Among them, logical filtering refers to situations that do not conform to actual logic. Logical filtering includes filtering out the license plate type that does not match the vehicle model. For example, if a yellow license plate matches a training car, then if the license plate type is yellow, the training car will not be filtered; if the license plate type is yellow, the sedan will be filtered out.
[0072] Specifically, the main control unit performs logic filtering on the multiple vehicle front photos to screen out the vehicle front photos with logical errors; and then removes the vehicle front photos with logical errors from the multiple vehicle front photos.
[0073] In this embodiment, logically erroneous vehicle head images are deleted through logical filtering, which can save computing resources.
[0074] In one embodiment, the first fusion result and multiple rear-end pictures are fused according to the shooting time of each picture to obtain a second fusion result, including: selecting a rear-end picture that meets a second condition from the multiple rear-end pictures as the target rear-end picture, and the second condition is related to the shooting time of each picture.
[0075] The second condition refers to a fusion condition that meets the target vehicle body image and the target vehicle front image corresponding to the target vehicle body, and then fuses the target vehicle rear image.
[0076] Specifically, the main control unit selects a vehicle rear photo that meets a second condition from a plurality of vehicle rear photos as a target vehicle rear photo, and the second condition is related to the shooting time of each photo.
[0077] In this embodiment, the target vehicle rear end picture is selected according to the first condition, which can improve the accuracy of selecting the target vehicle rear end picture.
[0078] In one embodiment, the second condition includes at least one of the following: the license plate information of the target vehicle rear image is consistent with the license plate information of the target vehicle front image corresponding to the target vehicle body image; the shooting time of the target vehicle rear image is greater than the stitching time of the target vehicle body image; the fourth difference between the shooting time of the target vehicle rear image and the stitching time of the target vehicle body image is greater than the second preset ratio, the second preset ratio is the ratio of the second camera distance to the vehicle speed corresponding to the target vehicle body image, and the second camera distance is the distance between the second camera and the third camera.
[0079] The stitching time of the target vehicle body image refers to the time of stitching the target vehicle body images taken by the second camera. The shooting time of the target vehicle rear image includes the detection time of the wheels in the target vehicle rear image. The second camera distance refers to the distance from the vehicle side camera to the vehicle rear camera.
[0080] Specifically, after the target vehicle body image is fused with the target vehicle front image corresponding to the target vehicle body image, the target vehicle rear image that meets the second condition is fused, and the second condition at least includes: the license plate information of the target vehicle rear image recognized by the third camera is consistent with the license plate information of the target vehicle front image corresponding to the target vehicle body image recognized by the first camera. The shooting time of the target vehicle rear image taken by the third camera is greater than the time for splicing the vehicle body images taken by the second camera. The fourth difference between the time of the target vehicle rear image taken by the third camera and the time of splicing the target vehicle body image taken by the second camera is greater than the second preset ratio, and the second preset ratio is the ratio of the second camera distance to the vehicle speed corresponding to the target vehicle body image recognized by the second camera, and the second camera distance is the actual distance of the scene between the second camera and the third camera.
[0081] In one possible embodiment, the difference between the shooting time in the target vehicle rear image and the stitching time of the target vehicle body image shot by the second camera is greater than the ratio of the distance from the vehicle side camera stitching point to the vehicle front recognition area to the vehicle speed corresponding to the target vehicle body. The specific formula is:
[0082] T1-T0>S / V
[0083] Among them, T0 represents the time when the second camera takes the target vehicle body picture for stitching; T1 represents the time when the target rear vehicle picture is taken; S represents the distance from the side camera to the rear camera (unit: meter); V represents the vehicle speed corresponding to the target vehicle body (unit: km / h).
[0084] In this embodiment, the accuracy of selecting the target vehicle rear image can be improved by limiting the second condition.
[0085] The present application embodiment provides a vehicle image fusion method, and a detailed embodiment is provided below. Figure 3 As shown, the specific steps of vehicle image fusion are described: wherein the first camera is the front camera, the second camera is the side camera, and the third camera is the rear camera.
[0086] Step 301: The front camera takes front pictures of different vehicles, detects and recognizes the license plate numbers through the front license plate recognition technology, and transmits them to the main control unit through the network protocol;
[0087] Step 302: The vehicle side camera takes pictures of the bodies of different vehicles, detects and identifies the vehicle speed, vehicle type, axle type, number of axles, and vehicle side stitching panoramic image information through the vehicle side overall scanning and stitching technology, and transmits it to the main control unit through the network protocol;
[0088] Step 303: the main control unit performs logic filtering on the plurality of vehicle front pictures, and deletes the vehicle front pictures that do not conform to logic;
[0089] Step 304: The main control unit fuses the front image corresponding to the target vehicle body from the multiple front images uploaded by the front camera according to the fusion condition of the vehicle body fusion front; wherein the fusion condition of the vehicle body fusion front includes at least one of the following:
[0090] The time it takes for the front camera to recognize the target vehicle is shorter than the time it takes for the side camera to recognize the wheel detection;
[0091] The difference between the time when the front camera recognizes the target vehicle and the time when the side camera recognizes the wheel detection is greater than the difference between the time when the front camera recognizes other vehicles and the time when the side camera recognizes the wheel detection;
[0092] The difference between the wheel detection time corresponding to the target vehicle body and the end time of the target vehicle head video stream recognition is greater than the ratio of the distance from the vehicle side camera splicing point to the vehicle head recognition area to the vehicle speed corresponding to the target vehicle body. The specific formula is:
[0093] T2-T1>S / V
[0094] Among them, T1 represents the end time of the target vehicle front video stream recognition; T2 represents the wheel detection time corresponding to the target vehicle body; S represents the distance from the vehicle side camera splicing point to the vehicle front recognition area (unit: meter); V represents the vehicle speed corresponding to the target vehicle body (unit: km / h).
[0095] The target vehicle head video stream recognition end time is greater than the wheel detection time of the previous vehicle corresponding to the target vehicle body image. The specific formula is:
[0096] T1>T0
[0097] Among them, T0 represents the wheel detection time of the vehicle previous to the vehicle corresponding to the target vehicle body image; T1 represents the end time of the target vehicle head video stream recognition.
[0098] Step 305, the rear camera takes pictures of the rear of different vehicles, detects and identifies the license plate number and the panoramic view of the rear of the vehicle through the rear license plate recognition technology, and transmits it to the main control unit through the network protocol; the front camera, the side camera and the rear camera use the same NTP server as the NTP time source for time synchronization, and synchronizes the time every 5 minutes to prevent the time of the three cameras from being out of sync.
[0099] Step 306: The main control unit performs logic filtering on the multiple rear-end pictures, and deletes the rear-end pictures that do not conform to logic;
[0100] Step 307: The main control unit fuses the rear image corresponding to the target body from the multiple rear images uploaded by the rear camera according to the fusion condition of the body fusion to the rear. The fusion condition of the body fusion to the rear includes at least one of the following:
[0101] The license plate information of the target vehicle rear image identified by the rear camera is consistent with the license plate information of the target vehicle front image corresponding to the target vehicle body image identified by the side camera;
[0102] The target rear end image taken by the rear camera has a shooting time greater than the time for stitching the vehicle body images taken by the second camera;
[0103] The difference between the detection time of the wheel in the target vehicle rear image and the stitching time of the target vehicle body image taken by the vehicle side camera is greater than the ratio of the distance from the vehicle side camera stitching point to the vehicle front recognition area to the vehicle speed corresponding to the target vehicle body. The specific formula is:
[0104] T1-T0>S / V
[0105] Among them, T0 represents the time for stitching the target vehicle body image taken by the vehicle side camera; T1 represents the time for taking the target vehicle rear image; S represents the distance from the vehicle side camera to the vehicle rear camera (unit: meter); V represents the vehicle speed corresponding to the target vehicle body (unit: km / h).
[0106] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0107] Based on the same inventive concept, the embodiment of the present application also provides a vehicle image fusion device for implementing the vehicle image fusion method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more vehicle image fusion device embodiments provided below can refer to the limitations of the vehicle image fusion method above, and will not be repeated here.
[0108] In one embodiment, Figure 4 As shown, a vehicle image fusion device is provided, including: an image acquisition module 410 and a fusion module 420, wherein:
[0109] The image acquisition module is used to acquire a plurality of vehicle front images obtained by taking the front images of different vehicles by the first camera and a target vehicle body image obtained by taking the vehicle body of the target vehicle by the second camera;
[0110] a fusion module, used to fuse the target vehicle body image and the multiple vehicle front images according to the shooting time of each image, so as to obtain a first fusion result, wherein the first fusion result is used to represent the target vehicle front image corresponding to the target vehicle body image, and the target vehicle front image is one of the multiple vehicle front images;
[0111] The image acquisition module is also used to acquire a plurality of rear end images of different vehicles taken by the third camera, wherein the first camera, the second camera and the third camera are arranged in sequence along the traveling direction of the vehicle;
[0112] The fusion module is also used to fuse the first fusion result and multiple rear-end pictures according to the shooting time of each picture to obtain a second fusion result. The second fusion result is used to represent the target front picture and the target rear picture corresponding to the target body picture. The target rear picture is one of the multiple rear-end pictures.
[0113] In one embodiment, the vehicle image fusion device further includes: a screening module. The screening module is used to select a vehicle front image that meets a first condition from multiple vehicle front images as a target vehicle front image, and the first condition is related to the shooting time of each image.
[0114] In one embodiment, the vehicle image fusion device further includes: a preset condition module. The preset condition module is used for the shooting time of the target vehicle front image to be less than the shooting time of the target vehicle body image; the first difference between the shooting time of the target vehicle front image and the shooting time of the target vehicle body image is greater than the second difference between the shooting time of any other vehicle front image and the shooting time of the target vehicle body image, and the other vehicle front images are multiple vehicle front images other than the target vehicle front image; the third difference between the shooting time of the target vehicle body image and the shooting time of the target vehicle front image is greater than the first preset ratio, and the first preset ratio is the ratio of the first camera distance to the vehicle speed corresponding to the target vehicle body image, and the first camera distance is the actual scene distance between the first camera and the second camera; the shooting time of the target vehicle front image is greater than the shooting time of the previous vehicle body image of the target vehicle body image.
[0115] In one embodiment, the screening module is further used to perform logical filtering on multiple vehicle front photos to screen out vehicle front photos with logical errors; and remove the vehicle front photos with logical errors from the multiple vehicle front photos.
[0116] In one embodiment, the screening module is further used to select a vehicle rear picture that meets a second condition from the plurality of vehicle rear pictures as the target vehicle rear picture, and the second condition is related to the shooting time of each picture.
[0117] In one embodiment, the preset condition module is used to ensure that the license plate information of the target vehicle rear image is consistent with the license plate information of the target vehicle front image corresponding to the target vehicle body image; the shooting time of the target vehicle rear image is greater than the stitching time of the target vehicle body image; the fourth difference between the shooting time of the target vehicle rear image and the stitching time of the target vehicle body image is greater than the second preset ratio, the second preset ratio is the ratio of the second camera distance to the vehicle speed corresponding to the target vehicle body image, and the second camera distance is the actual distance of the scene between the second camera and the third camera.
[0118] Each module in the above vehicle image fusion device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.
[0119] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a vehicle image fusion method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.
[0120] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0121] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.
[0122] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.
[0123] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.
[0124] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.
[0125] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.
[0126] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0127] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A vehicle image fusion method, characterized in that: The method comprises: Acquire multiple front images of different vehicles captured by the first camera and a target vehicle body image captured by the second camera; According to the shooting time of each picture, the target vehicle body picture and the multiple vehicle front pictures are fused to obtain a first fusion result, wherein the first fusion result is used to represent the target vehicle front picture corresponding to the target vehicle body picture, and the target vehicle front picture is one of the multiple vehicle front pictures; Acquire a plurality of rear end images of different vehicles taken by a third camera, wherein the first camera, the second camera and the third camera are sequentially arranged along a traveling direction of the vehicle; According to the shooting time of each picture, the first fusion result and the multiple rear-end pictures are fused to obtain a second fusion result, wherein the second fusion result is used to represent the target front picture and the target rear picture corresponding to the target body picture, and the target rear picture is one of the multiple rear pictures; The step of fusing the target vehicle body image and the plurality of vehicle head images according to the shooting time of each image to obtain a first fusion result includes: A vehicle front picture that meets a first condition is selected from the multiple vehicle front pictures as the target vehicle front picture, where the first condition is related to the shooting time of each picture.
2. The method according to claim 1, characterized in that: The difference between the wheel detection time corresponding to the target vehicle body and the end time of the target vehicle front video stream recognition is greater than the ratio of the distance from the vehicle side camera splicing point to the vehicle front recognition area to the vehicle driving speed corresponding to the target vehicle body.
3. The method according to claim 1, characterized in that The first condition includes at least one of the following: The shooting time of the target vehicle front image is less than the shooting time of the target vehicle body image; a first difference between the shooting time of the target vehicle front image and the shooting time of the target vehicle body image is greater than a second difference between the shooting time of any other vehicle front image and the shooting time of the target vehicle body image, wherein the other vehicle front images are images other than the target vehicle front image among the multiple vehicle front images; A third difference between the shooting time of the target vehicle body image and the shooting time of the target vehicle front image is greater than a first preset ratio, the first preset ratio is a ratio of the first camera distance to the vehicle speed corresponding to the target vehicle body image, and the first camera distance is an actual distance of the scene between the first camera and the second camera; The shooting time of the target vehicle front image is greater than the shooting time of the previous vehicle body image of the target vehicle body image.
4. The method according to claim 1, characterized in that: Before selecting a vehicle front picture satisfying a first condition from the plurality of vehicle front pictures as the target vehicle front picture, the method includes: Performing logical filtering on the multiple vehicle front pictures to filter out vehicle front pictures with logical errors; Eliminate the vehicle front picture with logical errors from the multiple vehicle front pictures.
5. The method according to claim 1, characterized in that The step of fusing the first fusion result with the plurality of rear vehicle images according to the shooting time of each image to obtain a second fusion result includes: A vehicle rear picture satisfying a second condition is selected from the plurality of vehicle rear pictures as the target vehicle rear picture, where the second condition is related to the shooting time of each picture.
6. The method according to claim 5, characterized in that The second condition includes at least one of the following: The license plate information of the target vehicle rear image is consistent with the license plate information of the target vehicle front image corresponding to the target vehicle body image; The shooting time of the target vehicle rear image is greater than the splicing time of the target vehicle body image; A fourth difference between the shooting time of the target rear vehicle image and the stitching time of the target vehicle body image is greater than a second preset ratio, the second preset ratio is a ratio of the second camera distance to the vehicle speed corresponding to the target vehicle body image, and the second camera distance is the actual distance of the scene between the second camera and the third camera.
7. A vehicle detection device, characterized in that: The device comprises: The image acquisition module is used to acquire a plurality of vehicle front images obtained by taking the front images of different vehicles by the first camera and a target vehicle body image obtained by taking the vehicle body of the target vehicle by the second camera; a fusion module, used for fusing the target vehicle body image and the multiple vehicle front images according to the shooting time of each image, so as to obtain a first fusion result, wherein the first fusion result is used to represent the target vehicle front image corresponding to the target vehicle body image, and the target vehicle front image is one of the multiple vehicle front images; The image acquisition module is also used to acquire a plurality of rear end images of different vehicles taken by a third camera, wherein the first camera, the second camera and the third camera are arranged in sequence along the traveling direction of the vehicle; The fusion module is further used to fuse the first fusion result and the multiple vehicle rear images according to the shooting time of each image to obtain a second fusion result, wherein the second fusion result is used to represent the target vehicle front image and the target vehicle rear image corresponding to the target vehicle body image, and the target vehicle rear image is one of the multiple vehicle rear images; The screening module is used to select a vehicle front picture that meets a first condition from multiple vehicle front pictures as a target vehicle front picture, and the first condition is related to the shooting time of each picture.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
