Vehicle image fusion method and device, computer device and storage medium

Abstract

The application relates to a vehicle image fusion method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a plurality of vehicle images of at least two detection positions; the plurality of vehicle images are obtained by respectively photographing vehicles running through cameras at the at least two detection positions; respectively matching the plurality of vehicle images of different detection positions according to time difference values between the plurality of vehicle images of different detection positions, to obtain vehicle images associated with the vehicle at each detection position; and fusing the vehicle images associated with the vehicle at each detection position. The method can match a plurality of vehicle images of different positions according to time difference values, determine a matching result of vehicle images under different angles, and accurately fuse the vehicle images associated with the vehicle at each detection position.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a vehicle image fusion method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] Existing vehicle model recognition technologies, such as those based on lasers or infrared gratings, suffer from drawbacks including a lack of effective and intuitive detection evidence, complex construction, high cost, and susceptibility to adverse weather conditions like rain, snow, and fog. While video-based image stitching recognition overcomes these shortcomings and has become the mainstream method, it also faces challenges: it is highly dependent on environmental conditions and is easily affected by external factors (light, non-motorized vehicles), leading to poor recognition results. Multi-sensor fusion-based vehicle model recognition methods, on the other hand, struggle with how to collaboratively process different types and operating modes of sensors to achieve optimal overall system performance, and cannot effectively fuse images and structured information of vehicles from different times and scenarios. Summary of the Invention

[0003] Therefore, it is necessary to provide a vehicle image fusion method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve efficiency in response to the aforementioned technical problems.

[0004] In a first aspect, this application provides a vehicle image fusion method, the method comprising:

[0005] Acquire multiple vehicle images from at least two detection locations; the multiple vehicle images are captured by cameras at the at least two detection locations, respectively, of moving vehicles;

[0006] The vehicle images at different detection locations are matched according to the time difference values ​​between the vehicle images at different detection locations to obtain the vehicle images associated with each detection location.

[0007] The vehicle images associated with each of the detection locations are fused together.

[0008] In one embodiment, matching the multiple vehicle images at different detection locations according to the time difference values ​​between the multiple vehicle images at different detection locations to obtain the vehicle image associated with each of the detection locations includes:

[0009] According to the multiple vehicle images at different detection positions, the image detection time of each vehicle image is obtained and the vehicle speed is identified;

[0010] Based on the distance between adjacent detection positions and the vehicle speed, the estimated travel time of the vehicle between the adjacent detection positions is calculated.

[0011] Based on the estimated travel time and the detection time of each vehicle image at the adjacent detection locations, a time difference value is generated between multiple vehicle images at the adjacent detection locations.

[0012] Based on the time difference value, among multiple vehicle images at adjacent detection locations, the vehicle image associated with each detection location is determined; the vehicle image at each detection location corresponds one-to-one with the adjacent detection location.

[0013] In one embodiment, determining the vehicle image associated with each detection location among multiple vehicle images at adjacent detection locations based on the time difference value includes:

[0014] Select the target time difference value based on the stated time difference value;

[0015] When the target time difference value is determined to be less than the interval anomaly threshold, the vehicle image corresponding to the target time difference value is determined as the vehicle image at each of the detection positions.

[0016] When the target time difference value is determined to be greater than the interval anomaly threshold, the matching degree between the vehicle images corresponding to the target time difference value is calculated, and the matching degree is used to determine whether the vehicle image corresponding to the target time difference value is the vehicle image at each of the detection positions.

[0017] In one embodiment, if one of the vehicle images to be merged, the first and second, does not contain license plate information, the matching degree includes color matching degree; if both the first and second vehicle images to be merged contain license plate information, the matching degree includes the color matching degree and the license plate matching degree, and the license plate matching degree has a higher priority than the color matching degree.

[0018] In one embodiment, before fusing the vehicle images associated with each of the detection locations, the method further includes:

[0019] When the vehicle speed is less than the vehicle speed threshold, the corresponding expected shooting time sequence is obtained according to the order of each detection position. It is then determined whether the expected shooting time sequence matches the shooting time of the vehicle images at different detection positions to obtain a shooting time matching result. Based on the shooting time matching result, it is determined whether to fuse the vehicle images associated with each detection position.

[0020] When the vehicle speed is greater than the vehicle speed threshold, determine whether to fuse the vehicle images associated with each detection position based on whether the shooting time interval corresponding to each detection position corresponds to the interval threshold parameter.

[0021] In one embodiment, before fusing the multiple vehicle images associated with each of the detection locations, the method further includes:

[0022] Obtain the order of position identifiers when the vehicle is photographed according to different detection positions;

[0023] Based on the location identifier order, the identifiers carried by the vehicle images at different detection locations are matched to obtain the identifier matching result;

[0024] Based on the identifier matching results, it is determined whether to fuse the vehicle images associated with each of the detection locations.

[0025] In one embodiment, the cameras at at least two detection locations respectively capture images of the moving vehicle, including:

[0026] When the camera at the first detection position detects a moving vehicle, it calculates the vehicle's speed;

[0027] The camera at the first detection position estimates the estimated time for the vehicle to reach the second detection position based on the vehicle's speed.

[0028] The camera at the first detection position sends the estimated shooting time to the camera at the second detection position, so that the camera at the second detection position can take a picture.

[0029] Secondly, this application also provides a vehicle image fusion apparatus. The apparatus includes:

[0030] An image acquisition module is used to acquire multiple vehicle images from at least two detection locations; the multiple vehicle images are captured by cameras at the at least two detection locations, respectively, of moving vehicles;

[0031] The image matching module is used to match multiple vehicle images at different detection locations according to the time difference value between the multiple vehicle images at different detection locations, so as to obtain the vehicle image associated with each detection location.

[0032] An image fusion module is used to fuse vehicle images associated with each of the detection locations.

[0033] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the vehicle image fusion steps in any of the above embodiments.

[0034] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the vehicle image fusion steps in any of the above embodiments.

[0035] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the vehicle image fusion steps in any of the above embodiments.

[0036] The aforementioned vehicle image fusion method, apparatus, computer equipment, storage medium, and computer program product acquire multiple vehicle images from at least two detection locations, resulting in vehicle images captured from different angles. The multiple vehicle images from different detection locations are matched according to the time difference value between them. This matching of vehicle information at different locations is determined by using the time difference value to identify the matching results for vehicle images at different angles. Without directly calculating image similarity, the associated vehicle images at each detection location can be directly obtained. Furthermore, the associated vehicle images at each detection location are fused, achieving precise fusion of the front, side, and rear images of the same vehicle, as well as the license plate number and vehicle model. This adds information such as vehicle side images and vehicle model to the existing toll collection system at logistics park entrances and exits, providing a basis and guarantee for toll collection based on vehicle model classification. Attached Figure Description

[0037] Figure 1 This is an application environment diagram of the vehicle image fusion method in one embodiment;

[0038] Figure 2 This is a flowchart illustrating a vehicle image fusion method in one embodiment;

[0039] Figure 3 This is a flowchart illustrating the vehicle image fusion method in another embodiment;

[0040] Figure 4 This is a flowchart illustrating a vehicle image fusion method in one embodiment;

[0041] Figure 5 This is a structural block diagram of a vehicle image fusion device in one embodiment;

[0042] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0044] The vehicle image fusion method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on another network server.

[0045] The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle systems. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0046] In one embodiment, such as Figure 2 As shown, a vehicle image fusion method is provided, which is applied to... Figure 1 Taking terminal 102 as an example, the explanation includes the following steps:

[0047] Step 202: Acquire multiple vehicle images from at least two detection locations; the multiple vehicle images are captured by cameras at at least two detection locations, respectively, of moving vehicles.

[0048] The detection position refers to the location of the camera. Multiple detection positions are used to determine the angle at which the vehicle is photographed. These angles capture different areas of the vehicle, resulting in front, side, and rear images. An example of a detection position is provided below. The camera is installed on the side of the entrance / exit road at a height of 1.2m–1.8m. The center line of the vehicle-side recognition unit's field of view is perpendicular to the vehicle's direction of travel on the entrance / exit road, with a downward viewing angle of 0–15 degrees, and a distance of 0.5–1.2m from the road edge. A 1.4–2mm fisheye lens is used. The horizontal viewing angle is adjusted to ensure that the road's direction of travel is horizontal in the image, guaranteeing that the vehicle moves horizontally within the image.

[0049] Multiple vehicle images at a detection location are obtained by capturing images of vehicles in motion from a specific angle. The vehicle motion process refers to the change in vehicle position on the road at different times, and the images captured as the vehicle position changes constitute multiple vehicle images. Each vehicle image can be identified, and the identification result is the extraction of vehicle information from the corresponding angle captured in the vehicle image. The information extracted from a single vehicle image can be information about one or more vehicles, and the information from one or more vehicles can be used to determine their associated vehicle images using the scheme of this application. Therefore, associated vehicle images are determined by detecting the location for image stitching.

[0050] When multiple vehicle images are individual images in a video, the detection location can determine the associated vehicle image and perform recognition, resulting in a highly reliable recognition result.

[0051] In response to video anomalies, the terminal achieves recognition by caching large single-frame images. The vehicle-side recognition unit detects the vehicle in real time and notifies the front and rear recognition units to use the cached large images for single-frame recognition.

[0052] In one embodiment, cameras at at least two detection locations capture images of a moving vehicle, including: when a camera at a first detection location detects a moving vehicle, calculating the vehicle speed; the camera at the first detection location estimating the estimated time for the moving vehicle to reach a second detection location based on the vehicle speed; and the camera at the first detection location sending the estimated time to the camera at the second detection location to enable the camera at the second detection location to capture an image.

[0053] The first detection position and the second detection position are corresponding detection positions. The information extracted from the vehicle image captured by the camera at the first detection position is used not only to determine the associated vehicle image, but also to determine the shooting timing of the camera at the second detection position, so as to control the camera at the second detection position to capture the vehicle according to the estimated shooting time.

[0054] For example, the first detection position is the position of the vehicle-side recognition unit, and the second detection position is the position of at least one of the vehicle-front recognition unit and the vehicle-rear recognition unit. When the vehicle-side recognition unit captures a picture of the front of the vehicle, it calculates the vehicle speed, estimates the estimated time for the vehicle to reach the corresponding position of the vehicle-front recognition unit based on the vehicle speed, and then sends the estimated time to the vehicle-front recognition unit so that the vehicle-front recognition unit can capture the vehicle at the estimated time. Next, when the vehicle-side recognition unit captures a picture of the rear of the vehicle, it calculates the vehicle speed, estimates the estimated time for the vehicle to reach the corresponding position of the vehicle-rear recognition unit based on the vehicle speed, and then sends the estimated time to the vehicle-rear recognition unit so that the vehicle-rear recognition unit can capture the vehicle at the estimated time. The vehicle-front recognition unit is a camera and corresponding equipment for capturing the front of the vehicle, and the vehicle-rear recognition unit is a camera and corresponding equipment for capturing the rear of the vehicle.

[0055] The cameras at the second detection position are controlled to capture images of the vehicle at estimated times. This includes: the front recognition unit capturing the front of the vehicle for the first time at the estimated time; and during the capture process, triggering the front recognition unit to capture an image whenever the side recognition unit detects that the vehicle has traveled a preset distance (e.g., 6 meters). Similarly, the rear recognition unit captures the rear of the vehicle at the estimated time. Regardless of whether it is the first capture, a timed cached image is obtained. This timed cached image is used to output a single-frame recognition result, which is the information extracted from the vehicle image. The timed cached image is an alternative result in this abnormal situation. The side recognition unit notifies the front and rear recognition units to use the cached large image for single-frame recognition by detecting the time the vehicle passes the side recognition unit in real time.

[0056] The cached large image single-frame recognition result is a backup result for handling abnormal situations. The vehicle-side recognition unit notifies the front and rear recognition units to use the cached large image for single-frame recognition by detecting the time when the vehicle passes the vehicle-side recognition unit in real time.

[0057] Specifically, multiple vehicle images at each detection location constitute a buffer queue. The video capture mechanism and buffering mechanism are as follows:

[0058] When the vehicle-side recognition unit detects a passing vehicle, it calculates the vehicle's speed and estimates the initial capture time to the front-facing recognition unit. It then sends a trigger signal along with the estimated capture time and ID of the front-facing unit. After the estimated capture time at the front, the vehicle-side recognition unit sends a buffer command every 6 meters of vehicle movement, triggering the front-facing unit to capture an image. Once the vehicle has completely passed the vehicle-side recognition unit, it calculates the vehicle's speed and estimates the initial capture time to the rear-facing recognition unit, sending a signal along with the estimated capture time and ID to the rear-facing recognition unit.

[0059] Correspondingly, if the front or rear vehicle recognition unit can output the recognition result of the video stream, the video shooting mechanism is followed, and the recognition result of the video stream is used first to extract the corresponding vehicle information for image fusion. If one of the front or rear vehicle recognition units cannot output the recognition result of the video stream, the unit that cannot output the recognition result of the video stream will activate a caching mechanism, caching a large image every 500ms and placing it in the cache queue. When a signal triggered by a caching command is received, the system searches for the cached large image with the closest time in the cache queue based on the estimated shooting time, performs single-frame recognition, and outputs the result. A trigger ID field is attached to the recognition result; the ID for the video stream result is 0, and the ID for the single-frame recognition result is the ID carried when triggered by the vehicle side, thus distinguishing between the two results.

[0060] Specifically, if the vehicle side recognition result can calculate the difference value of at least one dimension through the fusion algorithm, then the difference value of the corresponding dimension is used to find one of the vehicle front recognition result or the vehicle rear recognition result; if a recognition result is found, the single frame recognition result will not be used; otherwise, the single frame recognition result is used for the fusion method to select a suitable vehicle image to be fused.

[0061] Step 204: Match multiple vehicle images at different detection locations according to the time difference values ​​between them to obtain vehicle images associated with each detection location.

[0062] The temporal difference value is the difference between multiple vehicle images at different detection locations along the time axis. When multiple vehicle images at a certain detection location form an image set such as a cache queue or stack, the temporal difference values ​​are calculated for each image in this image set and for an image set at another detection location. This yields the vehicle images associated between the two image sets at each detection location. The vehicle images associated at the detection locations are the same vehicle images to be fused, and are vehicle images to be fused based on the temporal difference value calculation.

[0063] For example: if the camera at the first detection position captures images A1 and A2 sequentially, and the camera at the second detection position captures image B sequentially, then the time difference value between image A1 and image B is calculated, and then the time difference value between image A2 and image B is calculated. These two time difference values ​​are compared to obtain the time difference value matching result of image B. The time difference value matching result of image B is used to select the vehicle image associated with image B at the second detection position from images A1 and A2.

[0064] In one embodiment, NTP is used as a time synchronization tool for multiple cameras to ensure that the time of the cameras at each detection location is on the same time axis. Theoretically, this guarantees that when multiple vehicles are sequentially captured by the cameras at the detection locations, the recognition results of each vehicle will also be output sequentially. Secondly, when environmental or human factors cause discontinuities or deviations from the expected shooting time sequence, it helps to calculate based on the structured information in the fusion algorithm and the results to correct abnormal interference and filter erroneous results, thereby ensuring the correctness of the fusion result.

[0065] In one embodiment, multiple vehicle images at different detection locations are matched according to the time difference values ​​between them to obtain vehicle images associated with each detection location, including:

[0066] For multiple vehicle images at different detection locations, the image detection time of each vehicle image is obtained and the vehicle speed is identified. Based on the distance between adjacent detection locations and the vehicle speed, the estimated travel time of the vehicle between adjacent detection locations is calculated. Based on the estimated travel time and the detection time of each vehicle image at adjacent detection locations, a time difference value is generated between multiple vehicle images at adjacent detection locations. Based on the time difference value, the vehicle images associated with each detection location are determined among the multiple vehicle images at adjacent detection locations. There is a one-to-one correspondence between the vehicle images at each detection location and the adjacent detection locations.

[0067] Image detection time is the time it takes for a camera at a certain detection location to capture an image of a specific vehicle. Each vehicle image has its own image detection time. For multiple image detection times at a single detection location, these times should be arranged sequentially.

[0068] Unlike directly acquired image detection time, vehicle speed is obtained based on the recognition of at least two vehicle images at a detection location. The identified vehicle speed is the speed at which the vehicle travels between adjacent detection locations. Images captured by cameras at adjacent detection locations are matched according to the vehicle images if they are from different detection locations.

[0069] Calculating the estimated travel time of the vehicle between adjacent detection locations involves determining the time it takes for the vehicle to travel between these locations. This calculation can be performed using a ratio. The calculated estimated travel time can be used to estimate the shooting time or to determine the estimated shooting time for the initial shot.

[0070] In one embodiment, determining the vehicle image associated with each detection location from multiple vehicle images at adjacent detection locations based on the time difference value includes: when the time difference value is a time difference; searching among multiple vehicle images at adjacent detection locations based on the minimum time difference value; and determining the vehicle image at each detection location corresponding to the minimum time difference value as the vehicle image associated with that vehicle at each detection location. The number of vehicle images at each detection location corresponding to the minimum value corresponds one-to-one with the number of detection locations. Thus, under normal conditions where all cameras in each detection unit are capturing images, the selection of the target time difference value is achieved. It should be understood that the time difference value is not necessarily a time difference or a difference calculated based on a time difference; it can also be a difference calculated using methods such as ratios or variances.

[0071] Furthermore, regardless of whether the shooting is under normal or abnormal conditions, the vehicle image associated with the detection location can be determined. The interval anomaly threshold refers to a multiple of the threshold parameter for the time interval between adjacent detection locations, which can be twice. Consider a similar anomaly: after a vehicle A passes by the device, due to various abnormal reasons (such as following too closely and not being able to recognize the license plate), there is no recognition result for vehicle A in the front result queue of the vehicle front recognition unit, but there is a front result for the previous vehicle B in the front result queue (the previous vehicle B was recognized by vehicle A an hour earlier). In this case, the front result is for vehicle B, and the body result is for vehicle A, and their time difference will be abnormally large.

[0072] To address this issue, based on the time difference value, among multiple vehicle images at adjacent detection locations, the vehicle images associated with each detection location are determined, including:

[0073] Select the target time difference value based on the time difference value.

[0074] When the target time difference value is determined to be less than the interval anomaly threshold, the vehicle image corresponding to the target time difference value is determined as the vehicle image at each detection position. Specifically, when the target time difference value is less than the interval anomaly threshold, adjacent detection positions can normally capture vehicle images. The target time difference value is a selected time difference value, which can be the minimum value between vehicle images at different detection positions, or it can be selected using other methods.

[0075] When the target time difference value is greater than the interval anomaly threshold, the matching degree between the vehicle images corresponding to the target time difference value is calculated. Based on the matching degree, it is determined whether the vehicle image corresponding to the target time difference value is indeed the vehicle image at each detection location. Therefore, when the target time difference value is less than the interval anomaly threshold, there is no need to extract information from the images, and the matching image can be calculated relatively conveniently. Even when the target time difference value exceeds the interval anomaly threshold, the matching degree between vehicle images can still be calculated to determine whether the vehicle image corresponding to the target time difference value is indeed the vehicle image at each detection location.

[0076] Specifically, when one of the vehicle images to be merged, the first and second, does not contain license plate information, the matching degree includes color matching degree; when both the first and second vehicle images to be merged contain license plate information, the matching degree includes color matching degree and license plate matching degree, with the license plate matching degree having a higher priority than the color matching degree.

[0077] Specifically, color matching score is calculated by determining the similarity between at least two colors; this can be obtained by calculating the Euclidean distance between the two colors, or by calculating the similarity between the colors and the expected combination. When vehicle images of the front and side regions are fused, and this fusion process can be the first fusion, only color matching score needs to be calculated.

[0078] License plate matching degree is calculated based on the license plate information extracted from the vehicle image; it can be identified by any image recognition method and is used to determine whether the license plate information of different vehicle images matches. When the first vehicle image to be fused is obtained by fusing vehicle images of the front and side regions, then the first vehicle image to be fused contains license plate information, and therefore can be fused with the second vehicle image to be fused—the vehicle image of the rear region.

[0079] In one embodiment, the priority of license plate matching degree and color matching degree is mainly discussed, and this priority is used to determine the calculation order of the matching degree. When the license plate matching degree indicates that the license plate information between the first vehicle image to be fused and the second vehicle image to be fused is consistent, it is determined that the first vehicle image to be fused and the second vehicle image to be fused will be fused; when the license plate matching degree indicates that the license plate information between the first vehicle image to be fused and the second vehicle image to be fused is inconsistent, and both the first vehicle image to be fused and the second vehicle image to be fused are inconsistent with the vehicle information of other vehicle images, it is determined whether the first vehicle image to be fused and the second vehicle image to be fused will be fused by calculating the color matching degree.

[0080] In one embodiment, before fusing the vehicle images associated with each detection location, the method further includes:

[0081] When the vehicle speed is less than the vehicle speed threshold, the corresponding expected shooting time sequence is obtained according to the order of each detection position. It is then determined whether the expected shooting time sequence matches the shooting time of the vehicle images at different detection positions to obtain the shooting time matching result. Based on the shooting time matching result, it is determined whether to fuse the vehicle images associated with each detection position.

[0082] When the vehicle speed exceeds the vehicle speed threshold, determine whether to fuse the vehicle images associated with each detection location based on whether the shooting time interval corresponding to each detection location corresponds to the time interval threshold parameter.

[0083] The vehicle speed threshold is the maximum speed of the identified vehicle. When the vehicle speed calculated by the head-up recognition unit exceeds the speed threshold preset by the fusion algorithm, the passage of the vehicle through the side recognition unit or corresponding terminal is considered abnormal. This could be due to one of the following: a. The vehicle speed is not the actual moving speed; b. The current vehicle speed is too high, requiring processing time for the device; c. Other abnormal situations. Therefore, by using a relatively lenient judgment standard—whether the shooting time interval corresponding to each detection position corresponds to the time interval threshold parameter—the robustness of the fusion algorithm is improved.

[0084] When the vehicle speed is less than the vehicle speed threshold, the expected shooting time sequence is obtained according to the order of each detection position, and it is then determined whether there are any anomalies in the associated vehicle images. For example, if the order of the detection positions is the position of the front recognition unit and the position of the side recognition unit, then the expected shooting time sequence should be the shooting time of the front recognition unit and the shooting time of the side recognition unit. If the shooting time sequence matches the shooting time of the vehicle images at different detection positions, then the vehicle images associated with each detection position are fused; otherwise, the vehicle images associated with each detection position are not fused.

[0085] In one embodiment, before fusing multiple vehicle images associated with each detection location, the method further includes: obtaining the position identifier order when the vehicle is photographed at different detection locations; matching the identifiers carried by the vehicle images at different detection locations according to the position identifier order to obtain an identifier matching result; and determining whether to fuse the vehicle images associated with each detection location based on the identifier matching result.

[0086] The position identifier order is the sequence in which identifiers are generated at each detection position during the shooting process, used to determine whether images captured at different detection positions match.

[0087] For example, the position identification order refers to the position identification order of the vehicle images captured by the front detection unit as identification 1 and identification 2, respectively, and the position identification order of the vehicle images captured by the side recognition unit as identification A and identification B.

[0088] Then the vehicle image at the detection position of the front recognition unit should carry the identifiers 1 and 2 in sequence, and the vehicle image at the detection position of the side recognition unit should carry the identifiers A and B in sequence.

[0089] Therefore, it is determined that the vehicle image carrying identifier 1 and the vehicle image carrying identifier A are vehicle images associated with a certain vehicle at each detection location; while it is determined that the vehicle image carrying identifier 2 and the vehicle image carrying identifier B are vehicle images associated with another vehicle at each detection location.

[0090] Step 206: Fuse multiple vehicle images associated with each detection location.

[0091] In one embodiment, fusing multiple vehicle images associated with a vehicle at various detection locations means fusing vehicle images of the same vehicle associated with each detection location to obtain a fused image of the vehicle. For example, accurately capturing vehicle images of passing vehicles, including front view, side view, and side view images. Then, identifying structured vehicle model information from the front view, side view, and side view images respectively, and accurately fusing them together to obtain the corresponding vehicle image.

[0092] In the aforementioned vehicle image fusion method, multiple vehicle images from at least two detection locations are acquired, resulting in vehicle images captured from different angles. These images are then matched based on the temporal difference between them, determining the matching results for vehicle images at different angles. This method directly obtains the associated vehicle images at each detection location without directly calculating image similarity. Furthermore, it fuses these associated vehicle images at each detection location, accurately integrating images of the front, side, and rear of the same vehicle, along with its license plate number and vehicle model. This adds information such as side views and vehicle model to the existing toll collection system at logistics park entrances and exits, providing a basis and guarantee for toll collection based on vehicle model classification.

[0093] In one embodiment, the terminal uses the timeline as the primary basis, combined with structured information output by the front vehicle recognition unit (license plate number, captured image, front vehicle color, detection time, trigger ID, etc.), structured information output by the side vehicle recognition unit (large stitched image of the side vehicle, front vehicle detection time, rear vehicle detection time, vehicle speed, wheel start detection time, wheel end detection time, trigger ID, etc.), and structured information output by the rear vehicle recognition unit (license plate number, captured image, side vehicle color, detection time, trigger ID, etc.), to complete the information fusion of multi-angle captured images of a vehicle through a fusion algorithm.

[0094] In one embodiment, the first detection position is the position of the vehicle side recognition unit, and the second detection position is the position of the vehicle front recognition unit.

[0095] Let H be the result identified by the front detection unit. n , where n represents the nth recognition result of the vehicle front recognition; the image capture time of the vehicle front detection unit is H. n .T0, license plate number H on the front of the vehicle n .P, front color H n C, vehicle head speed H n .V, trigger location identifier ID is H n .ID, S is the distance from the recognition position of the front recognition unit to the position of the side detection unit.

[0096] Let the result be B on the vehicle side. n 'n' represents the nth recognition result of the vehicle side detection; the vehicle front detection time is B. n .T0, Wheel detection start time B n .T1, Wheel end detection time B n .T2, rear-end inspection time is B n .T3, the time it takes for the vehicle to pass the side of the vehicle is B. n .T4, side color B n .C, Train Length B n L, the speed of the two vehicles passing through is B. n .V, trigger location identifier ID is B n .ID.

[0097]

[0098]

[0099]

[0100] Wherein, n1 and n2 are irrelevant parameters. Thus, under normal circumstances, the vehicle front and side recognition units can be combined. Wherein, formula (1) is used to select the target time difference value; formula (2) obtains the corresponding expected shooting time sequence according to the order of each detection position, and determines that the expected shooting time sequence matches the shooting time of the vehicle images at different detection positions; formula (3) is used to determine that the identifiers carried by the vehicle images at different detection positions match according to the position identifier order.

[0101] Furthermore, when the vehicle speed exceeds the vehicle speed threshold, the system determines whether to fuse the vehicle images associated with each detection location based on whether the shooting time interval corresponding to the detection location in sequence corresponds to the time interval threshold parameter. n If V > P2, then the substitution formula (2) is as follows:

[0102]

[0103] Its purpose is to, when the vehicle head recognition unit calculates H n When V is greater than the preset threshold P2 of the fusion algorithm, the fusion algorithm considers the current vehicle passing through the device to be an abnormal situation, which may include the following: a. Vehicle speed H n a. V is not the actual speed of the moving vehicle; b. The current vehicle speed is too fast and the device needs to be given processing time; c. Other abnormal situations. Then a relatively lenient judgment threshold formula (4) is needed to replace formula (2) for the fusion algorithm, so as to improve the robustness of the fusion algorithm.

[0104] Furthermore, if the interval anomaly threshold is twice the time interval threshold parameter, and the target time difference value exceeds the interval anomaly threshold, then formula (5) is satisfied, as follows:

[0105]

[0106] When calculating the matching degree between vehicle images corresponding to the target time difference value, formula (6) is used, as follows:

[0107]

[0108] Wherein, similar1 is used to calculate the similarity between two colors, which is represented by the Euclidean distance between the two colors. If the two recognition results of the vehicle images to be fused satisfy formula (5), the system is considered to be in a special state, such as following another vehicle or a vehicle without a license plate. In this case, formula (6) needs to be added to ensure the correctness of the fusion algorithm. If the two recognition results do not satisfy formula (6), the system continues to wait for new recognition results to perform the fusion algorithm.

[0109] In one embodiment, the first detection position is the position of the vehicle side recognition unit, and the second detection position is the position of the vehicle rear recognition unit.

[0110] Let the result at the rear of the car be E. n 'n' represents the nth recognition result of the tail detection; the tail detection time is E. n .T0, license plate E at the rear of the car n .P, rear color E n .C, rear-end speed E n .V, trigger location identifier ID is E n .ID, S represents the distance from the recognition position of the rear-end recognition unit to the side-end detection unit. Let P3 be the threshold parameter for the time interval between the rear-end and side-end results, and P4 be the threshold parameter for the rear-end speed.

[0111] When E n P==H n When .P, the matching degree of this license plate is used directly to complete the matching fusion. Otherwise, the corresponding formulas (7)-(12) and (13) are used to complete the image fusion.

[0112]

[0113]

[0114]

[0115] Special cases, if The replacement formula (8) is as follows:

[0116]

[0117] Special case, if

[0118]

[0119] Then it must also meet the following conditions:

[0120]

[0121]

[0122] As mentioned before, similar1 is for color similarity calculation; similar2 is for calculating the similarity between two license plates. By comparing the strings of the two license plates, the number of identical digits is used to represent the similarity between the two license plates. The current fusion algorithm defaults to considering two license plates similar if they have 5 identical digits. When the result finally found by the fusion algorithm also satisfies formula (11), it means that the selected result may not be the most suitable result. Therefore, it is necessary to add judgment condition formulas (12) and (13) to tighten the judgment condition and improve the accuracy of the algorithm. If formulas (12) and (13) are not satisfied, then wait for new recognition results to perform fusion.

[0123] Therefore, this patented method can accurately fuse images of the front, side, and rear of the same vehicle, along with its license plate number and vehicle model, to form multi-dimensional vehicle data. This adds information such as vehicle side images and vehicle model to the existing toll collection systems at logistics park entrances and exits, providing a basis and guarantee for toll collection based on vehicle model. In actual testing, regardless of congestion or the presence of moving non-motorized vehicles and pedestrians in the background, the fusion rate can reach over 99.5%.

[0124] In one embodiment, such as Figure 3 As shown, the detection process is determined by taking pictures from three angles: the front recognition unit, the side recognition unit, and the rear recognition unit.

[0125] In the process of video stream recognition, the front recognition unit and the rear recognition unit output the corresponding video stream results at regular intervals. Based on the video stream results, a front result queue and a rear result queue are generated. Then, the front result queue and the rear result queue are filtered respectively, and the images are fused according to the filtered results.

[0126] After the vehicle-side recognition unit detects the front of the vehicle, it triggers the front recognition unit to periodically cache images. The vehicle-side recognition unit also triggers a periodic image capture every 6 meters the vehicle has moved, obtaining a single-frame recognition result for the front of the vehicle. Similarly, a similar method can be used to obtain the single-frame recognition result for the rear of the vehicle. This single-frame recognition result serves as a backup for handling abnormal situations. The vehicle-side recognition unit notifies the front and rear recognition units to use the cached large image for single-frame recognition by detecting the time it takes for the vehicle to pass it in real time.

[0127] After obtaining the result queues of the front, side, and rear of the vehicle, and filtering them separately, the images are fused according to the scheme of this embodiment. First, the images of the front and side of the vehicle are fused to obtain the first image of the vehicle to be fused. Then, the first image of the vehicle to be fused is fused with the second image of the vehicle to be fused (the image of the rear of the vehicle) to output the complete fused information of the vehicle.

[0128] In a more complete embodiment, such as Figure 4As shown. This patent provides a multi-camera, multi-angle image fusion technology that can accurately capture images of the front, rear, and side of a passing vehicle, identify structured vehicle model information, and accurately fuse vehicle images from various angles. Compared to existing technologies, this method uses multiple sensors of the same type but at different angles, and filters out interference factors in complex environments to extract effective vehicle model information and complete the recognition and fusion process. The specific steps are as follows:

[0129] S401. The camera is installed on the side of the entrance / exit road.

[0130] Specific installation parameters include an installation height of 1.2m to 1.8m, with the center line of the vehicle-side recognition unit's field of view perpendicular to the vehicle's direction of travel on the entrance / exit road, a downward viewing angle of 0 to 15 degrees, and a distance of 0.5 to 1.2m from the road edge. A 1.4 to 2mm fisheye lens should be used. The horizontal viewing angle should be adjusted to ensure that the road's direction of travel is horizontal in the image, guaranteeing that the vehicle moves horizontally within the image.

[0131] S402. When a vehicle passes the front recognition unit: The AI ​​algorithm identifies and outputs the vehicle's structured information, including the license plate, shooting time, front color, and a panoramic image of the front of the vehicle.

[0132] S403. When a vehicle passes the vehicle side recognition unit: the vehicle side stitching technology is used to detect and identify structured information such as vehicle speed, vehicle type, axle type, number of axles, and a panoramic image of the vehicle side stitching.

[0133] S404. When a vehicle passes the rear-end recognition unit: The AI ​​algorithm identifies and outputs the vehicle's structured information, including the license plate, shooting time, rear-end color, and a panoramic image of the rear of the vehicle.

[0134] The S405, the front recognition unit, and the rear recognition unit transmit the recognized vehicle information to the side recognition unit via network protocol.

[0135] S406. Fusion Timing: Find the front and rear results of the vehicle based on the side results. Once the side results are available, find the front and rear results that meet the conditions.

[0136] The S407, the front recognition unit, and the rear recognition unit produce two results: video stream recognition results and cached large image single-frame recognition results.

[0137] Step S407 is to ensure that each vehicle has complete front, side and rear data in the event of loss of vehicle front or rear data due to abnormal factors such as human error, environmental factors or lack of license plates.

[0138] a. Video stream recognition results: This type of result has the highest reliability and should be used first.

[0139] b. Cache large image single-frame recognition results. This type is an alternative result for dealing with abnormal situations. The vehicle-side recognition unit notifies the front and rear recognition units to use the cached large image for single-frame recognition by detecting the time when the vehicle passes the vehicle-side recognition unit in real time.

[0140] The specific caching mechanism in the result output mechanism of S407, the front recognition unit, and the rear recognition unit is as follows:

[0141] Vehicle-side recognition unit: a. When a vehicle is detected approaching, the unit calculates the vehicle's speed, estimates the time it will take to reach the front recognition unit, and sends a trigger signal along with the estimated time and ID to the front recognition unit. b. Subsequently, the front recognition unit is triggered again every time a vehicle is detected moving 6 meters. c. After a vehicle has completely passed the vehicle-side recognition unit, the unit calculates the vehicle's speed, estimates the time it will take to reach the rear recognition unit, and sends a signal along with the estimated time and ID to the rear recognition unit.

[0142] Vehicle front and rear recognition units: a. Output the recognition results of the video stream. b. Cache a large image every 500ms in the cache queue. When a trigger signal is received, based on the estimated time, search the cache queue for the cached large image with the closest time, perform single-frame recognition, and output the result. The recognition result includes a trigger ID field; the ID for the video stream result is 0, and the ID for the single-frame recognition result is the ID carried when triggered on the vehicle side, thus distinguishing between the two results.

[0143] If the vehicle-side results can be used to find corresponding front and rear results through the fusion algorithm, then the single-frame recognition results will not be used. If the fusion algorithm cannot find a matching front and rear result, then the single-frame recognition results will be used to allow the fusion method to select the most suitable one as the final result.

[0144] S408, Fusion Method: The vehicle side recognition unit fuses the results of the front, side and rear of the vehicle within the queue according to the fusion algorithm and outputs them.

[0145] Algorithm for fusing side and front vehicle results: Condition 1: The front vehicle shooting time is less than the start time of the side vehicle shooting. This is to filter out the front vehicle results of the previous vehicle. The formula is as follows:

[0146]

[0147] Condition 2: Subtract the vehicle front shooting time from the wheel shooting time, and then subtract the shooting time when the vehicle reaches the side recognition unit to obtain the time difference value. Then calculate the minimum value of the time difference value. The purpose is to ensure the matching of the current vehicle front result and the vehicle side result, and to filter out the vehicle front result of the previous or next vehicle when multiple vehicle front results appear due to congestion. The formula is as follows:

[0148]

[0149] Condition 3: The recognition time of the front of the car is longer than the start time of the previous car's side.

[0150] Condition 4: The front ID is 0, or the front ID is equal to the side ID.

[0151] Condition 5: When the vehicle speed reaches the threshold P2, appropriately relax the matching condition 1, satisfying the following conditions:

[0152]

[0153] This addresses the issue of situations where the device is still processing recognition or other abnormalities at high vehicle speeds and therefore does not output results.

[0154] Condition 6: If the minimum value in Condition 2 is greater than the threshold of 2*P1, then the following additional color similarity conditions must be met to complete the matching and fusion:

[0155]

[0156] If all the above conditions are met, the vehicle front selection result is obtained. If none of them are met, the system continues to wait for a new recognition result. If the waiting timeout occurs or the side result of the next vehicle is received, the result of the single frame recognition is directly used as a substitute result for fusion to ensure that each side result can find the vehicle front result.

[0157] Algorithm for fusing side and rear view results:

[0158] Condition 1: Use the license plate at the rear of the vehicle to find the license plate of the front and side of the vehicle that has already been merged. If the license plate numbers match, then the front, side and rear of the vehicle are the same vehicle and can be merged directly.

[0159] Condition 2: The time for recognizing the rear of the vehicle is longer than the time for stitching the side of the vehicle to end. This is to filter out the results of the rear of the previous vehicle.

[0160] Condition 3: Subtract the vehicle side / rear-viewing time from the vehicle rear-viewing time, and then calculate the minimum of the following: the time it takes for the vehicle to travel from the side recognition unit to the rear-view recognition unit. This minimum value will be used to filter out false or missed detections of the rear-view results. The formula is as follows:

[0161]

[0162] Condition 4: The rear ID is 0, or the rear ID is less than or equal to the side ID.

[0163] If the above conditions are met, and the waiting time expires or the side result of the next vehicle is received, the single-frame icon recognition result is used as a substitute result for fusion to ensure that the rear result can be found for each side result.

[0164] S409. The three camera recognition units use the same NTP server as the NTP time source for time synchronization, synchronizing the time every 5 minutes to prevent the three cameras from becoming out of sync.

[0165] S410. Through the above steps, the fusion results of the front, side and rear images of the same vehicle can be obtained, thereby completing the fusion work and providing a clear and reliable basis for vehicle charging at the entrance and exit of the logistics park.

[0166] In one embodiment, this application relates to a method for fusing images of the same vehicle captured by multiple cameras from multiple angles at the entrance and exit of a logistics park, which includes:

[0167] Installation Steps: The camera should be installed on the side of the entrance / exit road at a height of 1.2m–1.8m. The center line of the field of view should be perpendicular to the direction of vehicle travel on the entrance / exit road, with a downward viewing angle of 0–15 degrees, and a distance of 0.5–1.2m from the road edge. A 2mm fisheye lens should be used. Adjust the horizontal viewing angle to ensure that the road surface and vehicle direction are horizontal in the image, guaranteeing that vehicles move horizontally in the image. Specifically, the camera should be installed on the side of the entrance / exit road at a height of 1.2m–1.8m. The center line of the field of view should be perpendicular to the direction of vehicle travel on the entrance / exit road, with a downward viewing angle of 0–15 degrees, and a distance of 0.5–1.2m from the road edge. A 1.4–2mm fisheye lens should be used. Adjust the horizontal viewing angle to ensure that the road surface and vehicle direction are horizontal in the image, guaranteeing that vehicles move horizontally in the image.

[0168] The recognition steps of the front and rear vehicle recognition units are as follows: the structured information output by the front vehicle recognition unit (license plate number, captured image, color, detection time, etc.) and the structured information output by the rear vehicle recognition unit (license plate number, captured image, color, detection time, etc.).

[0169] The vehicle-side recognition unit's recognition steps include: outputting structured information (a large stitched image of the vehicle side, front detection time, rear detection time, vehicle speed, wheel detection time, etc.); calculating the vehicle speed; predicting the time it takes for the vehicle to pass the front and rear recognition units; and triggering single-frame recognition by the front and rear recognition units. Specifically, calculating the vehicle speed and estimating the time from the vehicle-side unit, predicting the time it takes for the vehicle to pass the front recognition unit, and triggering the front recognition unit to output a single-frame license plate recognition result, ensures reliable alternative front recognition results for the fusion algorithm.

[0170] The process of fusing images of the front and side of the vehicle involves fusing the images based on factors such as detection time, license plate number, color, and the start time of front-end detection. The fusion process is further refined by filtering for false positives or determining other anomalies to confirm the completion of the front and side image fusion.

[0171] Side and rear vehicle image fusion: Side and rear vehicle images are fused using detection time, license plate number, color, and the end time of rear vehicle detection. The fusion of side and rear vehicle images is determined by filtering for false positives or other anomaly assessment criteria.

[0172] Therefore, as a crucial node in the logistics system, vehicle model recognition equipment is an important component of the informatization and unmanned construction of logistics parks. Through video-based vehicle model recognition equipment, the multi-angle images, passing videos, and structured vehicle identification information output by the equipment can be clearly, effectively, and intuitively presented to users, providing a basis for logistics park charging and ensuring efficient access to and from the logistics park.

[0173] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0174] Based on the same inventive concept, this application also provides a vehicle image fusion apparatus for implementing the vehicle image fusion method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more vehicle image fusion apparatus embodiments provided below can be found in the limitations of the vehicle image fusion method described above, and will not be repeated here.

[0175] In one embodiment, such as Figure 5 As shown, a vehicle image fusion device is provided. The device includes:

[0176] The image acquisition module 502 is used to acquire multiple vehicle images at at least two detection locations; the multiple vehicle images are captured by the cameras at the at least two detection locations, respectively, of the moving vehicles;

[0177] The image matching module 504 is used to match multiple vehicle images at different detection locations according to the time difference value between the multiple vehicle images at different detection locations, so as to obtain the vehicle image associated with each detection location.

[0178] Image fusion module 506 is used to fuse vehicle images associated with each of the detection locations.

[0179] In one embodiment, the image matching module 504 is configured to:

[0180] According to the multiple vehicle images at different detection positions, the image detection time of each vehicle image is obtained and the vehicle speed is identified;

[0181] Based on the distance between adjacent detection positions and the vehicle speed, the estimated travel time of the vehicle between the adjacent detection positions is calculated.

[0182] Based on the estimated travel time and the detection time of each vehicle image at the adjacent detection locations, a time difference value is generated between multiple vehicle images at the adjacent detection locations.

[0183] Based on the time difference value, among multiple vehicle images at adjacent detection locations, the vehicle image associated with each detection location is determined; the vehicle image at each detection location corresponds one-to-one with the adjacent detection location.

[0184] In one embodiment, the image matching module 504 is specifically used for:

[0185] Select the target time difference value based on the stated time difference value;

[0186] When the target time difference value is determined to be less than the interval anomaly threshold, the vehicle image corresponding to the target time difference value is determined as the vehicle image at each of the detection positions.

[0187] When the target time difference value is determined to be greater than the interval anomaly threshold, the matching degree between the vehicle images corresponding to the target time difference value is calculated, and the matching degree is used to determine whether the vehicle image corresponding to the target time difference value is the vehicle image at each of the detection positions.

[0188] In one embodiment, if one of the vehicle images to be merged, the first and second, does not contain license plate information, the matching degree includes color matching degree; if both the first and second vehicle images to be merged contain license plate information, the matching degree includes the color matching degree and the license plate matching degree, and the license plate matching degree has a higher priority than the color matching degree.

[0189] In one embodiment, the image matching module 504 is further configured to:

[0190] When the vehicle speed is less than the vehicle speed threshold, the corresponding expected shooting time sequence is obtained according to the order of each detection position. It is then determined whether the expected shooting time sequence matches the shooting time of the vehicle images at different detection positions to obtain a shooting time matching result. Based on the shooting time matching result, it is determined whether to fuse the vehicle images associated with each detection position.

[0191] When the vehicle speed is greater than the vehicle speed threshold, determine whether to fuse the vehicle images associated with each detection position based on whether the shooting time interval corresponding to each detection position corresponds to the interval threshold parameter.

[0192] In one embodiment, the image matching module 504 is further configured to:

[0193] Obtain the order of position identifiers when the vehicle is photographed according to different detection positions;

[0194] Based on the location identifier order, the identifiers carried by the vehicle images at different detection locations are matched to obtain the identifier matching result;

[0195] Based on the identifier matching results, it is determined whether to fuse the vehicle images associated with each of the detection locations.

[0196] In one embodiment, the image acquisition module 502 is used for:

[0197] When the camera at the first detection position detects a moving vehicle, it calculates the vehicle's speed;

[0198] The camera at the first detection position estimates the estimated time for the vehicle to reach the second detection position based on the vehicle's speed.

[0199] The camera at the first detection position sends the estimated shooting time to the camera at the second detection position, so that the camera at the second detection position can take a picture.

[0200] Each module in the aforementioned vehicle image fusion device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0201] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a vehicle image fusion method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0202] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0203] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0204] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0205] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0206] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0207] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0208] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0209] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A vehicle image fusion method, characterized in that, The method includes: Acquire multiple vehicle images from at least two detection locations; the multiple vehicle images are captured by cameras at the at least two detection locations, respectively, of moving vehicles; The vehicle images at different detection locations are matched according to the time difference values ​​between the vehicle images at different detection locations to obtain the vehicle images associated with each detection location. When the vehicle speed is less than the vehicle speed threshold, the corresponding expected shooting time sequence is obtained according to the order of each detection position. It is then determined whether the expected shooting time sequence matches the shooting time of the vehicle images at different detection positions to obtain a shooting time matching result. Based on the shooting time matching result, it is determined whether to fuse the vehicle images associated with each detection position. When the vehicle speed is greater than the vehicle speed threshold, determine whether the vehicle images associated with each detection position should be fused according to whether the shooting time interval corresponding to each detection position corresponds to the interval threshold parameter. If the shooting time matching result indicates a match, and the shooting time interval corresponds to the interval threshold parameter, then the vehicle images associated with each of the detection locations are fused.

2. The method according to claim 1, characterized in that, The step of matching multiple vehicle images at different detection locations according to the time difference values ​​between the multiple vehicle images at different detection locations to obtain vehicle images associated with each detection location includes: According to the multiple vehicle images at different detection positions, the image detection time of each vehicle image is obtained and the vehicle speed is identified; Based on the distance between adjacent detection positions and the vehicle speed, the estimated travel time of the vehicle between the adjacent detection positions is calculated. Based on the estimated travel time and the detection time of each vehicle image at the adjacent detection locations, a time difference value is generated between multiple vehicle images at the adjacent detection locations. Based on the time difference value, among multiple vehicle images at adjacent detection locations, the vehicle image associated with each detection location is determined; the vehicle image at each detection location corresponds one-to-one with the adjacent detection location.

3. The method according to claim 2, characterized in that, The step of determining, based on the time difference value, the vehicle image associated with each detection location among multiple vehicle images at adjacent detection locations includes: Select the target time difference value based on the stated time difference value; When the target time difference value is determined to be less than the interval anomaly threshold, the vehicle image corresponding to the target time difference value is determined as the vehicle image at each of the detection positions. When the target time difference value is determined to be greater than the interval anomaly threshold, the matching degree between the vehicle images corresponding to the target time difference value is calculated, and the matching degree is used to determine whether the vehicle image corresponding to the target time difference value is the vehicle image at each of the detection positions.

4. The method according to claim 3, characterized in that, If one of the vehicle images to be merged (first and second) does not contain license plate information, the matching degree includes color matching degree; if both the first and second vehicle images to be merged contain license plate information, the matching degree includes color matching degree and license plate matching degree, and the license plate matching degree has a higher priority than the color matching degree.

5. The method according to claim 1, characterized in that, The determination of whether the shooting time interval corresponding to the order of the detection positions corresponds to the interval threshold parameter includes: Based on the difference between the front detection time at the front detection position and the front detection time at the side detection position, the corresponding shooting time interval between the front detection position and the side detection position is determined. If the shooting time interval is less than the interval threshold parameter, then the shooting time interval corresponds to the interval threshold parameter; If the shooting time interval is greater than or equal to the interval threshold parameter, then the shooting time interval does not correspond to the interval threshold parameter.

6. The method according to claim 1, characterized in that, Before fusing the multiple vehicle images associated with each of the detection locations, the method further includes: Obtain the order of position identifiers when the vehicle is photographed according to different detection positions; Based on the location identifier order, the identifiers carried by the vehicle images at different detection locations are matched to obtain the identifier matching result; Based on the identifier matching results, it is determined whether to fuse the vehicle images associated with each of the detection locations.

7. The method according to any one of claims 1 to 6, characterized in that, The cameras at at least two detection locations each capture images of the moving vehicle, including: When the camera at the first detection position detects a moving vehicle, it calculates the vehicle's speed; The camera at the first detection position estimates the estimated time for the vehicle to reach the second detection position based on the vehicle's speed. The camera at the first detection position sends the estimated shooting time to the camera at the second detection position, so that the camera at the second detection position can take a picture.

8. A vehicle image fusion device, characterized in that, The device includes: An image acquisition module is used to acquire multiple vehicle images from at least two detection locations; the multiple vehicle images are captured by cameras at the at least two detection locations, respectively, of moving vehicles; The image matching module is used to match multiple vehicle images at different detection locations according to the time difference value between the multiple vehicle images at different detection locations, so as to obtain the vehicle image associated with each detection location. The image matching module is used to: when the vehicle speed is less than a vehicle speed threshold, obtain the corresponding expected shooting time sequence according to the order of each detection position, determine whether the expected shooting time sequence matches the shooting time of vehicle images at different detection positions, and obtain a shooting time matching result; based on the shooting time matching result, determine whether to fuse the vehicle images associated with each detection position; when the vehicle speed is greater than the vehicle speed threshold, determine whether to fuse the vehicle images associated with each detection position according to whether the shooting time interval corresponding to the order of each detection position corresponds to the interval threshold parameter. The image fusion module is used to fuse vehicle images associated with each of the detection locations if the shooting time matching result indicates a match and the shooting time interval corresponds to the interval threshold parameter.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images