ETC data-based thunder-vision fusion result optimization method and apparatus, and computer device

Abstract

The invention discloses an ETC data-based thunder-vision fusion result optimization method and apparatus, and a computer device. The method comprises the following steps: acquiring radar data acquired by a millimeter wave radar, an image shot by a camera and ETC data acquired by an ETC antenna; performing spatial synchronization on the image and a map by establishing a homography matrix, and performing time synchronization on the image after spatial synchronization to obtain a synchronized image; carrying out license plate matching by using ETC data and the synchronized image; fusing the synchronized image with the radar data to obtain a fusion result; and outputting the final license plate information. By implementing the method, the advantages of the radar and the visual sensor can be combined, so that the precision and robustness of target detection and recognition are greatly improved, the limitation of a single sensor in a complex environment is effectively overcome, and through cooperation with an ETC system, the target detection and recognition efficiency is improved. And the traffic management efficiency in complex scenes such as expressways and tunnels is further improved.

Description

Technical Field

[0001] The present invention relates to an intelligent transportation system, and more specifically to a method, a device and a computer device for optimizing radar-visual fusion results based on ETC data. Background Art

[0002] With the continuous development of intelligent transportation systems, applications such as vehicle detection and tracking have become increasingly popular. A variety of sensors are widely used in this system, each with its own unique advantages. Radar sensors perform well in adverse weather conditions and can provide stable and reliable detection data in rain, snow, fog and other environments; visual sensors can capture rich image information and identify details such as the color, model, and license plate number of the vehicle; and electronic toll collection equipment can accurately identify relevant information about the vehicle, including the model and license plate number, through wireless communication between the on-board electronic tag and the roadside unit.

[0003] At present, most of these sensors work independently in intelligent transportation systems without effective data fusion. Radar sensors are mainly used to detect vehicle position and speed, while visual sensors focus on identifying vehicle type and license plate number. In complex environments such as highways or tunnels, radar can provide stable data, but its accuracy is low when identifying stationary or slow-moving vehicles, and it cannot distinguish vehicle features in detail. Although visual sensors can obtain rich image information, their detection accuracy and reliability may be greatly reduced under complex conditions such as low light, reflection or occlusion.

[0004] Therefore, it is necessary to design a new method that combines the advantages of radar and visual sensors, thereby greatly improving the accuracy and robustness of target detection and recognition, effectively overcoming the limitations of a single sensor in complex environments, and through collaboration with the ETC system, further improving traffic management efficiency in complex scenarios such as highways and tunnels. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the prior art and provide a method, device and computer equipment for optimizing radar-visual fusion results based on ETC data.

[0006] To achieve the above object, the present invention adopts the following technical solution: a method for optimizing radar-visual fusion results based on ETC data, comprising:

[0007] Obtain radar data collected by millimeter-wave radar, images taken by cameras, and ETC data collected by ETC antennas;

[0008] The image is spatially synchronized with the map by establishing a homography matrix, and the spatially synchronized image is temporally synchronized to obtain a synchronized image;

[0009] Using the ETC data and the synchronized image to perform license plate matching to obtain a license plate recognition result;

[0010] fusing the synchronized image with the radar data to obtain a fusion result;

[0011] The final license plate information is output according to the fusion result and the license plate recognition result.

[0012] A further technical solution is: the image is spatially synchronized with the map by establishing a homography matrix, and the spatially synchronized image is temporally synchronized to obtain a synchronized image, including:

[0013] Selecting at least four feature points in the map and the image as corresponding points;

[0014] Use the corresponding points to calculate the homography matrix using the findHomography function in the OpenCV library;

[0015] Projecting the map to the viewing angle of the camera using the homography matrix to obtain a spatially synchronized image;

[0016] The spatially synchronized image is time synchronized to obtain a synchronized image.

[0017] A further technical solution is: performing time synchronization on the spatially synchronized image to obtain a synchronized image, comprising:

[0018] The spatially synchronized image is time synchronized using a network time protocol to obtain a synchronized image.

[0019] A further technical solution is: using the ETC data and the synchronized image to perform license plate matching to obtain a license plate recognition result, including:

[0020] Creating a matching table based on existing license plate characters, wherein the matching table records similarity scores between different characters;

[0021] The license plate characters are extracted from the ETC data and the synchronized image respectively, and the similarity between the two license plate characters is calculated. The successfully matched license plate characters are determined in combination with the matching degree table to obtain a license plate recognition result.

[0022] The further technical solution is: extracting license plate characters from the ETC data and the synchronized image respectively, calculating the similarity of the two license plate characters, and determining the successfully matched license plate characters in combination with the matching degree table to obtain a license plate recognition result, including:

[0023] The license plate characters are extracted from the ETC data and the synchronized image respectively, the two license plate characters are compared by a template matching algorithm, the similarity of the two license plate characters is calculated, and the successfully matched license plate characters are determined in combination with the matching degree table to obtain a license plate recognition result.

[0024] A further technical solution is: fusing the synchronized image with the radar data to obtain a fusion result, including:

[0025] Identifying the target area in the synchronized image and the target area in the radar data;

[0026] Calculating the correlation between the target area in the synchronized image and the target area in the radar data;

[0027] When the correlation degree is greater than a set threshold, the two target regions are combined to obtain a fusion result.

[0028] A further technical solution is: the calculation of the correlation between the target area in the synchronized image and the target area in the radar data includes:

[0029] The distance between the center of the target area of ​​the radar data and the center of the target area in the synchronized image is calculated, and the intersection-over-union ratio of the target area in the synchronized image and the target area of ​​the radar data is calculated to obtain a correlation degree.

[0030] A further technical solution is: when the correlation is greater than a set threshold, the two target areas are combined to obtain a fusion result, including:

[0031] According to the distance in the correlation degree being less than the set first threshold and the intersection-over-union ratio being greater than the set second threshold, it is determined that the correlation degree is greater than the set threshold, and the two target regions are unioned to obtain a fusion result.

[0032] The present invention also provides a device for optimizing radar-visual fusion results based on ETC data, comprising:

[0033] An acquisition unit, used to acquire radar data collected by the millimeter-wave radar, images taken by the camera, and ETC data collected by the ETC antenna;

[0034] A synchronization unit, used for spatially synchronizing the image with the map by establishing a homography matrix, and temporally synchronizing the spatially synchronized image to obtain a synchronized image;

[0035] A matching unit, used for performing license plate matching using the ETC data and the synchronized image to obtain a license plate recognition result;

[0036] A fusion unit, used for fusing the synchronized image with the radar data to obtain a fusion result;

[0037] An output unit is used to output final license plate information according to the fusion result and the license plate recognition result.

[0038] The present invention further provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the above method when executing the computer program.

[0039] The beneficial effects of the present invention compared with the prior art are as follows: the present invention realizes the complementary advantages of radar and visual sensors by acquiring millimeter-wave radar, camera images and ETC data and combining multiple sensor information; firstly, the camera image is spatially and temporally synchronized with the high-precision map through the homography matrix and time synchronization technology to ensure the accurate coordination of the image and environmental data; then, the ETC data is used to match the synchronized image with the license plate, thereby improving the accuracy of license plate recognition; then, by fusing the synchronized image with the radar data, the robustness of target positioning and detection is optimized; finally, the license plate recognition result and the fused data are combined to output the final license plate information, thereby effectively improving the target recognition accuracy in complex environments, and by working in collaboration with the ETC system, further enhancing the management efficiency in complex traffic scenarios.

[0040] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0042] Figure 1 A schematic diagram of an application scenario of a method for optimizing radar-visual fusion results based on ETC data provided by an embodiment of the present invention;

[0043] Figure 2 A schematic diagram of a flow chart of a method for optimizing radar-visual fusion results based on ETC data provided by an embodiment of the present invention;

[0044] Figure 3 Schematic diagram of the sub-process of the method for optimizing the radar-visual fusion result based on ETC data provided by the embodiment of the present invention Figure 1 ;

[0045] Figure 4Schematic diagram of the sub-process of the method for optimizing the radar-visual fusion result based on ETC data provided by the embodiment of the present invention Figure 2 ;

[0046] Figure 5 Schematic diagram of the sub-process of the method for optimizing the radar-visual fusion result based on ETC data provided by the embodiment of the present invention Figure 3 ;

[0047] Figure 6 A schematic diagram of map points provided by an embodiment of the present invention;

[0048] Figure 7 A schematic diagram of punctuation marks in an image captured by a camera according to an embodiment of the present invention;

[0049] Figure 8 A schematic diagram of a converted image provided by an embodiment of the present invention;

[0050] Fig. 9 A radar-vision fusion area diagram provided in an embodiment of the present invention;

[0051] Fig.10 A schematic diagram of a fusion process provided by an embodiment of the present invention;

[0052] Fig.11 A schematic block diagram of a device for optimizing radar-visual fusion results based on ETC data provided by an embodiment of the present invention;

[0053] Fig.12 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0056] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0057] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0058] See also Figure 1 and Figure 2 , Figure 1 A schematic diagram of an application scenario of a method for optimizing radar-vision fusion results based on ETC data provided in an embodiment of the present invention. Figure 2 A schematic flow chart of a method for optimizing radar-visual fusion results based on ETC data provided in an embodiment of the present invention. The method for optimizing radar-visual fusion results based on ETC data is applied to a central system, and the millimeter-wave radar, camera and ETC antenna are integrated on the road-side system. By fusing the millimeter-wave radar, camera images and ETC data, the accuracy and robustness of target detection and recognition are improved. First, the homography matrix is ​​used to achieve spatial synchronization between the image and the map, and a high-precision image is obtained through time synchronization. Then, the ETC data is combined with the synchronized image to perform license plate matching and accurately identify the license plate information. In addition, the synchronized image is fused with the radar data, the correlation of the target area is calculated, and the fusion effect is judged by setting a threshold. This method effectively overcomes the limitations of a single sensor in a complex environment, especially in complex scenes such as highways and tunnels, and improves the efficiency of traffic management through the collaboration of the ETC system.

[0059] Figure 2 FIG. 1 is a flow chart of a method for optimizing radar-visual fusion results based on ETC data provided by an embodiment of the present invention. Figure 2 As shown, the method includes the following steps S110 to S150.

[0060] S110, acquiring radar data collected by the millimeter wave radar, images captured by the camera, and ETC data collected by the ETC antenna.

[0061] In this embodiment, the millimeter wave radar sensor: a millimeter wave radar device suitable for the application scenario (such as traffic monitoring or autonomous driving assistance) is selected. Such a device can transmit and receive reflected millimeter wave signals, and determine the distance, speed and angle of the target by analyzing these signals.

[0062] Set the radar's operating frequency, scanning range, resolution and other parameters according to specific needs. After initializing the radar device, it starts to continuously collect environmental information and generate data frames containing information such as target distance, relative speed, angle, etc. The raw radar data usually needs to be preprocessed, such as filtering to remove noise, clustering algorithms to attribute multiple point clouds to a single target, and coordinate transformation to map the detection results to the global coordinate system.

[0063] Equip high-resolution cameras and ensure that they are installed in positions and angles that cover key areas to capture clear license plate images and other traffic elements. Set the camera's video encoding format (such as H.264), frame rate (FPS), resolution and other parameters to suit storage and transmission requirements. You can use continuous recording or event-triggered mode (such as starting shooting when a vehicle enters a specific area) to control when to start recording video clips. Apply enhancement techniques (such as denoising and brightness adjustment) to the captured images to improve image quality; implement target detection and tracking algorithms to identify and locate objects of interest (such as vehicles and their license plates).

[0064] ETC readers / antennas are installed above or on both sides of the road to communicate wirelessly with the electronic tags on the vehicle, realizing the non-stop toll collection function while obtaining the vehicle identity information. Following DSRC (Dedicated Short Range Communication) or similar standards, a stable and reliable two-way communication link is established to ensure fast and accurate data exchange.

[0065] Whenever a vehicle passes through the ETC lane, the system automatically reads the information in the vehicle unit, including but not limited to the license plate number, vehicle model, and time stamp of the passage, and generates a corresponding transaction log. The received information is checked for integrity and verification code to prevent incorrect data from being entered into the system.

[0066] S120 , spatially synchronizing the image with the map by establishing a homography matrix, and temporally synchronizing the spatially synchronized image to obtain a synchronized image.

[0067] In this embodiment, the synchronized image refers to an image obtained after spatial and temporal synchronization between the map and the image.

[0068] In intelligent transportation systems, millimeter wave radar and cameras perceive traffic information in different ways, resulting in differences in the frequency, spatial distribution and dimension of the data collected by the two. Therefore, in order to achieve the same frequency and dimension presentation of radar data, synchronization must be performed in both space and time.

[0069] In one embodiment, see Figure 3 , the above-mentioned step S120 may include steps S121 to S124.

[0070] S121. Select at least four feature points in the map and the image as corresponding points.

[0071] In this embodiment, Figure 6 The high-precision maps shown and Figure 7 In the image taken by the camera shown, at least four easily identifiable and stable feature points are selected as corresponding points. These points should be distributed in different positions of the image, such as corners of intersections, traffic signs or street lights, etc., to ensure the stability of the homography matrix and the accuracy of the transformation result.

[0072] The selected feature points must meet the following conditions: they are not only clearly visible in both images, but also have relatively fixed positions and are not easily affected by environmental changes.

[0073] S122, using the corresponding points to calculate the homography matrix through the findHomography function in the OpenCV library.

[0074] In this embodiment, the OpenCV computer vision library provides a function named findHomography for calculating the homography matrix according to the selected corresponding points.

[0075] The homography matrix H is a 3x3 matrix that describes the geometric transformation relationship between two planes. Specifically, it can transform a point (x, y) on one plane to the corresponding point (x', y') on another plane. The formula is: Among them, (x, y) is the point in the high-precision map, and (x′, y′) is the corresponding point in the camera image.

[0076] S123: Project the map to the viewing angle of the camera using the homography matrix to obtain a spatially synchronized image.

[0077] In this embodiment, the high-precision map under the bird's-eye view can be converted into a perspective that matches the camera image through the calculated homography matrix, thereby achieving a consistent conversion from the high-precision mapping picture to the camera image. Figure 8 shown.

[0078] Specifically, using the calculated homography matrix H, all points on the high-precision map are transformed into the corresponding camera image coordinate system according to the above formula.

[0079] After this step, the original high-precision map will be deformed to the same angle and scale as the camera image, achieving spatial synchronization between the two. At this time, the map data and image data are based on the same spatial reference, but there may still be a time asynchrony.

[0080] S124: Perform time synchronization on the spatially synchronized image to obtain a synchronized image.

[0081] In this embodiment, even if they are synchronized in space, different sensors may have different sampling rates or delays, so it is necessary to ensure the temporal consistency between them.

[0082] Use the Network Time Protocol (NTP) or other similar technologies to synchronize the clocks of all sensing devices so that the timestamps they record are consistent. This step ensures that all collected data is synchronized not only in space but also in time, which is the so-called "space-time synchronization".

[0083] In summary, through the above steps S121 to S124, data from different sources (such as millimeter wave radar, camera) can be synchronized in both space and time, and finally synchronized image data that accurately reflects the real world situation can be obtained. Such synchronization processing is crucial to improving the performance of intelligent transportation systems and helps to monitor and manage traffic flows more accurately.

[0084] S130, performing license plate matching using the ETC data and the synchronized image to obtain a license plate recognition result.

[0085] In this embodiment, the license plate recognition result refers to the result obtained after matching the license plate characters appearing in the ERC data and the synchronized image, that is, the license plate characters appearing in both the ERC data and the synchronized image.

[0086] In the intelligent transportation system, the license plate image captured by the video may be affected by factors such as driving at night, driving against the light, a damaged license plate or bad weather, resulting in blurred images or difficulty in recognition. Therefore, matching the characters recognized by the ETC (electronic toll collection system) with the characters in the video can significantly improve the recognition rate. Specifically, this embodiment achieves the acquisition of license plate recognition results through the following steps.

[0087] In one embodiment, see Figure 4 , the above-mentioned step S130 may include steps S131 to S132.

[0088] S131. Create a matching table based on existing license plate characters, where the matching table records similarity scores between different characters.

[0089] In this embodiment, a matching table containing all possible license plate characters is constructed. This table records the similarity scores between different characters and is used to evaluate whether two characters may be mistaken for each other.

[0090] For example, in actual applications, the characters "0" and "Q" are very similar in some cases, which can easily lead to recognition errors. Through the pre-built matching table, the relationship between the two characters can be more accurately judged in subsequent processing.

[0091] S132, extracting license plate characters from the ETC data and the synchronized image respectively, and calculating the similarity between the two license plate characters, and determining the successfully matched license plate characters in combination with the matching degree table to obtain a license plate recognition result.

[0092] In this embodiment, license plate characters are extracted from the ETC data and the synchronized image respectively, the two license plate characters are compared by a template matching algorithm, the similarity of the two license plate characters is calculated, and the successfully matched license plate characters are determined in combination with the matching degree table to obtain a license plate recognition result.

[0093] Specifically, OCR (Optical Character Recognition) technology is used to extract the characters on the license plate from the synchronized image.

[0094] At the same time, the license plate character information from the ETC system is obtained. The ETC system usually has a higher recognition accuracy because it directly reads the tag information on the vehicle based on radio frequency communication.

[0095] The character information from the above two sources is compared, and the template matching algorithm is used to compare the differences between the two. The overall similarity is calculated based on the similarity score provided by the matching table. If the calculated overall similarity score is higher than the preset threshold (such as 95%), the match is considered successful; otherwise, the character is rejected to avoid false positives.

[0096] Specifically, a standard template library containing all possible license plate characters (such as letters and numbers) is created. Each template is an idealized character representation, which can be a pre-defined image or font model.

[0097] For each standard template, its unique visual features are extracted, such as edges, contours, textures, etc. These features will be used in the subsequent matching process.

[0098] The license plate characters extracted from the image are normalized, including size adjustment, position correction, brightness and contrast adjustment, etc., to ensure that they have similar properties to the characters in the template library.

[0099] Use image processing technology to remove noise from character images, improve character clarity, and enhance matching accuracy.

[0100] Apply the sliding window technique to move the window over the target character image and calculate the similarity between the sub-image within the window and the template. Calculate the similarity score between the template and the target character based on the information provided by the matching degree table. This usually involves pixel-level comparison and using the matching degree table to evaluate possible confusions between different characters. Set a similarity threshold (e.g., 95%). If the calculated overall similarity score is higher than this threshold, the match is considered successful; otherwise, the character recognition is rejected.

[0101] Considering that there may be cases of character deformation or partial occlusion in the actual situation, multiple templates can be used for matching and the best matching result can be selected. Combine the license plate format rules and other known information (such as the abbreviation of the province) to further verify the correctness of the matching result.

[0102] Comprehensively consider the information from ETC data and video images, and obtain the most matching license plate characters through the template matching algorithm. Once the best matching characters that meet the conditions are found, the final license plate recognition result is obtained.

[0103] Suppose there is the following scenario:

[0104] The license plate characters recognized by the ETC system are "Yue B12345".

[0105] The license plate captured in the video image may be blurred or partially occluded, but certain characters can still be read through OCR technology, such as "Yue B1_34_".

[0106] In this example, use the template matching algorithm to compare each character in the video image with the corresponding character in the ETC data one by one. For difficult-to-identify characters (such as the "_" here), all possible character templates will be tried for matching until the closest one is found.

[0107] Use the matching degree table to assist in judging which characters are easily confused and adjust the similarity score accordingly. Finally, confirm the license plate characters with successful matching to form a complete license plate recognition result.

[0108] In summary, through the above steps S131 to S132, not only the accuracy of license plate character recognition is improved, but also the spatial and temporal consistency of the recognition result is ensured. This method combines the efficiency of the ETC system and the advantages of video monitoring, and can effectively overcome the challenges faced by a single recognition method, providing more reliable data support for intelligent traffic management. Finally, the license plate recognition result refers to the license plate characters that appear in both the ETC data and the synchronized image, ensuring the accuracy and reliability of the recognition.

[0109] S140. Fuse the synchronized image with the radar data to obtain a fusion result.

[0110] In this embodiment, the above-mentioned fusion result refers to a process of combining data from the millimeter-wave radar and the video detector through a series of specific processing processes to obtain more accurate and comprehensive target information.

[0111] In one embodiment, see Figure 5 , the above-mentioned step S140 may include steps S141 to S143.

[0112] S141, identifying a target area in the synchronized image and a target area in the radar data.

[0113] In this embodiment, the target area refers to the area where the vehicle is located.

[0114] First, on the basis of the spatiotemporal synchronization, the targets in the synchronized images and radar data are identified and their respective target areas are determined. Here, the target area refers to a specific spatial range that may contain objects of interest (such as vehicles, pedestrians, etc.).

[0115] For each identified target, its corresponding ROI (Region of Interest) is defined, including but not limited to target type, pixel position, and corresponding spatial position.

[0116] S142, calculating the correlation between the target area in the synchronized image and the target area in the radar data.

[0117] Specifically, the distance between the center of the target area of ​​the radar data and the center of the target area in the synchronized image is calculated, and the intersection-and-union ratio of the target area in the synchronized image and the target area of ​​the radar data is calculated to obtain the correlation degree.

[0118] Calculate the target area of ​​the radar data (denoted as S R , with its center point being O R ) and the target area in the synchronized image (denoted as S P , with its center point being O P ) between the centers ΔO, that is, |O R -O P |.

[0119] Furthermore, the ratio of the intersection area to the union area of ​​the two target regions is calculated, namely, IOU (Intersection over Union). This ratio reflects the degree of overlap between the two regions. IOU = (S R ∩S P ) / (S R ∪S P ).

[0120] Based on the above calculation results, the preset target correlation threshold O is used. r and S r Only when the center distance between the radar and the video target area is less than O r , and the intersection-joint ratio is greater than S r Only when the two are considered to exist effectively can it be considered that there is a correlation between them.

[0121] Refers to a quantitative indicator used to measure the similarity or correlation between targets detected by millimeter-wave radar and targets captured by video detectors. It specifically reflects the degree of match between targets identified by two different sources (i.e., radar data and video images). The calculation of correlation takes into account the following aspects:

[0122] Center distance (ΔO): This refers to the center point O of the radar target area R and the center point O of the target area of ​​the image P If this distance is less than the set threshold O r , it is believed that the two targets may represent the same actual object.

[0123] Intersection over Union (IOU): It is the abbreviation of Intersection over Union, which is used to describe the degree of overlap between two target areas. It is the ratio of the intersection area to the union area of ​​the two areas. When the IOU is greater than the set threshold S r , it means that the two target regions have significant overlap, further supporting the possibility that they correspond to the same physical object.

[0124] Therefore, the correlation combines the above two indicators. Only when both meet the corresponding conditions, that is, the center distance is small enough and the intersection ratio is large enough, is it considered that there is a valid correlation between the radar target and the video target. This correlation indicates that the data of the two sensors are likely to point to the same object in the real world, thus providing a basis for subsequent data fusion.

[0125] In short, in radar-vision fusion applications, correlation is the criterion used to determine whether targets from different sensors should be recognized as the same entity, which is crucial to improving the accuracy and reliability of multi-sensor systems.

[0126] The specific calculation formula is as follows: O r , S r is the target correlation threshold, which is only valid when the center distance between the radar and the video target area is less than O r , the intersection ratio is greater than S r When , the fusion target of the two is output; ΔO, IOU is as follows Fig.10If a radar target is not successfully associated with any video target, the target is output separately; if a radar target is successfully associated with multiple video targets, a highly correlated fusion target is output.

[0127] S143. When the correlation degree is greater than a set threshold, the two target regions are combined to obtain a fusion result.

[0128] In this embodiment, according to the distance in the correlation degree being less than the set first threshold value and the intersection-over-union ratio being greater than the set second threshold value, it is determined that the correlation degree is greater than the set threshold value, and the two target areas are unioned to obtain a fusion result. If a radar target is successfully associated with at least one video target, the two target areas are unioned to form a final fusion target. The union here means that all available information from the two sensors is comprehensively considered to provide a more complete target description. If the target area of ​​a radar fails to be successfully associated with the target area of ​​any image, the information of the target areas of these radars is output separately.

[0129] If a radar target can be successfully associated with multiple video targets at the same time, the combination with the highest correlation (i.e., the smallest center distance and the largest intersection-over-union ratio) is selected as the fusion target output.

[0130] In summary, in the process of executing step S140, the target area of ​​the synchronized image and radar data is identified, the correlation is calculated, and whether to fuse is determined according to a preset threshold, and finally a fusion result is obtained. This process not only improves the limitations of a single sensor information, but also enhances the system's adaptability and accuracy in complex environments.

[0131] S150. Output final license plate information according to the fusion result and the license plate recognition result.

[0132] In step S140, the fusion analysis of data from multiple sensors (such as millimeter wave radar and video images) has been completed. This process may include but is not limited to target correlation calculation, tracking algorithm application, etc., to ensure that data from different sources can accurately correspond to the same physical object. This fusion result provides more abundant and reliable basic information for license plate recognition.

[0133] At the same time, in a separate processing path, a high-resolution camera is used to capture the vehicle image, and the license plate number and other relevant information are extracted from the image through advanced optical character recognition (OCR) technology or other dedicated license plate recognition algorithms. The results of this step constitute the specific content of the license plate recognition.

[0134] After entering the S150 stage, the system will combine the above two aspects of information - the fused multi-sensor data and the license plate recognition results obtained directly from the image - to conduct a comprehensive evaluation. Specifically, the system will consider the following factors:

[0135] Consistency check: Verify that the information from different sources is consistent. For example, if two sensors are pointing at the same car, their corresponding license plate information should be the same.

[0136] Confidence score: Each recognition result is usually accompanied by a confidence score indicating the accuracy of the recognition. The system selects the license plate information with the highest confidence score as the candidate.

[0137] Timestamp synchronization: Ensure that all data points involved in decision-making occur in a similar time period, thereby reducing misjudgments due to delays.

[0138] After the above comprehensive judgment, the system will select the most credible license plate information as the final output. This means that only when all relevant conditions are met and the system is sure that the selected license plate information is correct, it will be officially output to the user or recorded for subsequent application scenarios, such as traffic management, security monitoring, etc.

[0139] In summary, step S150 not only reflects how to use multi-source data to improve the accuracy of license plate recognition, but also demonstrates how to ensure the reliability of output results through intelligent decision-making mechanisms, thereby providing more accurate services for actual application scenarios.

[0140] The above method aims to improve the accuracy and robustness of license plate recognition, especially in complex environments. ETC equipment can accurately read the vehicle's license plate information through wireless communication technology. The algorithm matches the license plate characters recognized by ETC with the license plate characters captured in the video, and optimizes the recognition process by calculating the similarity, thereby solving the problem of blurred or difficult to recognize license plate images under weather conditions such as night driving, backlight, fast driving, rain, snow, and fog. This optimization can accurately perceive the real-time passing of vehicles, improve data quality, and provide more reliable support for traffic management departments.

[0141] The key technologies of the algorithm include: high-precision image map, homography matrix, coordinate transformation, time alignment technology and template matching algorithm. They work together to achieve efficient fusion of radar and visual sensor data. Compared with the prior art, the method of this embodiment accurately projects the high-precision map to the camera's perspective by establishing a homography matrix, ensuring that the geometric relationship between the map and the camera image is accurate. Using the inverse transformation of the homography matrix, the image coordinates are converted into geographic coordinates (latitude and longitude information) to achieve accurate conversion from image to geographic location. By ensuring the data time synchronization of the visual sensor and the radar sensor, errors caused by time difference are avoided. Through the template matching algorithm, the license plate image taken by the visual sensor is matched with the license plate information recognized by the ETC device to improve the accuracy of license plate recognition.

[0142] In addition, high-precision map images play an important auxiliary role in radar-vision fusion. By aligning the target position detected by the radar with the lane lines, road signs and other information in the map, the accuracy of target positioning is significantly improved.

[0143] In summary, the method of this embodiment effectively combines the advantages of radar and visual sensors, significantly improves the accuracy and robustness of license plate recognition and target detection, and overcomes the limitations of a single sensor in complex environments. Through integration with ETC equipment, this algorithm further improves the efficiency of traffic management in complex scenarios such as highways and tunnels, providing strong support for the efficient operation of intelligent transportation systems.

[0144] The above-mentioned radar-visual fusion result optimization method based on ETC data realizes the complementary advantages of radar and visual sensors by acquiring millimeter-wave radar, camera images and ETC data, and combining multiple sensor information; first, the camera image is spatially and temporally synchronized with the high-precision map through the homography matrix and time synchronization technology to ensure the accurate coordination of the image and environmental data; then, the ETC data is used to match the synchronized image with the license plate, which improves the accuracy of license plate recognition; then, by fusing the synchronized image with the radar data, the robustness of target positioning and detection is optimized; finally, the license plate recognition result and the fused data are combined to output the final license plate information, which effectively improves the target recognition accuracy in complex environments, and by working in collaboration with the ETC system, further enhances the management efficiency in complex traffic scenarios.

[0145] Fig.11 FIG. 3 is a schematic block diagram of a radar-visual fusion result optimization device 300 based on ETC data provided by an embodiment of the present invention. Fig.11As shown, corresponding to the above-mentioned method for optimizing the results of thunder and vision fusion based on ETC data, the present invention further provides a device 300 for optimizing the results of thunder and vision fusion based on ETC data. The device 300 for optimizing the results of thunder and vision fusion based on ETC data includes a unit for executing the above-mentioned method for optimizing the results of thunder and vision fusion based on ETC data, and the device can be configured in a server. Fig.11 The device 300 for optimizing the radar-vision fusion result based on ETC data includes an acquisition unit 301 , a synchronization unit 302 , a matching unit 303 , a fusion unit 304 and an output unit 305 .

[0146] The acquisition unit 301 is used to acquire radar data collected by the millimeter-wave radar, images taken by the camera, and ETC data collected by the ETC antenna; the synchronization unit 302 is used to spatially synchronize the image with the map by establishing a homography matrix, and to temporally synchronize the spatially synchronized image to obtain a synchronized image; the matching unit 303 is used to use the ETC data and the synchronized image to match the license plate to obtain a license plate recognition result; the fusion unit 304 is used to fuse the synchronized image with the radar data to obtain a fusion result; the output unit 305 is used to output the final license plate information according to the fusion result and the license plate recognition result.

[0147] In one embodiment, the synchronization unit 302 includes:

[0148] A selection subunit is used to select at least four feature points in the map and the image as corresponding points; a calculation subunit is used to use the corresponding points to calculate the homography matrix through the findHomography function in the OpenCV library; a projection subunit is used to project the map to the viewing angle of the camera using the homography matrix to obtain a spatially synchronized image; a time synchronization subunit is used to perform time synchronization on the spatially synchronized image to obtain a synchronized image.

[0149] In one embodiment, the time synchronization subunit is used to perform time synchronization on the spatially synchronized image using a network time protocol to obtain a synchronized image.

[0150] In one embodiment, the matching unit 303 includes:

[0151] The table creation subunit is used to create a matching table based on existing license plate characters, and the matching table records the similarity scores between different characters; the similarity calculation subunit is used to extract license plate characters from the ETC data and the synchronized image respectively, and calculate the similarity of the two license plate characters, and determine the successfully matched license plate characters in combination with the matching table to obtain the license plate recognition result.

[0152] In one embodiment, the similarity calculation subunit is used to extract license plate characters from the ETC data and the synchronized image respectively, compare the two license plate characters through a template matching algorithm, calculate the similarity of the two license plate characters, and determine the successfully matched license plate characters in combination with the matching table to obtain a license plate recognition result.

[0153] In one embodiment, the fusion unit 304 includes:

[0154] The identification subunit is used to identify the target area in the synchronized image and the target area in the radar data; the correlation calculation subunit is used to calculate the correlation between the target area in the synchronized image and the target area in the radar data; the union subunit is used to union the two target areas when the correlation is greater than a set threshold to obtain a fusion result.

[0155] In one embodiment, the correlation calculation subunit is used to calculate the distance between the center of the target area of ​​the radar data and the center of the target area in the synchronized image, and calculate the intersection and union ratio of the target area in the synchronized image and the target area of ​​the radar data to obtain the correlation.

[0156] In one embodiment, the union subunit is used to determine that the correlation degree is greater than the set threshold value based on the distance in the correlation degree being less than the set first threshold value and the intersection-union ratio being greater than the set second threshold value, and to union the two target areas to obtain a fusion result.

[0157] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned radar-vision fusion result optimization device 300 based on ETC data and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and brevity of description, it will not be repeated here.

[0158] The above-mentioned device 300 for optimizing the radar-visual fusion result based on ETC data can be implemented in the form of a computer program. The computer program can be used in the following manner: Fig.12 Runs on the computer device shown.

[0159] See also Fig.12 , Fig.12 5 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a server, wherein the server may be an independent server or a server cluster composed of multiple servers.

[0160] See also Fig.12The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0161] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, and when the program instructions are executed, the processor 502 can execute a radar-visual fusion result optimization method based on ETC data.

[0162] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0163] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a method for optimizing radar-visual fusion results based on ETC data.

[0164] The network interface 505 is used to communicate with other devices over the network. Fig.12 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0165] The processor 502 is used to run the computer program 5032 stored in the memory to implement the following steps:

[0166] Acquire radar data collected by a millimeter-wave radar, images taken by a camera, and ETC data collected by an ETC antenna; spatially synchronize the image with a map by establishing a homography matrix, and temporally synchronize the spatially synchronized image to obtain a synchronized image; perform license plate matching using the ETC data and the synchronized image to obtain a license plate recognition result; fuse the synchronized image with the radar data to obtain a fusion result; and output final license plate information based on the fusion result and the license plate recognition result.

[0167] In one embodiment, when the processor 502 implements the step of spatially synchronizing the image with the map by establishing a homography matrix and temporally synchronizing the spatially synchronized image to obtain a synchronized image, the processor 502 specifically implements the following steps:

[0168] At least four feature points are selected in the map and the image as corresponding points; the homography matrix is ​​calculated using the corresponding points through the findHomography function in the OpenCV library; the map is projected to the viewing angle of the camera using the homography matrix to obtain a spatially synchronized image; and the spatially synchronized image is time synchronized to obtain a synchronized image.

[0169] In one embodiment, when the processor 502 implements the step of performing time synchronization on the spatially synchronized image to obtain a synchronized image, the processor 502 specifically implements the following steps:

[0170] The spatially synchronized image is time synchronized using a network time protocol to obtain a synchronized image.

[0171] In one embodiment, when the processor 502 implements the step of using the ETC data and the synchronized image to perform license plate matching to obtain a license plate recognition result, the processor 502 specifically implements the following steps:

[0172] A matching table is created based on existing license plate characters, and the matching table records similarity scores between different characters; license plate characters are extracted from the ETC data and the synchronized image respectively, and the similarity between the two license plate characters is calculated, and the successfully matched license plate characters are determined in combination with the matching table to obtain a license plate recognition result.

[0173] In one embodiment, when the processor 502 implements the step of extracting license plate characters from the ETC data and the synchronized image, respectively, calculating the similarity of the two license plate characters, and determining the successfully matched license plate characters in combination with the matching degree table to obtain the license plate recognition result, the processor 502 specifically implements the following steps:

[0174] The license plate characters are extracted from the ETC data and the synchronized image respectively, the two license plate characters are compared by a template matching algorithm, the similarity of the two license plate characters is calculated, and the successfully matched license plate characters are determined in combination with the matching degree table to obtain a license plate recognition result.

[0175] In one embodiment, when the processor 502 implements the step of fusing the synchronized image with the radar data to obtain a fusion result, the processor 502 specifically implements the following steps:

[0176] Identify the target area in the synchronized image and the target area in the radar data; calculate the correlation between the target area in the synchronized image and the target area in the radar data; when the correlation is greater than a set threshold, find the union of the two target areas to obtain a fusion result.

[0177] In one embodiment, when the processor 502 implements the step of calculating the correlation between the target area in the synchronized image and the target area in the radar data, the processor 502 specifically implements the following steps:

[0178] The distance between the center of the target area of ​​the radar data and the center of the target area in the synchronized image is calculated, and the intersection-over-union ratio of the target area in the synchronized image and the target area of ​​the radar data is calculated to obtain a correlation degree.

[0179] In one embodiment, when the processor 502 implements the step of obtaining a fusion result by obtaining a union of the two target regions when the correlation degree is greater than a set threshold, the processor 502 specifically implements the following steps:

[0180] According to the distance in the correlation degree being less than the set first threshold and the intersection-over-union ratio being greater than the set second threshold, it is determined that the correlation degree is greater than the set threshold, and the two target regions are unioned to obtain a fusion result.

[0181] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0182] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0183] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor executes the following steps:

[0184] Acquire radar data collected by a millimeter-wave radar, images taken by a camera, and ETC data collected by an ETC antenna; spatially synchronize the image with a map by establishing a homography matrix, and temporally synchronize the spatially synchronized image to obtain a synchronized image; perform license plate matching using the ETC data and the synchronized image to obtain a license plate recognition result; fuse the synchronized image with the radar data to obtain a fusion result; and output final license plate information based on the fusion result and the license plate recognition result.

[0185] In one embodiment, when the processor executes the computer program to implement the step of spatially synchronizing the image with the map by establishing a homography matrix, and temporally synchronizing the spatially synchronized image to obtain a synchronized image, the processor specifically implements the following steps:

[0186] At least four feature points are selected in the map and the image as corresponding points; the homography matrix is ​​calculated using the corresponding points through the findHomography function in the OpenCV library; the map is projected to the viewing angle of the camera using the homography matrix to obtain a spatially synchronized image; and the spatially synchronized image is time synchronized to obtain a synchronized image.

[0187] In one embodiment, when the processor executes the computer program to implement the step of performing time synchronization on the spatially synchronized image to obtain a synchronized image, the processor specifically implements the following steps:

[0188] The spatially synchronized image is time synchronized using a network time protocol to obtain a synchronized image.

[0189] In one embodiment, when the processor executes the computer program to implement the step of using the ETC data and the synchronized image to perform license plate matching to obtain a license plate recognition result, the processor specifically implements the following steps:

[0190] A matching table is created based on existing license plate characters, and the matching table records similarity scores between different characters; license plate characters are extracted from the ETC data and the synchronized image respectively, and the similarity between the two license plate characters is calculated, and the successfully matched license plate characters are determined in combination with the matching table to obtain a license plate recognition result.

[0191] In one embodiment, when the processor executes the computer program to implement the steps of extracting license plate characters from the ETC data and the synchronized image, respectively, calculating the similarity of the two license plate characters, and determining the successfully matched license plate characters in combination with the matching degree table to obtain the license plate recognition result, the following steps are specifically implemented:

[0192] The license plate characters are extracted from the ETC data and the synchronized image respectively, the two license plate characters are compared by a template matching algorithm, the similarity of the two license plate characters is calculated, and the successfully matched license plate characters are determined in combination with the matching degree table to obtain a license plate recognition result.

[0193] In one embodiment, when the processor executes the computer program to implement the step of fusing the synchronized image with the radar data to obtain a fusion result, the processor specifically implements the following steps:

[0194] Identify the target area in the synchronized image and the target area in the radar data; calculate the correlation between the target area in the synchronized image and the target area in the radar data; when the correlation is greater than a set threshold, find the union of the two target areas to obtain a fusion result.

[0195] In one embodiment, when the processor executes the computer program to implement the step of calculating the correlation between the target area in the synchronized image and the target area in the radar data, the processor specifically implements the following steps:

[0196] The distance between the center of the target area of ​​the radar data and the center of the target area in the synchronized image is calculated, and the intersection-over-union ratio of the target area in the synchronized image and the target area of ​​the radar data is calculated to obtain a correlation degree.

[0197] In one embodiment, when the processor executes the computer program to implement the step of finding the union of the two target regions to obtain a fusion result when the correlation degree is greater than a set threshold, the processor specifically implements the following steps:

[0198] According to the distance in the correlation degree being less than the set first threshold and the intersection-over-union ratio being greater than the set second threshold, it is determined that the correlation degree is greater than the set threshold, and the two target regions are unioned to obtain a fusion result.

[0199] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0200] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0201] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0202] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0203] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0204] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for optimizing radar-visual fusion results based on ETC data, characterized in that: include: Obtain radar data collected by millimeter-wave radar, images taken by cameras, and ETC data collected by ETC antennas; The image is spatially synchronized with the map by establishing a homography matrix, and the spatially synchronized image is temporally synchronized to obtain a synchronized image; Using the ETC data and the synchronized image to perform license plate matching to obtain a license plate recognition result; fusing the synchronized image with the radar data to obtain a fusion result; The final license plate information is output according to the fusion result and the license plate recognition result.

2. The method for optimizing the radar-visual fusion results based on ETC data according to claim 1, characterized in that: The step of spatially synchronizing the image with the map by establishing a homography matrix and temporally synchronizing the spatially synchronized image to obtain a synchronized image includes: Selecting at least four feature points in the map and the image as corresponding points; Use the corresponding points to calculate the homography matrix using the findHomography function in the OpenCV library; Projecting the map to the viewing angle of the camera using the homography matrix to obtain a spatially synchronized image; The spatially synchronized image is time synchronized to obtain a synchronized image.

3. The method for optimizing the radar-visual fusion results based on ETC data according to claim 2, characterized in that: The step of performing time synchronization on the spatially synchronized image to obtain a synchronized image includes: The spatially synchronized image is time synchronized using a network time protocol to obtain a synchronized image.

4. The method for optimizing the radar-visual fusion result based on ETC data according to claim 1, characterized in that: The using the ETC data and the synchronized image to perform license plate matching to obtain a license plate recognition result includes: Creating a matching table based on existing license plate characters, wherein the matching table records similarity scores between different characters; The license plate characters are extracted from the ETC data and the synchronized image respectively, and the similarity between the two license plate characters is calculated. The successfully matched license plate characters are determined in combination with the matching degree table to obtain a license plate recognition result.

5. The method for optimizing the radar-visual fusion result based on ETC data according to claim 4, characterized in that: The extracting of license plate characters from the ETC data and the synchronized image respectively, calculating the similarity of the two license plate characters, and determining the successfully matched license plate characters in combination with the matching degree table to obtain a license plate recognition result, includes: The license plate characters are extracted from the ETC data and the synchronized image respectively, the two license plate characters are compared by a template matching algorithm, the similarity of the two license plate characters is calculated, and the successfully matched license plate characters are determined in combination with the matching degree table to obtain a license plate recognition result.

6. The method for optimizing the radar-visual fusion result based on ETC data according to claim 1, characterized in that: The step of fusing the synchronized image with the radar data to obtain a fusion result includes: Identifying the target area in the synchronized image and the target area in the radar data; Calculating the correlation between the target area in the synchronized image and the target area in the radar data; When the correlation degree is greater than a set threshold, the two target regions are combined to obtain a fusion result.

7. The method for optimizing the radar-visual fusion result based on ETC data according to claim 6, characterized in that: The calculating the correlation between the target area in the synchronized image and the target area in the radar data includes: The distance between the center of the target area of ​​the radar data and the center of the target area in the synchronized image is calculated, and the intersection-over-union ratio of the target area in the synchronized image and the target area of ​​the radar data is calculated to obtain a correlation degree.

8. The method for optimizing the radar-visual fusion result based on ETC data according to claim 7, characterized in that: When the correlation degree is greater than a set threshold, the two target regions are merged to obtain a fusion result, including: According to the distance in the correlation degree being less than the set first threshold and the intersection-over-union ratio being greater than the set second threshold, it is determined that the correlation degree is greater than the set threshold, and the two target regions are unioned to obtain a fusion result.

9. A device for optimizing the results of radar-visual fusion based on ETC data, characterized in that: include: An acquisition unit, used to acquire radar data collected by the millimeter-wave radar, images taken by the camera, and ETC data collected by the ETC antenna; A synchronization unit, used for spatially synchronizing the image with the map by establishing a homography matrix, and temporally synchronizing the spatially synchronized image to obtain a synchronized image; A matching unit, used for performing license plate matching using the ETC data and the synchronized image to obtain a license plate recognition result; A fusion unit, used for fusing the synchronized image with the radar data to obtain a fusion result; An output unit is used to output final license plate information according to the fusion result and the license plate recognition result.

10. A computer device, characterized in that: The computer device comprises a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

Images