A multi-source information fusion recognition vehicle non-sensing toll method and system

Abstract

The application provides a vehicle non-inductive toll collection method and system based on multi-source information fusion recognition, and the steps of the method include: when a vehicle travels to a toll area, the vehicle information of the vehicle is recognized by a front camera, and a vehicle ID corresponding to the vehicle information is constructed, the vehicle information including a vehicle license plate, a vehicle color and a vehicle position; after the vehicle travels through the shooting range of the front camera, the vehicle license plate is recognized again by a position tracking camera, the recognized vehicle license plate is compared with the current vehicle license plate corresponding to the vehicle ID, and it is determined whether to update the vehicle information and the vehicle ID; when the vehicle travels to a toll island, the vehicle license plate is recognized again by a lane camera device, the recognized vehicle license plate is compared with the current vehicle license plate corresponding to the vehicle ID, and it is determined whether to update the vehicle information and the vehicle ID; a transaction is performed based on the final vehicle information and the vehicle ID, and the vehicle ID is deleted after the transaction.

Description

Technical Field

[0001] This invention relates to the field of electronic toll collection technology for highways, and in particular to a method and system for contactless toll collection of vehicles based on multi-source information fusion and identification. Background Technology

[0002] Electronic Toll Collection (ETC), also known as non-stop toll collection, is an automated toll collection method that utilizes advanced information and communication technologies. This system involves installing antennas and vehicle type recognition systems at the entrances and exits of toll stations on toll roads, highways, bridges, and tunnels, and installing on-board units (such as ETC devices or dedicated electronic cards) on vehicles. When vehicles pass through these toll stations, they do not need to stop or pay manually; the system automatically identifies the vehicle information and deducts the corresponding toll from pre-stored funds.

[0003] ETC uses Dedicated Short Range Communication (DSRC) technology to achieve non-stop toll collection through wireless signal interaction. In practical applications, it is prone to encountering interference from adjacent lanes, reflections from trucks, and reflections from information boards, which can cause ETC transaction failures. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a vehicle contactless toll collection method based on multi-source information fusion and identification, in order to eliminate or improve one or more defects existing in the prior art.

[0005] One aspect of the present invention provides a vehicle contactless toll collection method based on multi-source information fusion and recognition. The method is based on a front-mounted recognition device and a lane-mounted camera device. The front-mounted recognition device is positioned in front of the lane-mounted camera device and includes a front-mounted camera and a location-tracking camera. The steps of the method include:

[0006] When a vehicle travels into a toll area, the front-facing camera identifies the vehicle's information and constructs a corresponding vehicle ID, which includes the vehicle's license plate, color, and location.

[0007] After the vehicle passes the shooting range of the front camera, the vehicle license plate is identified again by the location tracking camera. The identified vehicle license plate is compared with the current vehicle license plate corresponding to the vehicle ID to determine whether to update the vehicle information and vehicle ID.

[0008] When a vehicle reaches the toll island, the vehicle's license plate is re-identified by the lane camera equipment. The identified license plate is compared with the current license plate corresponding to the vehicle ID to determine whether to update the vehicle information and vehicle ID.

[0009] Transactions are conducted based on the final vehicle information and vehicle ID, and the vehicle ID is deleted after the transaction.

[0010] By deploying multi-source information fusion identification equipment in the toll station area, this solution performs multi-source fusion identification of information such as the location, license plate, OBU, and vehicle color of vehicles passing through the toll station area, and links with the toll system in real time to accurately identify vehicles that have completed transactions. This improves the passage ratio of ETC vehicles and the accuracy of license plate payment and prepayment identification in manual toll lanes, ultimately improving the accuracy and reliability of toll collection at toll stations and providing a better travel service experience.

[0011] In some embodiments of the present invention, the front-facing identification device further includes a roadside antenna. If the vehicle is an ETC vehicle equipped with an on-board terminal, in the step of identifying the vehicle information through the front-facing camera, the OBU record is obtained from the vehicle's on-board terminal through the roadside antenna, and the vehicle license plate in the OBU record is compared with the vehicle license plate identified by the front-facing camera.

[0012] In some embodiments of the present invention, in the step of comparing the vehicle license plate in the OBU record with the vehicle license plate identified by the front camera, if the difference between the vehicle license plate in the OBU record and the vehicle license plate identified by the front camera is less than a preset number of characters, then the vehicle license plate in the OBU record is used as the vehicle license plate in the vehicle information.

[0013] In some embodiments of the present invention, in the step of re-identifying the vehicle license plate through the location tracking camera, comparing the identified vehicle license plate with the current vehicle license plate corresponding to the vehicle ID, and determining whether to update the vehicle information and vehicle ID, the vehicle license plate and vehicle position are re-identified through the location tracking camera, the vehicle license plate identified by the location tracking camera is compared with the vehicle license plate in the current vehicle information, and the vehicle position identified by the location tracking camera is compared with the vehicle position in the current vehicle information to determine whether to update the vehicle information and vehicle ID.

[0014] In some embodiments of the present invention, the location tracking camera and the front-facing camera are in one-to-one correspondence. The location tracking camera tracks the corresponding vehicle that has moved out of the field of view of the front-facing camera. In the step of comparing the license plate of the vehicle identified by the location tracking camera with the license plate of the vehicle in the current vehicle information, and comparing the vehicle position identified by the location tracking camera with the vehicle position in the current vehicle information, to determine whether to update the vehicle information and vehicle ID:

[0015] If the vehicle location identified by the location tracking camera is the same as the vehicle location in the current vehicle information, and the difference between the vehicle license plate in the current vehicle information and the vehicle license plate identified by the location tracking camera is less than a preset number of characters, then the vehicle information and vehicle ID remain unchanged.

[0016] If the vehicle position identified by the location tracking camera is different from the vehicle position in the current vehicle information, then update the vehicle position in the vehicle information.

[0017] If the vehicle location identified by the location tracking camera is different from the vehicle location in the current vehicle information, and the difference between the vehicle license plate in the current vehicle information and the vehicle license plate identified by the location tracking camera is not less than a preset number of characters, then the vehicle location and vehicle license plate in the vehicle information are updated; and a new vehicle ID is generated to update the original vehicle ID.

[0018] In some embodiments of the present invention, when a vehicle approaches a toll island, the lane camera device re-identifies the vehicle's license plate, compares the identified license plate with the current license plate corresponding to the vehicle ID, and determines whether to update the vehicle information and vehicle ID.

[0019] If the difference between the license plate of the vehicle identified by the lane camera and the license plate of any vehicle in the current database is less than a preset number of characters, and the vehicle position of the vehicle identified by the lane camera is the same as the vehicle position in the vehicle information, then a correspondence between the vehicle information and the vehicle identified by the lane camera is established, and the vehicle information and vehicle ID of the vehicle remain unchanged.

[0020] If the difference between the license plate of the vehicle identified by the lane camera and the license plate of any vehicle in the current database is less than a preset number of characters, but the vehicle position of the vehicle identified by the lane camera is different from the vehicle position in the vehicle information, then the vehicle position in the vehicle information is updated, and the vehicle ID remains unchanged.

[0021] If the difference between the license plate of the vehicle identified by the lane camera and the license plate of any vehicle in the current database is not less than a preset number of characters, the lane camera will create an ID for the vehicle information corresponding to that vehicle information.

[0022] In some embodiments of the present invention, in the step of conducting a transaction based on the final vehicle information and vehicle ID, and deleting the vehicle ID after the transaction, if the vehicle is an ETC vehicle equipped with an on-board terminal, the toll system conducts an ETC transaction for the vehicle, deletes the vehicle ID after the transaction, and issues a gate-raising command. After the gate-raising command is issued, it continuously monitors whether the gate is raised. If the gate is not raised, it sends a prompt command to the toll collector's control terminal.

[0023] In some embodiments of the present invention, in the steps of conducting a transaction based on the final vehicle information and vehicle ID, and deleting the vehicle ID after the transaction, if the vehicle is not an ETC vehicle equipped with an on-board terminal, the toll collection system adopts a contactless automatic deduction toll collection mode. The contactless automatic deduction toll collection mode includes license plate payment prepayment, in which the toll collection system backend automatically generates a prepayment or license plate payment record, and issues a gate-raising command based on the current vehicle information. After the gate-raising command is issued, it continuously monitors whether the gate is raised. If the gate is not raised, a prompt command is sent to the toll collector control terminal.

[0024] A second aspect of the present invention also provides a vehicle contactless toll collection system based on multi-source information fusion and recognition. The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method described above.

[0025] A third aspect of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned vehicle contactless toll collection method based on multi-source information fusion and recognition.

[0026] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the text, or may be learned by practice of the invention. The objects and other advantages of the invention will become apparent from the description and the accompanying drawings.

[0027] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0028] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0029] Figure 1 This is a schematic diagram of one implementation method of the vehicle contactless toll collection method based on multi-source information fusion and identification in this scheme;

[0030] Figure 2 This is a schematic diagram of the equipment layout for this plan;

[0031] Figure 3 This is a schematic diagram of the ETC vehicle toll collection process in this scheme;

[0032] Figure 4 This is a schematic diagram of the toll collection process for non-ETC vehicles in this scheme. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0034] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0035] ETC (Electronic Toll Collection) has been widely adopted on highways nationwide. However, in practical applications, inconsistencies in the sensitivity of on-board units (OBUs), differences in the regional control of roadside units (RSUs), and wireless signal interference issues such as interference from adjacent lanes, truck reflections, and information board reflections can lead to ETC transaction failures.

[0036] In addition, even if an ETC transaction is successful, when a vehicle passes through an ETC lane, issues such as following other vehicles or vehicle identification errors may occur, resulting in the barrier not lifting in time when the vehicle encounters it in the toll lane. These low-probability events all have a certain impact on the traffic efficiency of both ETC and manual toll lanes.

[0037] In manual toll lanes, some toll stations support pre-payment transactions and "license plate payment" transactions that automatically deduct fees through license plate recognition. The success rate of license plate recognition is greatly affected by light reflection and occlusion, and there are also certain recognition errors.

[0038] like Figure 1 and 2 As shown, this invention proposes a vehicle contactless toll collection method based on multi-source information fusion and recognition. The method is based on a front-mounted recognition device and a lane-mounted camera device. The front-mounted recognition device is positioned in front of the lane-mounted camera device and includes a front-mounted camera and a location-tracking camera. The steps of the method include:

[0039] Step S100: When the vehicle is driving into the toll area, the vehicle information is identified by the front camera and a vehicle ID corresponding to the vehicle information is constructed. The vehicle information includes the vehicle license plate, vehicle color and vehicle location.

[0040] In the specific implementation process, if the vehicle is an ETC vehicle equipped with an on-board unit, the vehicle information also includes the OBU serial number of the on-board ETC.

[0041] Step S200: After the vehicle passes the shooting range of the front camera, the vehicle license plate is identified again by the position tracking camera. The identified vehicle license plate is compared with the current vehicle license plate corresponding to the vehicle ID to determine whether to update the vehicle information and vehicle ID.

[0042] Specifically, the front-end recognition equipment also includes a long-range camera. When a vehicle approaches the toll station, the long-range camera identifies the vehicle's license plate, color, and location information, and performs continuous location recognition. If the location recognition result matches the result identified by the front-end camera, a unique vehicle ID is assigned.

[0043] In some embodiments of the present invention, the vehicle location information in the vehicle information is the lane where the vehicle is located.

[0044] Step S300: When the vehicle arrives at the toll island, the vehicle license plate is re-identified by the lane camera device. The identified vehicle license plate is compared with the current vehicle license plate corresponding to the vehicle ID to determine whether to update the vehicle information and vehicle ID.

[0045] After a vehicle enters the toll island, the RSU (Roadside Unit) first identifies the OBU (On-Board Unit) information installed in the ETC (Electronic Toll Collection) system. The camera then identifies the vehicle's license plate, color, and location information, and the results are returned to the toll system. Once inside the toll island, the vehicle's trajectory is tracked and identified by position-tracking cameras until it enters the lane isolated by the toll island. For overlapping areas of different camera fields of view, vehicle position identification is achieved through position matching. If the center point error of the vehicle's position detected by different cameras at the same time is less than 1 meter, it can be determined to be the same vehicle. Each camera has a fixed position-tracking field of view, and different positions in the images it captures can be pre-calibrated using latitude and longitude or relative position calibration.

[0046] Step S400: Transaction is performed based on the final vehicle information and vehicle ID, and the vehicle ID is deleted after the transaction.

[0047] The device layout of the present invention is as follows Figure 2As shown, a front-facing camera and a location tracking camera are added to the existing lane equipment. The front-facing camera mainly consists of a camera and a vehicle registration unit (RSU), mounted on a front gantry. Depending on the on-site installation conditions at the toll station, the front-facing camera can also be mounted on a roadside L-pole or pillar. The front-facing recognition equipment includes an RSU, used to identify information from the on-board ETC terminal (OBU); and a camera, used to identify license plates and vehicle location information. To more accurately identify vehicle positions, location detection equipment such as lidar or millimeter-wave radar can be added. Alternatively, photoelectric vehicle detectors or inductive loop detectors can be laid in the camera's capture area. When a vehicle passes through the detection area, the corresponding camera is triggered to capture and identify the vehicle, increasing the accuracy of vehicle and license plate recognition. Whether or not location detection equipment is added does not affect the implementation effect of this invention. A tri-lens camera can also be used for the front-facing camera. Figure 2 In order to ensure the accuracy and consistency of location recognition, the front-facing camera and the location tracking camera are deployed in a two-way manner, with the recognition area of ​​the rear location tracking camera partially overlapping with the recognition area of ​​the lane camera.

[0048] After a vehicle enters the toll station area, the front-end recognition device first identifies the vehicle and its location information. Then, as the vehicle passes through the sensing area of ​​the front gantry / pillar, the vehicle and its location information are identified again. When the vehicle enters the toll station's isolation lane, the lane camera device identifies the license plate and its location information. Each camera, upon identifying the license plate information, simultaneously provides a confidence score for the identification result, allowing for secondary evaluation if the identification results from different cameras differ.

[0049] Location tracking cameras can be installed beneath the toll plaza canopy. The installation location and number should be selected based on the camera parameters to ensure coverage of the entire toll plaza area, including vehicle license plates and location information. Once a vehicle enters the toll plaza, lane cameras identify the vehicle's license plate and location information. The lane cameras detect the vehicle's longitudinal distance within the lane and verify that the license plate matches the information identified by the location tracking cameras.

[0050] This solution uses multiple cameras to continuously identify the vehicle's driving path and location, and integrates the identification results from other vehicle detection equipment such as RSUs to perform real-time trajectory tracking of vehicles passing through the toll station and identify their transaction status. This allows the system to know the transaction status when the vehicle arrives at the toll lane exit, enabling automated or semi-automated operation and improving the efficiency of toll station traffic.

[0051] By deploying multi-source information fusion identification equipment in the toll station area, this solution performs multi-source fusion identification of information such as the location, license plate, OBU, and vehicle color of vehicles passing through the toll station area, and links with the toll system in real time to accurately identify vehicles that have completed transactions. This improves the passage ratio of ETC vehicles and the accuracy of license plate payment and prepayment identification in manual toll lanes, ultimately improving the accuracy and reliability of toll collection at toll stations and providing a better travel service experience.

[0052] In some embodiments of the present invention, the front-facing identification device further includes a roadside antenna. If the vehicle is an ETC vehicle equipped with an on-board terminal, in the step of identifying the vehicle information through the front-facing camera, the OBU record is obtained from the vehicle's on-board terminal through the roadside antenna, and the vehicle license plate in the OBU record is compared with the vehicle license plate identified by the front-facing camera.

[0053] In some embodiments of the present invention, in the step of comparing the vehicle license plate in the OBU record with the vehicle license plate identified by the front camera, if the difference between the vehicle license plate in the OBU record and the vehicle license plate identified by the front camera is less than a preset number of characters, then the vehicle license plate in the OBU record is used as the vehicle license plate in the vehicle information.

[0054] If the license plate recognized by the camera differs from the license plate read by the RSU from the OBU by no more than two characters, the vehicle color recognition result matches the OBU record, and there is only one vehicle in the camera's field of view, then the license plate of that vehicle is based on the one in the OBU; otherwise, it is recorded as two different vehicles. The camera continuously monitors vehicle position information and updates its position at set intervals until the vehicle leaves the detection range of the front gantry camera.

[0055] In some embodiments of the present invention, in the step of re-identifying the vehicle license plate through the location tracking camera, comparing the identified vehicle license plate with the current vehicle license plate corresponding to the vehicle ID, and determining whether to update the vehicle information and vehicle ID, the vehicle license plate and vehicle position are re-identified through the location tracking camera, the vehicle license plate identified by the location tracking camera is compared with the vehicle license plate in the current vehicle information, and the vehicle position identified by the location tracking camera is compared with the vehicle position in the current vehicle information to determine whether to update the vehicle information and vehicle ID.

[0056] In some embodiments of the present invention, the location tracking camera and the front-facing camera are in one-to-one correspondence. The location tracking camera tracks the corresponding vehicle that has moved out of the field of view of the front-facing camera. In the step of comparing the license plate of the vehicle identified by the location tracking camera with the license plate of the vehicle in the current vehicle information, and comparing the vehicle position identified by the location tracking camera with the vehicle position in the current vehicle information, to determine whether to update the vehicle information and vehicle ID:

[0057] If the vehicle location identified by the location tracking camera is the same as the vehicle location in the current vehicle information, and the difference between the vehicle license plate in the current vehicle information and the vehicle license plate identified by the location tracking camera is less than a preset number of characters, then the vehicle information and vehicle ID remain unchanged.

[0058] If the vehicle position identified by the location tracking camera is different from the vehicle position in the current vehicle information, then update the vehicle position in the vehicle information.

[0059] If the vehicle location identified by the location tracking camera is different from the vehicle location in the current vehicle information, and the difference between the vehicle license plate in the current vehicle information and the vehicle license plate identified by the location tracking camera is not less than a preset number of characters, then the vehicle location and vehicle license plate in the vehicle information are updated; and a new vehicle ID is generated to update the original vehicle ID.

[0060] Specifically, the location tracking camera can perform secondary recognition of vehicle license plates. If the location and license plate are consistent with those recognized by the front-facing camera, the camera will continuously detect and update the vehicle's location until it enters the toll lane with the isolation island. Otherwise, if the license plate recognized by the location tracking camera and the front-facing camera are different, but the location, color, and number of characters differ by no more than two, the license plate recognized by the front-facing camera will be used, and the vehicle ID will remain unchanged. If the license plate and vehicle color recognized by the location tracking camera and the front-facing camera are the same, but the location is different, the location recognized by the location tracking camera will be used, and the vehicle ID will remain unchanged. If the location is different and the number of characters differs by more than two, a new vehicle ID number will be assigned to the vehicle, corresponding to the new license plate number and location information.

[0061] In some embodiments of the present invention, when a vehicle approaches a toll island, the lane camera device re-identifies the vehicle's license plate, compares the identified license plate with the current license plate corresponding to the vehicle ID, and determines whether to update the vehicle information and vehicle ID.

[0062] If the difference between the license plate of the vehicle identified by the lane camera and the license plate of any vehicle in the current database is less than a preset number of characters, and the vehicle position of the vehicle identified by the lane camera is the same as the vehicle position in the vehicle information, then a correspondence between the vehicle information and the vehicle identified by the lane camera is established, and the vehicle information and vehicle ID of the vehicle remain unchanged.

[0063] If the difference between the license plate of the vehicle identified by the lane camera and the license plate of any vehicle in the current database is less than a preset number of characters, but the vehicle position of the vehicle identified by the lane camera is different from the vehicle position in the vehicle information, then the vehicle position in the vehicle information is updated, and the vehicle ID remains unchanged.

[0064] If the difference between the license plate of the vehicle identified by the lane camera and the license plate of any vehicle in the current database is not less than a preset number of characters, the lane camera will create an ID for the vehicle information corresponding to that vehicle information.

[0065] The lane camera identifies the vehicle's location and license plate information within the ETC toll island. If the identified license plate matches the existing vehicle's license plate and location, the vehicle's ID remains unchanged. If the lane camera fails to identify the license plate information or the identified license plate information differs from the previous result by no more than two characters, and the vehicle location matches, the license plate corresponding to the vehicle ID is used. If the location is different but the license plate is the same, the vehicle's ID remains unchanged and the location information is updated. If the location is different and the license plate differs by more than two characters, a new ID number, license plate number, and location information are set for the vehicle.

[0066] like Figure 3 As shown, in some embodiments of the present invention, in the step of conducting a transaction based on the final vehicle information and vehicle ID, and deleting the vehicle ID after the transaction, if the vehicle is an ETC vehicle equipped with an on-board terminal, the toll system conducts an ETC transaction for the vehicle, deletes the vehicle ID after the transaction, and issues a gate-raising command. After the gate-raising command is issued, it continuously monitors whether the gate is raised. If the gate is not raised, it sends a prompt command to the toll collector's control terminal.

[0067] In some embodiments of the present invention, if the vehicle is an ETC vehicle equipped with an on-board terminal and the ETC transaction fails, but there is OBU information, the barrier will automatically be raised to allow passage, and the display device will show the amount due.

[0068] When a vehicle's ETC transaction is successful, if the vehicle passes through the lane barrier normally, the vehicle ID is cancelled. If the barrier does not lift when the vehicle reaches the lane barrier, but the toll system has confirmed the successful ETC transaction, the system can automatically raise the barrier to allow passage, or the toll collector can remotely raise the barrier with a single click. This one-click operation saves the toll collector time in checking and confirming the vehicle's transaction status on-site. If the ETC transaction fails, but the toll system has identified the vehicle's OBU information, the toll collector can use a handheld device to complete the ETC transaction, or the system can record the transaction amount and raise the barrier to allow passage. The toll display shows the amount and prompts for deduction from the backend. Users can check their passage records and toll amounts on the toll system application. If an ETC vehicle enters a manual toll lane, since the toll system has automatically recorded the amount due, it can directly prompt the toll collector with the amount. The toll collector only needs to confirm the payment before releasing the vehicle.

[0069] For ETC vehicles, this invention automatically identifies and issues a barrier-raising command when an ETC transaction is successful but the barrier fails to raise in time at the lane exit due to equipment malfunction or other reasons. Conversely, when an ETC transaction fails but the system has already identified the license plate and OBU information, the toll system performs background deduction and promptly raises the barrier to allow passage. This achieves functions such as automatic multi-source information identification, supplementary transactions, and automatic fault handling. When an ETC vehicle enters a manual toll lane, the toll system has already automatically recorded the amount due and can directly display the amount to the toll collector, who only needs to confirm. This invention effectively improves the passage efficiency of ETC vehicles at toll stations and reduces the manual processing costs and time for toll collectors.

[0070] When a vehicle enters the manual toll lane, if it is an ETC vehicle, the toll collector will be prompted with the exact amount to be charged, and the toll collector only needs to confirm it; if it is an MTC vehicle and the driver chooses to stop and pay, the amount to be charged will be displayed, and the vehicle will leave after payment.

[0071] like Figure 4 As shown, in some embodiments of the present invention, in the step of conducting a transaction based on the final vehicle information and vehicle ID, and deleting the vehicle ID after the transaction, if the vehicle is not an ETC vehicle equipped with an on-board terminal, the toll collection system adopts a contactless automatic deduction toll collection mode. The contactless automatic deduction toll collection mode includes license plate payment prepayment, which is automatically generated by the toll collection system backend. The system then issues a gate-raising command based on the current vehicle information and continuously monitors whether the gate is raised after the gate-raising command is issued. If the gate is not raised, a prompt command is sent to the toll collector control terminal.

[0072] If a non-ETC vehicle enters an ETC lane, the toll collector will handle the situation on-site. If the vehicle pays the toll normally at the toll window, the barrier will be raised to allow passage, and the vehicle ID will be cancelled. If the vehicle needs to check its route information or the amount due, the toll system will automatically display the recognized license plate to the toll collector, who will then release the vehicle after completing the payment and cancelling the vehicle ID.

[0073] If the toll station supports contactless payment modes such as license plate payment and prepayment, when a vehicle enters the toll lane, the toll system will automatically raise the barrier to allow passage if the vehicle's license plate information and color information are correct before the vehicle enters. The system will automatically generate a prepayment or license plate payment record.

[0074] If the toll station supports contactless payment methods such as license plate payment and prepayment, the system will automatically raise the barrier at the lane exit if it has already recognized the vehicle's license plate and color information when the vehicle enters the toll lane. The toll system backend will automatically generate a prepayment or license plate payment record. Employing multi-source information fusion recognition and continuous license plate recognition can effectively improve the license plate recognition rate and solve the problem of license plate recognition errors or failures due to factors such as lighting and obstruction in toll lane cameras, thereby improving toll collection efficiency and traffic flow efficiency.

[0075] This invention applies to scenarios involving ETC vehicles and MTC vehicles at exit toll stations, as well as scenarios involving ETC vehicles at entrance toll stations.

[0076] This invention improves the accuracy of vehicle license plate recognition by matching license plates from multiple sets of cameras. Each vehicle has three states from entry to exit: "entering the toll plaza," "entering the nth toll lane," and "exiting." Therefore, it also has automatic inspection image acquisition functions, as well as road traffic volume and vehicle speed statistics functions. At the same time, continuous location recognition also provides a data source for the digital twin of the toll station.

[0077] The front-facing recognition RSU and cameras, location tracking cameras, lane cameras, and toll lane control equipment such as toll displays, lane transaction RSUs, lane barriers, and self-service payment devices are all synchronized and communicate online with the toll management system. Once each vehicle enters the toll station, its license plate, on-board unit (OBU), vehicle color, and location information are identified and confirmed, the toll management system assigns a unique vehicle ID as a marker for multi-source information fusion identification. The location information corresponding to this ID is updated at a set time frequency, specifically, 100 milliseconds.

[0078] This solution deploys multi-source information fusion recognition equipment in toll station areas to perform multi-source fusion recognition of information such as vehicle location, license plate, OBU, and vehicle color of vehicles passing through the toll station area. It also links with the toll system in real time to accurately identify vehicles that have completed transactions, thereby increasing the passage rate of ETC vehicles and improving the accuracy of license plate payment and prepayment recognition in manual toll lanes. Ultimately, it improves the accuracy and reliability of toll collection at toll stations and provides a better travel service experience.

[0079] This solution integrates and identifies vehicles entering the toll station area using multi-source information, and then performs real-time unique identification to achieve automatic matching of vehicle, license plate, OBU, and user information. This enables vehicles to pass through toll stations without stopping in typical abnormal scenarios, and completes the automatic and seamless toll collection function when ETC transactions are abnormal.

[0080] This invention also provides a vehicle contactless toll collection system based on multi-source information fusion and recognition. The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method described above.

[0081] This invention also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the steps of the aforementioned vehicle contactless toll collection method based on multi-source information fusion and recognition. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0082] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0083] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0084] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0085] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for contactless vehicle toll collection based on multi-source information fusion and recognition, characterized in that, The method is based on a front-facing recognition device and a lane camera device. The front-facing recognition device is positioned in front of the lane camera device and includes a front-facing camera and a position tracking camera. The steps of the method include: When a vehicle travels into a toll area, the front-facing camera identifies the vehicle's information and constructs a corresponding vehicle ID, which includes the vehicle's license plate, color, and location. The front-end recognition device also includes a roadside antenna. If the vehicle is an ETC vehicle equipped with an on-board terminal, in the step of recognizing the vehicle information through the front-end camera, the OBU record is obtained from the vehicle's on-board terminal through the roadside antenna. The vehicle license plate in the OBU record is compared with the vehicle license plate recognized by the front-end camera. In the step of comparing the vehicle license plate in the OBU record with the vehicle license plate recognized by the front-end camera, if the difference between the vehicle license plate in the OBU record and the vehicle license plate recognized by the front-end camera is less than a preset number of characters, then the vehicle license plate in the OBU record is used as the vehicle license plate in the vehicle information. After the vehicle passes the shooting range of the front camera, the vehicle license plate is identified again by the location tracking camera. The identified vehicle license plate is compared with the current vehicle license plate corresponding to the vehicle ID to determine whether to update the vehicle information and vehicle ID. In the step of re-identifying the vehicle license plate using the location tracking camera, comparing the identified license plate with the current license plate corresponding to the vehicle ID, and determining whether to update the vehicle information and vehicle ID, the location tracking camera re-identifies the vehicle license plate and vehicle location, compares the license plate identified by the location tracking camera with the license plate in the current vehicle information, compares the vehicle location identified by the location tracking camera with the vehicle location in the current vehicle information, and determines whether to update the vehicle information and vehicle ID. Similarly, the location tracking camera compares the license plate identified by the location tracking camera with the license plate in the current vehicle information, and compares the vehicle location identified by the location tracking camera with the vehicle location in the current vehicle information, and determines whether to update the vehicle information and vehicle ID. In step D: if the vehicle location identified by the location tracking camera is the same as the vehicle location in the current vehicle information, and the difference between the vehicle license plate in the current vehicle information and the vehicle license plate identified by the location tracking camera is less than a preset number of characters, then the vehicle information and vehicle ID remain unchanged; if the vehicle location identified by the location tracking camera is different from the vehicle location in the current vehicle information, then the vehicle location in the vehicle information is updated; if the vehicle location identified by the location tracking camera is different from the vehicle location in the current vehicle information, and the difference between the vehicle license plate in the current vehicle information and the vehicle license plate identified by the location tracking camera is not less than a preset number of characters, then the vehicle location and vehicle license plate in the vehicle information are updated; and a new vehicle ID is generated, updating the original vehicle ID. When a vehicle reaches the toll island, the vehicle's license plate is re-identified by the lane camera equipment. The identified license plate is compared with the current license plate corresponding to the vehicle ID to determine whether to update the vehicle information and vehicle ID. Transactions are conducted based on the final vehicle information and vehicle ID, and the vehicle ID is deleted after the transaction.

2. The vehicle contactless toll collection method based on multi-source information fusion and recognition according to claim 1, characterized in that, When a vehicle approaches the toll island, the lane camera system re-identifies the vehicle's license plate, compares the identified license plate with the current license plate corresponding to the vehicle ID, and determines whether to update the vehicle information and vehicle ID. If the difference between the license plate of the vehicle identified by the lane camera and the license plate of any vehicle in the current database is less than a preset number of characters, and the vehicle position of the vehicle identified by the lane camera is the same as the vehicle position in the vehicle information, then a correspondence between the vehicle information and the vehicle identified by the lane camera is established, and the vehicle information and vehicle ID of the vehicle remain unchanged. If the difference between the license plate of the vehicle identified by the lane camera and the license plate of any vehicle in the current database is less than a preset number of characters, but the vehicle position of the vehicle identified by the lane camera is different from the vehicle position in the vehicle information, then the vehicle position in the vehicle information is updated, and the vehicle ID remains unchanged. If the difference between the license plate of the vehicle identified by the lane camera and the license plate of any vehicle in the current database is not less than a preset number of characters, the lane camera will create an ID for the vehicle information corresponding to that vehicle information.

3. The vehicle contactless toll collection method based on multi-source information fusion and recognition according to claim 1, characterized in that, In the process of conducting a transaction based on the final vehicle information and vehicle ID, and deleting the vehicle ID after the transaction, if the vehicle is an ETC vehicle equipped with an on-board terminal, the ETC toll collection system conducts an ETC transaction for the vehicle, deletes the vehicle ID after the transaction, and issues a gate-raising command. After the gate-raising command is issued, it continuously monitors whether the gate is raised. If the gate is not raised, it sends a prompt command to the toll collector's control terminal.

4. The vehicle contactless toll collection method based on multi-source information fusion and recognition according to claim 1, characterized in that, In the steps of conducting transactions based on the final vehicle information and vehicle ID, and deleting the vehicle ID after the transaction, if the vehicle is not an ETC vehicle with an on-board terminal installed, the toll collection system adopts a contactless automatic deduction mode. The contactless automatic deduction mode includes license plate payment prepayment, in which the toll collection system backend automatically generates prepayment or license plate payment records, and issues a gate-raising command based on the current vehicle information. After the gate-raising command is issued, it continuously monitors whether the gate is raised. If the gate is not raised, it sends a prompt command to the toll collector's control terminal.

5. A vehicle contactless toll collection system based on multi-source information fusion and recognition, characterized in that, The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method as described in any one of claims 1 to 4.

Images