An ETC transaction risk early warning method and system based on visual identification and closed-loop updating

Abstract

The application provides an ETC transaction risk early warning method and system based on visual identification and closed-loop updating, comprising: collecting lane scene images through an image acquisition module of a vehicle-mounted ETC device, and identifying scene features in real time by using an image recognition model; performing similarity matching on the identified scene features and a pre-stored risk scene feature library, and issuing a graded early warning prompt if the similarity exceeds a similarity threshold; when an abnormality occurs after the early warning, such as transaction failure or contradiction between the visual judgment result and the communication judgment result, automatically recording incremental samples, and based on the mechanism of local clustering analysis and cloud federated learning cooperation, continuously updating and optimizing the risk feature library and the early warning threshold in a closed loop. The application can actively warn high-risk passing scenes, significantly improve the ETC transaction success rate, accurately define the responsibility through visual cross-validation, continuously adapt to complex traffic environments by using self-learning ability, and ultimately realize the cooperative optimization of passing efficiency and user experience.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation electronic toll collection technology, and in particular to an ETC transaction risk early warning method and system based on visual recognition and closed-loop updating. Background Technology

[0002] Against the backdrop of the rapid development of intelligent transportation systems, Electronic Toll Collection (ETC) systems have become a key technology for improving highway traffic efficiency and reducing congestion. Traditional ETC systems primarily rely on Dedicated Short Range Communication (DSRC) technology to complete data exchange between vehicles and roadside equipment, enabling automatic toll deduction and traffic control. Existing solutions typically record basic data such as transaction time, transaction result, and signal strength in the on-board unit and store logs using the ESAM security module for post-event troubleshooting and transaction reconciliation. However, this log mechanism based on a single communication decision is gradually revealing significant limitations in increasingly complex real-world traffic scenarios.

[0003] Existing solutions lack the ability to visually verify the actual execution effect of transactions, failing to identify contradictory scenarios such as "communication succeeds but the barrier does not lift" or "communication fails but payment is successfully deducted," leading to difficulties in determining responsibility. The system only passively records data after a transaction fails, unable to provide early warnings for high-frequency failure scenarios (such as rainy or foggy weather, narrow lanes, or large vehicles obstructing traffic), nor can it provide users with real-time operational guidance. Furthermore, due to the lack of a self-learning mechanism, the system struggles to extract effective risk characteristics from historical failures, resulting in recurring similar problems and a persistently high user complaint rate. These deficiencies severely limit the reliability of the ETC system and the user experience in dynamic traffic environments. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide an ETC transaction risk warning method and system based on visual recognition and closed-loop update, in order to eliminate or improve one or more defects existing in the prior art, and solve the problems of lack of transaction status verification, passive response failure, insufficient scenario adaptability, and lack of human-computer interaction in traditional ETC systems in complex traffic environments.

[0005] On one hand, this invention provides an ETC transaction risk warning method based on visual recognition and closed-loop update. The method is applied to an on-board ETC device, which pre-stores a risk scenario feature library. The method includes: The image acquisition module of the vehicle-mounted ETC device acquires images of the lane scene in front of the vehicle, and performs visual recognition through a pre-trained image recognition model to extract scene features from the images; wherein, the scene features include the state of the guardrail, the color of the indicator lights, the weather environment, the lane attributes, and the vehicle status; The similarity matching calculation is performed between the scenario features and the risk features in the risk scenario feature library; the risk scenario feature library is constructed based on historical scenario features, which correspond to historical ETC transaction anomaly records, including transaction failure records and transaction contradiction records; wherein, the transaction contradiction record refers to the record where the transaction result judged based on visual recognition is inconsistent with the transaction result judged based on dedicated short-range communication; If the similarity matching result exceeds the current preset similarity threshold, a corresponding warning instruction is generated, and a graded warning prompt is issued to the driver through the human-machine interaction module of the vehicle-mounted ETC device. If an ETC transaction anomaly still occurs after an alert, the scenario characteristics and transaction data of the current transaction are recorded as incremental samples, and the risk scenario feature library and the similarity threshold are updated in a closed loop based on the incremental samples.

[0006] In some embodiments of the present invention, the image recognition model is a convolutional neural network, which adopts a depthwise separable convolutional structure, including an input layer, multiple feature extraction layers, multiple classification layers and a feature output layer, wherein the classification layer is used to output the railing status and the indicator light color.

[0007] In some embodiments of the present invention, the method further includes, before execution: The transaction failure record file of the vehicle-mounted ETC device is expanded by adding a scene feature field. The scene feature field includes a barrier status field, an indicator light color field, a confidence level field, and a comprehensive feature field. The comprehensive feature field includes quantitative feature parameters of weather environment, lane attributes, and vehicle status. The transaction success rate file of the vehicle-mounted ETC device is expanded by adding a vision-communication consistency rate field and an early warning effectiveness field.

[0008] In some embodiments of the present invention, the closed-loop update of the risk scenario feature library based on incremental samples includes a local update mode: Based on a first preset frequency, and when the vehicle to which the on-board ETC device belongs is stationary, the DBSCAN density clustering algorithm is invoked to perform cluster analysis on the incremental samples, resulting in multiple clusters. The common features of all samples within each cluster are extracted to generate new risk features, and the new risk features are updated to the local risk scenario feature library.

[0009] In some embodiments of the present invention, the closed-loop update of the risk scenario feature library based on incremental samples further includes a cloud-based collaborative update mode: Based on the second preset frequency, each vehicle-mounted ETC device encrypts and uploads the local incremental samples to the cloud server; The system receives incremental samples uploaded by various vehicle-mounted ETC devices via the cloud server and generates optimized risk scenario feature library parameters through global model training using a federated learning algorithm. The system then updates the local risk scenario feature library based on these parameters. The first preset frequency is higher than the second preset frequency.

[0010] In some embodiments of the present invention, the step of issuing graded warning prompts to the driver through the human-machine interaction module of the on-board ETC device includes: If the similarity matching result exceeds the first-level threshold, a first-level warning is triggered, and an audio-visual prompt is given through the local buzzer and display screen of the vehicle-mounted ETC device. If the similarity matching result exceeds the second-level threshold but does not exceed the first-level threshold, a second-level warning is triggered, and the prompt information is pushed to the vehicle central control system via Bluetooth for graphical display; wherein, the first-level threshold is greater than the second-level threshold.

[0011] In some embodiments of the present invention, the method further includes a liability prediction step: After an ETC transaction anomaly occurs, a contradiction analysis is performed between the transaction result determined by visual recognition based on the image recognition model and the transaction result determined by the dedicated short-range communication. If visual recognition determines that the transaction has failed while the dedicated short-range communication determines that the transaction has succeeded, a pre-judgment identifier is generated indicating that the responsible party is the toll station's back-end system. If visual recognition determines the transaction is successful while the dedicated short-range communication determines the transaction is unsuccessful, and the vehicle's lane deviation is found to be greater than the preset safety tolerance, then a pre-judgment label indicating that the responsible party is the driver due to improper operation is generated.

[0012] In some embodiments of the present invention, after performing the method, the method further includes: Continuously monitor and provide feedback on transaction results following early warning alerts; The effectiveness rate of early warnings triggered by specific risk scenario characteristics is calculated, whereby the early warning effectiveness rate is the proportion of successful ETC transactions after an early warning to the total number of early warnings. Based on the dynamic changes in the warning effectiveness rate, the similarity threshold for triggering warnings of specific risk scenarios is adaptively adjusted: when the warning effectiveness rate continues to increase, the corresponding similarity threshold is reduced; when the warning effectiveness rate continues to decrease, the corresponding similarity threshold is increased.

[0013] On the other hand, the present invention provides an ETC transaction risk warning system based on visual recognition and closed-loop update, the system being integrated into an on-board ETC device, the system comprising: The image acquisition and recognition module is used to acquire images of the lane scene in front of the vehicle, and to perform visual recognition through a pre-trained image recognition model to extract scene features from the images; The risk matching module, connected to the image acquisition and recognition module, is used to perform similarity matching calculations between the extracted scene features and risk features in the pre-stored risk scene feature library; The early warning module, connected to the risk matching module, is used to generate a corresponding early warning instruction and issue a graded early warning to the driver when the similarity matching result exceeds the current preset similarity threshold. The closed-loop update module is connected to the image acquisition and recognition module and the risk matching module respectively. When an ETC transaction anomaly still occurs after an early warning, it records the scenario characteristics and transaction data of the current transaction as incremental samples, and updates the risk scenario feature library and the similarity threshold based on the incremental samples.

[0014] On the other hand, the present invention provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of any of the methods mentioned above.

[0015] This invention provides an ETC transaction risk warning method and system based on visual recognition and closed-loop updating. The method includes: acquiring lane scene images through the image acquisition module of the onboard ETC device and using an image recognition model to identify scene features in real time; matching the identified scene features with a pre-stored risk scene feature library for similarity; issuing a tiered warning if the similarity exceeds a threshold; automatically recording incremental samples when transaction failures or discrepancies between visual and communication judgments occur after a warning, and continuously optimizing and updating the risk feature library and warning thresholds in a closed loop based on a mechanism of local clustering analysis and cloud-based federated learning. This invention proactively warns of high-risk traffic scenarios, significantly improving the success rate of ETC transactions, accurately defining responsibility through visual cross-validation, and continuously adapting to complex traffic environments using self-learning capabilities, ultimately achieving synergistic optimization of traffic efficiency and user experience.

[0016] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0017] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings: Figure 1 This is a schematic diagram illustrating the steps of an ETC transaction risk warning method based on visual recognition and closed-loop update in one embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of the structure of an ETC transaction risk warning system based on visual recognition and closed-loop update in one embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0021] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0022] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0023] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0024] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0025] It should be emphasized here that the step markers mentioned below are not a limitation on the order of the steps, but should be understood as meaning that the steps can be executed in the order mentioned in the embodiments, or in a different order than in the embodiments, or several steps can be executed simultaneously.

[0026] To address the problems of traditional ETC systems in complex traffic environments, such as lack of transaction status verification, passive response failures, insufficient scenario adaptability, and lack of human-computer interaction, this invention provides an ETC transaction risk early warning method based on visual recognition and closed-loop updates, applicable to on-board ETC devices, such as... Figure 1 As shown, the method includes the following steps S101~S104: Step S101: The image acquisition module of the vehicle-mounted ETC device acquires an image of the lane scene in front of the vehicle, and performs visual recognition through a pre-trained image recognition model to extract the scene features of the image; wherein, the scene features include the guardrail status, indicator light color, weather environment, lane attributes and vehicle status; Step S102: Perform similarity matching calculation between the scene features and the risk features in the risk scene feature library; the risk scene feature library is constructed based on historical scene features, which correspond to historical ETC transaction anomaly records, including transaction failure records and transaction contradiction records; wherein, the transaction contradiction record refers to the record where the transaction result judged based on visual recognition is inconsistent with the transaction result judged based on dedicated short-range communication; Step S103: If the similarity matching result exceeds the current preset similarity threshold, a corresponding warning instruction is generated, and a graded warning prompt is issued to the driver through the human-machine interaction module of the vehicle-mounted ETC device. Step S104: When an ETC transaction anomaly still occurs after the warning, record the scenario characteristics and transaction data of the current transaction as an incremental sample, and perform a closed-loop update of the risk scenario feature library and the similarity threshold based on the incremental sample.

[0027] Specifically, in step S101, the image of the lane scene in front of the vehicle is first acquired using the image acquisition module pre-set in the vehicle-mounted ETC device, and then the scene features of the image are extracted using a pre-trained image recognition model.

[0028] In some embodiments, the image acquisition module employs a miniature camera, utilizing an ultra-small, low-power CMOS sensor module directly embedded within the on-board unit (OBU) housing. For example, this miniature camera uses a 120°~140° wide-angle lens to ensure coverage of critical areas such as lanes, guardrails, and indicator lights in front of the vehicle. It supports VGA (640x480) or 720p (1280x720) resolution, a frame rate of 15-30fps, and low-light (≥5 lux) imaging capabilities to ensure usability at night, in tunnels, and in inclement weather.

[0029] In some embodiments, the image acquisition module is connected to the main control chip of the vehicle-mounted ETC device through interfaces such as MIPI CSI or DVP.

[0030] In some embodiments, the image recognition model employs a lightweight convolutional neural network (CNN), preferably, an architecture such as the MobileNet series (V2 / V3) or SqueezeNet. These architectures extensively use depthwise separable convolutions, which greatly reduce the number of parameters and computational cost while ensuring high accuracy.

[0031] In some embodiments, the image recognition model includes an input layer, multiple feature extraction layers, multiple classification layers, and a feature output layer, wherein the classification layer is used to output the state of the railing (raised, lowered, half-raised) and the color of the indicator light (red, green, yellow).

[0032] In some embodiments, the training steps of the image recognition model include: A large number of publicly available traffic scene images are used to construct a training set. Each image is labeled with information such as the status of the guardrail, the color of the indicator light, the position of the ETC lane line, the offset between the vehicle and the lane line, and the weather conditions.

[0033] Construct an initial model; the specific model structure can be found in the model used in the above examples.

[0034] The initial model is trained using a training set. A batch of images is input into the initial model to obtain prediction results. The total loss between the prediction results and the true labels is calculated using a loss function. The gradient of the total loss with respect to each model parameter is calculated to perform backpropagation and determine the direction and magnitude of parameter adjustments. An optimizer, such as Adam, is used to update the model parameters based on the gradient. Iterative training continues until the model loss converges to a low value, resulting in the final image recognition model.

[0035] In some embodiments, the trained model is optimized to convert it into a form suitable for deployment on embedded devices. For example, based on the model trained above with 32-bit floating-point precision, considering its high precision, it would suffer from large size, slow inference speed, and high power consumption on embedded devices. Therefore, the model is converted to INT8 (8-bit integer) precision, reducing the model size by approximately 75%, increasing the inference speed by 2 to 4 times, and significantly reducing power consumption. Simultaneously, based on pruning techniques, unimportant weights or channels in the network are removed, further compressing the model.

[0036] The trained image recognition model can extract scene features from an input image, including: barrier status, indicator light color, weather environment, lane attributes, and vehicle status. Weather environment can be indirectly determined through overall image blur and rain / snow noise. Weather environment includes weather type, light intensity, and visibility level; lane attributes include lane type, lane width, curve curvature, and lane line integrity; and vehicle status includes vehicle speed, distance to the vehicle in front, and lane departure.

[0037] In some embodiments, the log storage structure of the on-board ETC device is extended, including: The transaction success rate file (EF07 file) in the root directory of the Embedded Security Control Module (ESAM) has been expanded to include two fields: Visual-Communication Consistency Rate and Alert Effectiveness Rate. The Visual-Communication Consistency Rate refers to the proportion of transactions where the visual recognition result (based on an image recognition model) matches the transaction result where the result is based on dedicated short-range communication, out of the total number of transactions. The Alert Effectiveness Rate refers to the proportion of transactions that succeeded after receiving an alert, out of the total number of alerts triggered.

[0038] In some embodiments, the visual-communication consistency rate is calculated through the following steps: After each transaction is completed, visual judgment results and communication judgment results are obtained. The visual judgment results come from the judgment of the image acquisition module and the image recognition model. For example, if the barrier is detected to be raised and the indicator light is green, it is judged as successful; otherwise, it is considered a failure. The communication judgment results come from the communication feedback (success / failure) of the roadside unit (RSU).

[0039] Compare whether the visual judgment results and the communication judgment results are consistent.

[0040] The system pre-maintains two counters: a total transaction count counter and a consistency count counter. The total transaction count counter is incremented by 1 each time a transaction is completed. If the visual judgment result and the communication judgment result are consistent, the consistency count counter is incremented by 1.

[0041] After each transaction is completed, the total transaction count counter and the consistency count counter are immediately updated, and the latest visual-communication consistency rate is calculated and written to the EF07 file. For example, the visual-communication consistency rate is stored as a percentage value (such as an integer or floating-point number from 0 to 100) in a designated field of the EF07 file.

[0042] The transaction failure log file (EF02 file) in the root directory of the Embedded Security Control Module (ESAM) is expanded to include scene feature fields, such as barrier status, indicator light color, confidence level, and comprehensive feature fields. The comprehensive feature fields further include quantitative feature parameters such as weather environment, lane attributes, and vehicle status. The scene feature fields are shown in Table 1.

[0043] Table 1. Definition of Scene Feature Fields In some embodiments, the composite features are quantized into a 128-bit feature vector, encoded in binary bit field form, as shown in Table 2.

[0044] Table 2 Definition of Comprehensive Feature Fields In step S102, the similarity matching calculation is performed between the scene features and the risk features in the risk scene feature library.

[0045] In some embodiments, the risk scenario feature library is stored in a newly added EF07 file in the DF01 directory of the embedded security control module (ESAM) of the vehicle-mounted ETC device.

[0046] In some embodiments, cosine similarity or Euclidean distance is used to calculate the similarity between scene features and risk features in the risk scene feature library.

[0047] The risk scenario feature library is built based on historical scenario features, which correspond to historical ETC transaction anomaly records. These anomaly records include transaction failure records and transaction inconsistency records. Transaction inconsistency records refer to records where the transaction result determined based on visual recognition is inconsistent with the transaction result determined based on dedicated short-range communication.

[0048] In some embodiments, the steps for constructing the risk scenario feature library include: A large number of historical transaction anomaly records (including transaction failure records and transaction conflict records) and corresponding historical scene features are collected to construct a sample set. Clustering algorithms, such as DBSCAN and K-Means, are used to analyze these massive failure and conflict samples to extract the most common and shared high-risk scene patterns. For example, nationwide, the failure rate of vehicles exceeding 40km / h at toll stations on curves in rainy weather is extremely high. An initial risk scene feature library is constructed based on the extracted high-risk scene patterns.

[0049] Known and well-defined fault rules in the industry are directly encoded into feature vectors and injected into the initial risk scenario feature library. For example, an installation angle deviation of more than 30 degrees for on-board ETC devices can easily lead to transaction failure; another example is that DSRC communication requires a certain amount of time according to standards, so "excessive vehicle speed" is itself a risk factor, which can be injected into a simple risk pattern of "vehicle speed > 50km / h". By transforming prior knowledge into structured feature patterns and making them part of the initial risk scenario feature library, the system is ensured to adhere to the most basic business rules from the outset.

[0050] Through continuous application, by accumulating new transaction anomaly records and scenario features, new and more refined risk features are continuously extracted and added to the risk scenario feature library, making the risk scenario feature library more and more perfect through continuous learning.

[0051] In step S103, if the similarity matching result exceeds the current preset similarity threshold, a corresponding warning instruction is generated, and a graded warning prompt is issued to the driver through the human-machine interaction module of the vehicle-mounted ETC device.

[0052] In some embodiments, the warning instruction is a structured data packet containing all the necessary metadata for the warning. The instruction content typically includes: Warning levels: Level 1 (high risk), Level 2 (medium risk).

[0053] Risk type: Specific risk patterns identified from the risk scenario feature library, such as: Pattern_Speeding on a curve in the rain.

[0054] Recommended actions: Pre-set action suggestions based on the type of risk, such as: slowing down, maintaining a safe distance, adjusting the angle, etc.

[0055] Fill parameters: Dynamic values ​​generated based on real-time data, such as: recommended deceleration to 20km / h.

[0056] When the similarity match result exceeds the first-level threshold (a higher threshold means a very high match rate and extremely high risk), a first-level warning is triggered. The buzzer on the vehicle's ETC device is activated, emitting a rapid, continuous "beep beep beep" sound, such as three short beeps. This is a strong warning and cannot be ignored. Simultaneously, the display screen on the vehicle's ETC device will light up, displaying a warning in concise and bright text. For example, the message might read: "Toll station 500 meters ahead, likely to fail, please reduce speed to below 20 km / h!" When the similarity match result exceeds the second-level threshold but does not exceed the first-level threshold (moderate matching degree, with some risk), a second-level warning is triggered. The warning information is pushed to the vehicle's central control system via Bluetooth, and / or displayed graphically on the vehicle's screen as a pop-up window, banner notification, or navigation prompt. For example, the prompt might read: "The lane ahead is narrow; please keep your lane centered." or "Large vehicle detected ahead; it is recommended to maintain a distance of at least 5 meters." In some embodiments, after issuing an early warning, the system continuously monitors the transaction results (successful or failed transactions) following the early warning, and statistically analyzes the early warning effectiveness rate triggered by specific risk scenario characteristics. The early warning effectiveness rate is the proportion of successful ETC transactions after the early warning to the total number of early warnings, and is stored in the transaction success rate file (EF07).

[0057] Based on the dynamic changes in the effectiveness of early warnings, the similarity threshold for triggering early warnings of specific risk scenarios is adaptively adjusted: when the effectiveness of early warnings continues to increase, the corresponding similarity threshold is lowered; when the effectiveness of early warnings continues to decrease, the corresponding similarity threshold is raised.

[0058] In step S104, if an ETC transaction anomaly still occurs after the warning, the scenario characteristics and transaction data of the current transaction are recorded as incremental samples, and the risk scenario feature library and similarity threshold are updated in a closed loop based on the incremental samples.

[0059] In some embodiments, incremental samples are stored in an expanded transaction failure log file (EF02) and labeled "incremental sample".

[0060] In some embodiments, the closed-loop update method of the risk scenario feature library adopts an update mode of local autonomous iteration and cloud collaborative optimization.

[0061] After each ETC transaction, the transaction result is assessed. If a transaction fails or conflicts occur despite a warning, the corresponding scenario characteristics and transaction data are recorded as incremental samples. In this invention, only negative samples (transaction failures / conflicts) are collected because successful transactions are the norm, and the system needs to learn how to avoid anomalies. Simultaneously, samples with an image recognition model confidence level below a preset value (e.g., 60%) are filtered out. This is because low confidence level implies that the image recognition itself may be inaccurate (e.g., a brief blur in the camera image), and learning based on unreliable data will generate "noise" and mislead the model.

[0062] Local autonomous iteration: Based on a preset frequency, such as early morning each day when the vehicle is stationary, the local update process is automatically initiated. The DBSCAN density clustering algorithm is invoked to perform cluster analysis on recently stored incremental samples (e.g., 30 samples) in the EF02 file, resulting in multiple clusters. Common features of all samples within each cluster are extracted to generate new risk features, such as "three consecutive failures all in rainy weather + speed > 40 km / h". These new risk features are then updated to the local risk scenario feature library. Simultaneously, the similarity threshold for the corresponding risk features is dynamically optimized. For example, if a risk feature is highly effective, the similarity threshold is lowered from 85% to 80% to make it more sensitive.

[0063] Cloud-based collaborative optimization: Based on a second preset frequency, such as weekly (the frequency of cloud-based collaborative optimization is lower than the frequency of local autonomous iteration), the cloud server collects incremental samples from various in-vehicle ETC devices via Bluetooth in an encrypted manner. Federated learning technology is then used to train a more powerful global model in the cloud.

[0064] Based on federated learning, each in-vehicle ETC device only uploads the parameters needed for model updates, such as gradients. The cloud server aggregates these parameters to update the global model, while the original data remains on the vehicle, greatly protecting user privacy. Simultaneously, the cloud server possesses more data and can uncover deeper patterns. For example, it might discover that the "large vehicle occlusion" feature is a very strong influencing factor nationwide, thus increasing the weight of this feature in the global model. Furthermore, the cloud server can train different segmented models for different regions (such as the rainy south or the foggy north), achieving precise optimization.

[0065] The vehicle-mounted ETC device downloads the updated model from the cloud server, thus obtaining a new risk scenario feature library. The old and new risk scenario feature libraries are run in parallel (early warning judgments and prompts are still issued by the old risk scenario feature library) to verify whether the early warning suggestions of the new risk scenario feature library are more effective, such as an increase in early warning effectiveness of more than 10%.

[0066] After the new risk scenario feature library is put into application, new data will be generated, including new successful transaction records and new failed transaction records. New incremental samples will be generated based on the new failed transaction records, and the learning cycle will continue. This will enable the system to automatically adapt to the ever-changing new environment (new road sections, new weather) and truly achieve full-scenario self-adaptation.

[0067] In some embodiments, the ETC transaction risk warning method based on visual recognition and closed-loop update provided by the present invention further includes liability prediction, the steps of which include: After an ETC transaction anomaly occurs, a contradiction analysis is performed on the visual judgment results and the communication judgment results.

[0068] If the visual judgment result is a transaction failure while the communication judgment result is a transaction success (e.g., a successful transaction but the barrier is not raised, or a failed transaction but the payment is successfully deducted), then a pre-judgment identifier is generated indicating that the responsible party is the toll station's back-end system.

[0069] If the visual judgment result indicates a successful transaction while the communication judgment result indicates a failed transaction, and the vehicle's lane deviation is detected to be greater than the preset safety tolerance, then a pre-judgment label indicating that the responsible party is the driver due to improper operation will be generated.

[0070] The present invention will be further described below with reference to specific embodiments.

[0071] At a highway toll station, the ETC lane experienced multiple DSRC communication interruptions due to excessive vehicle speed (exceeding 40 km / h), resulting in transaction failures. The EF02 file records "Speed ​​= 55 km / h", "DSRC communication duration greater than 0.6s", "Visual recognition barrier status = 2 (fallen)", and "Indicator light color = 2 (red)".

[0072] After clustering 30 recent failed samples of the lane, the vehicle-mounted ETC device extracted new risk characteristics: "speed > 40km / h + lane is a curve", and updated the risk scenario feature library.

[0073] When a vehicle approaches the lane, the image recognition model extracts the scene features of the image captured by the image acquisition module at this time: "curve sign + current vehicle speed = 45km / h". The similarity with the corresponding risk features in the risk scene feature library reaches 90%, triggering a level one warning: the on-board ETC device beeps 3 times and the display shows "Curve toll station ahead, it is recommended to reduce speed to below 20km / h".

[0074] After the warning was implemented, the average speed of vehicles in the lane dropped to 25 km / h, the success rate of DSRC communication improved, and the proportion of transaction failures caused by excessive speed decreased.

[0075] At the entrance of a service area, the ETC lane is close to the truck weighing area, and transactions frequently fail due to "small cars following large vehicles too closely and being blocked". The EF02 file records "distance between adjacent vehicles < 3m (large vehicle)", "DSRC signal strength < -85dBm" and "visual recognition of the height of the vehicle in front > 2.5m (determined to be a large vehicle)".

[0076] After clustering analysis of 50 recent similar failure samples, the vehicle-mounted ETC device extracted new risk characteristics: "large vehicle following behind + distance between vehicles < 5m + lane width < 3.5m", and updated the risk scenario feature database.

[0077] When a vehicle approaches the area, the image acquisition module and image recognition model detect that "a large vehicle is entering the ETC lane 50m ahead". The system immediately calculates the current distance to be 4m and triggers a level 2 warning: a message is pushed to the vehicle's navigation system via Bluetooth, and a pop-up window prompts "There is a large vehicle ahead. It is recommended to keep a distance of more than 5m and wait for the large vehicle to pass before entering the lane".

[0078] After the warning was issued, the average distance between cars and the car in front in the area increased to 6 meters, the communication failure rate caused by obstruction decreased, and traffic efficiency improved.

[0079] Some users experienced issues such as "Visual recognition of guardrail / indicator light failed" and "Transaction interrupted" due to vehicle replacement or temporary removal of the on-board ETC device, which caused the device angle to shift (the image acquisition module was not facing the front of the lane). The EF02 file recorded "Image tilt angle > 30°", "Guardrail status recognition result = 0 (not recognized)" and "Image recognition model confidence < 60%".

[0080] When three or more records of the same type of abnormality are accumulated, the system determines it to be a high-risk scenario of "abnormal device angle" and pushes a message to the vehicle's infotainment system via Bluetooth: "The angle of the vehicle's ETC device may be off (the image acquisition module must be facing the front of the lane). It is recommended to re-paste (place horizontally, tilt angle < 5°)", along with a diagram showing the correct installation.

[0081] After the warning, users checked and adjusted the angle of their vehicle-mounted ETC devices, which reduced the failure rate of visual recognition caused by angle issues and the false alarm rate of "device failure" after-sales service.

[0082] Corresponding to the above method, the present invention also provides an ETC transaction risk early warning system based on visual recognition and closed-loop update, which is integrated into the vehicle-mounted ETC device, such as... Figure 2 As shown, the system mainly includes: The image acquisition and recognition module is used to acquire images of the lane scene in front of the vehicle and perform visual recognition through a pre-trained image recognition model to extract scene features from the images.

[0083] The risk matching module, connected to the image acquisition and recognition module, is used to perform similarity matching calculations between the extracted scene features and risk features in the pre-stored risk scene feature library.

[0084] The early warning module, connected to the risk matching module, is used to generate corresponding early warning instructions and issue graded early warning prompts to the driver when the similarity matching result exceeds the current preset similarity threshold.

[0085] The closed-loop update module is connected to the image acquisition and recognition module and the risk matching module respectively. When an ETC transaction anomaly still occurs after an alert, it records the scenario characteristics and transaction data of the current transaction as incremental samples, and updates the risk scenario feature library and similarity threshold based on the incremental samples.

[0086] In summary, this invention provides an ETC transaction risk warning method and system based on visual recognition and closed-loop updating, comprising: acquiring lane scene images through the image acquisition module of the on-board ETC device, and identifying scene features in real time using an image recognition model; matching the identified scene features with a pre-stored risk scene feature library for similarity, and issuing a graded warning if the similarity exceeds a threshold; when transaction failure still occurs after the warning or when there is an anomaly where the visual judgment result contradicts the communication judgment result, automatically recording incremental samples, and continuously optimizing and updating the risk feature library and warning threshold based on a mechanism of local clustering analysis and cloud-based federated learning. This invention can proactively warn of high-risk traffic scenarios, significantly improve the success rate of ETC transactions, accurately define responsibilities through visual cross-validation, and continuously adapt to complex traffic environments using self-learning capabilities, ultimately achieving synergistic optimization of traffic efficiency and user experience.

[0087] Corresponding to the above method, the present invention also provides an electronic device including a computer device, the computer device including a processor and a memory, the memory storing computer instructions, the processor executing the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the electronic device performs the steps of the method as described above.

[0088] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned method. The computer-readable storage medium may be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0089] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0090] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0091] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0092] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for early warning of ETC transaction risks based on visual recognition and closed-loop update, characterized in that, The method is applied to an on-board ETC device, which pre-stores a risk scenario feature database. The method includes: The image acquisition module of the vehicle-mounted ETC device acquires images of the lane scene in front of the vehicle, and performs visual recognition through a pre-trained image recognition model to extract scene features from the images; wherein, the scene features include the state of the guardrail, the color of the indicator lights, the weather environment, the lane attributes, and the vehicle status; The similarity matching calculation is performed between the scenario features and the risk features in the risk scenario feature library; the risk scenario feature library is constructed based on historical scenario features, which correspond to historical ETC transaction anomaly records, including transaction failure records and transaction contradiction records; wherein, the transaction contradiction record refers to the record where the transaction result judged based on visual recognition is inconsistent with the transaction result judged based on dedicated short-range communication; If the similarity matching result exceeds the current preset similarity threshold, a corresponding warning instruction is generated, and a graded warning prompt is issued to the driver through the human-machine interaction module of the vehicle-mounted ETC device. When an ETC transaction anomaly still occurs after an alert, the scenario characteristics and transaction data of the current transaction are recorded as incremental samples, and the risk scenario feature library and the similarity threshold are updated in a closed loop based on the incremental samples. The process of updating the risk scenario feature library based on the incremental samples includes a local update mode and a cloud-based collaborative update mode. The local update mode involves: based on a first preset frequency, and when the vehicle to which the on-board ETC device belongs is stationary, calling the DBSCAN density clustering algorithm to perform cluster analysis on the incremental samples, obtaining multiple clusters, extracting the common features of all samples within each cluster, generating new risk features, and updating the local risk scenario feature library with these new risk features. The cloud-based collaborative update mode involves: based on a second preset frequency, each on-board ETC device encrypts and uploads its local incremental samples to a cloud server; receiving optimized risk scenario feature library parameters generated by the cloud server through aggregating the incremental samples uploaded by each on-board ETC device and using a federated learning algorithm for global model training; and updating the local risk scenario feature library based on these parameters. The first preset frequency is higher than the second preset frequency.

2. The ETC transaction risk early warning method based on visual recognition and closed-loop update according to claim 1, characterized in that, The image recognition model is a convolutional neural network, which adopts a depthwise separable convolutional structure, including an input layer, multiple feature extraction layers, multiple classification layers, and a feature output layer. The classification layer is used to output the status of the railing and the color of the indicator light.

3. The ETC transaction risk early warning method based on visual recognition and closed-loop update according to claim 1, characterized in that, Before executing the method, the following is also included: The transaction failure record file of the vehicle-mounted ETC device is expanded by adding a scene feature field. The scene feature field includes a barrier status field, an indicator light color field, a confidence level field, and a comprehensive feature field. The comprehensive feature field includes quantitative feature parameters of weather environment, lane attributes, and vehicle status. The transaction success rate file of the vehicle-mounted ETC device is expanded by adding a vision-communication consistency rate field and an early warning effectiveness field.

4. The ETC transaction risk early warning method based on visual recognition and closed-loop update according to claim 1, characterized in that, The step of issuing tiered warning prompts to the driver through the human-machine interaction module of the on-board ETC device includes: If the similarity matching result exceeds the first-level threshold, a first-level warning is triggered, and an audio-visual prompt is given through the local buzzer and display screen of the vehicle-mounted ETC device. If the similarity matching result exceeds the second-level threshold but does not exceed the first-level threshold, a second-level warning is triggered, and the prompt information is pushed to the vehicle central control system via Bluetooth for graphical display; wherein, the first-level threshold is greater than the second-level threshold.

5. The ETC transaction risk early warning method based on visual recognition and closed-loop update according to claim 1, characterized in that, The method also includes a liability prediction step: After an ETC transaction anomaly occurs, a contradiction analysis is performed between the transaction result determined by visual recognition based on the image recognition model and the transaction result determined by the dedicated short-range communication. If visual recognition determines that the transaction has failed while the dedicated short-range communication determines that the transaction has succeeded, a pre-judgment identifier is generated indicating that the responsible party is the toll station's back-end system. If visual recognition determines the transaction is successful while the dedicated short-range communication determines the transaction is unsuccessful, and the vehicle's lane deviation is found to be greater than the preset safety tolerance, then a pre-judgment label indicating that the responsible party is the driver due to improper operation is generated.

6. The ETC transaction risk early warning method based on visual recognition and closed-loop update according to claim 1, characterized in that, After executing the method, the following is also included: Continuously monitor and provide feedback on transaction results following early warning alerts; The effectiveness rate of early warnings triggered by specific risk scenario characteristics is calculated, whereby the early warning effectiveness rate is the proportion of successful ETC transactions after an early warning to the total number of early warnings. Based on the dynamic changes in the warning effectiveness rate, the similarity threshold for triggering warnings of specific risk scenarios is adaptively adjusted: when the warning effectiveness rate continues to increase, the corresponding similarity threshold is reduced; when the warning effectiveness rate continues to decrease, the corresponding similarity threshold is increased.

7. An ETC transaction risk early warning system based on visual recognition and closed-loop update, characterized in that, The system is integrated into an on-board ETC device and, when executed, implements the steps of the method as described in any one of claims 1 to 6. The system includes: The image acquisition and recognition module is used to acquire images of the lane scene in front of the vehicle, and to perform visual recognition through a pre-trained image recognition model to extract scene features from the images; The risk matching module, connected to the image acquisition and recognition module, is used to perform similarity matching calculations between the extracted scene features and risk features in the pre-stored risk scene feature library; The early warning module, connected to the risk matching module, is used to generate a corresponding early warning instruction and issue a graded early warning to the driver when the similarity matching result exceeds the current preset similarity threshold. The closed-loop update module is connected to the image acquisition and recognition module and the risk matching module respectively. When an ETC transaction anomaly still occurs after an early warning, it records the scenario characteristics and transaction data of the current transaction as incremental samples, and updates the risk scenario feature library and the similarity threshold based on the incremental samples.

8. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1 to 6.

Images