Truck ETC (Electronic Toll Collection) application system and method based on multi-modal data fusion dynamic detection

Abstract

The invention relates to the technical field of intelligent traffic, in particular to a truck ETC (electronic toll collection) application system and method based on multi-modal data fusion dynamic detection, and the method comprises the steps: collecting multi-modal data, and carrying out the link association through a unified transaction identifier; image features, text logic labels and track time sequence features are extracted based on a deep learning model, and feature fusion is realized by using a cross-modal attention mechanism; performing cross verification of license plate consistency, vehicle type and load matching, license authenticity and the like by adopting fusion features, and generating a risk score; and calling an adaptive anomaly detection algorithm to perform dynamic risk identification in combination with a scoring result, and generating a multi-modal evidence chain for a high-risk case. According to the method, data-driven consistency verification and automatic risk control in the application process are realized, and the real-time adaptive capacity and the abnormity prevention and control level in complex scenes such as cold chain transportation and hazardous chemical substance transportation are enhanced.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation technology, specifically a truck ETC application system and method based on multimodal data fusion and dynamic detection. Background Technology

[0002] With the rapid development of intelligent transportation, the demand for ETC (Electronic Toll Collection) applications for trucks has exploded. However, traditional systems rely on manual review and static rule verification, resulting in lengthy, inefficient processes that are susceptible to fraud. Existing technologies fragment the verification process for structured data (such as vehicle registration information) and unstructured data (such as vehicle front view images), lacking automated cross-validation capabilities. Key logic relies on manual intervention, increasing operating costs and making it difficult to meet the real-time compliance requirements of complex scenarios such as cross-regional transportation and special vehicle types.

[0003] To address the above issues, it is necessary to build an intelligent and dynamic ETC application system for trucks. Summary of the Invention

[0004] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0005] Given the following technical problems in the existing technology: the existing drive circuit occupies a large space and has potential risks.

[0006] To address the aforementioned technical problems, this invention provides the following technical solution: a method for applying for ETC (Electronic Toll Collection) for trucks based on multimodal data fusion and dynamic detection, comprising... S1. Multimodal data acquisition: Receives structured data, unstructured data and real-time streaming data, and stores them in a unified format. The structured data includes applicant and vehicle information, the unstructured data includes certificate image data, and the real-time streaming data includes dynamic data pushed by the vehicle equipment and the road network center. S2. Multimodal feature extraction step: Perform joint modeling on the multimodal data using a deep learning model to extract high-dimensional feature parameters; S3. Cross-verification and risk scoring: Based on the scenario-based parameter settings, the multimodal feature fusion and cross-verification algorithm is invoked to perform fusion verification on key indicators, generating a multimodal fusion consistency score and a global risk label. The key indicators include license plate consistency, load-vehicle matching, and authenticity of documents. S4. Anomaly Detection and Collaborative Handling: Based on the results of the multimodal fusion consistency score and global risk label, adaptive anomaly detection is performed, and standardized case reports are automatically generated for high-risk behaviors. The reports are then sent to the road network center in real time through a secure communication channel, and joint inspection results are received, thereby achieving closed-loop risk control of the application process.

[0007] As a preferred technical solution for ETC application for trucks based on multimodal data fusion and dynamic detection, image features are extracted by ResNet-50 network to extract license plate, vehicle type and image clarity, text features are extracted by BERT and LSTM models to extract regional consistency and load-vehicle type logical labels, and trajectory features are modeled by LSTM network and combined with dynamic time warping and statistical anomaly analysis to generate trajectory anomaly index.

[0008] As a preferred technical solution for ETC application for trucks based on multimodal data fusion and dynamic detection, this method uses a multi-head cross-modal attention mechanism to dynamically fuse image, text, and trajectory features, and outputs a fusion risk score, a multimodal matching score, and scenario adaptation weights for strategy adjustment.

[0009] As a preferred technical solution for a truck ETC application method based on multimodal data fusion and dynamic detection, the high-dimensional feature parameters include image quality confidence, text logical consistency score, trajectory anomaly index, certificate image and template matching degree, and vehicle type recognition confidence.

[0010] As a preferred technical solution for the ETC application method for trucks based on multimodal data fusion dynamic detection, the adaptive anomaly detection includes calling an adaptive anomaly detection algorithm for multimodal data fusion, fusing image modal confidence, text logical consistency, trajectory anomaly index and anomaly event indicator function to calculate the final risk score, and classifying it into low-risk, review or high-risk states according to thresholds.

[0011] As a preferred technical solution for a truck ETC application method based on multimodal data fusion and dynamic detection, the multimodal feature fusion and cross-validation algorithm (MFCA) formula is as follows: Where λ is the scene weight vector. Confidence score for image modality. Confidence score of the text modality To determine the degree of matching between the certificate image and the standard template, In this context, k is a variable parameter, representing the license plate consistency verification of multimodal fusion features. Multimodal fusion features: Load-vehicle dynamic matching Multimodal fusion features for verifying the authenticity of certificates , For , , The sum of the products of the penalty coefficient, where This is the penalty coefficient.

[0012] As a preferred technical solution for ETC application for trucks based on multimodal data fusion and dynamic detection, a MFCA of less than 0.5 indicates low risk. When MFCA is greater than or equal to 0.5 and less than 0.7, enhanced verification or manual review is required. A MFCA of 0.7 or higher indicates a high risk.

[0013] As a preferred technical solution for a truck ETC application method based on multimodal data fusion and dynamic detection, the adaptive anomaly detection algorithm of the multimodal data fusion is described. The formula is, Where: λ is the scene weight vector. Confidence score for image modality. Confidence score for text modality. This is a trajectory anomaly index. For the consistency scoring of certificates and licenses, 1. 2. 3 is the penalty coefficient. *) is an indicator function. This represents the percentage deviation between the actual load and the legally mandated weight limit. To compare the maximum deviation between the declared route and the actual trajectory.

[0014] As a preferred technical solution for ETC application for trucks based on multimodal data fusion and dynamic detection, a MFAM value of less than 0.5 indicates low risk. When MFAM is greater than or equal to 0.5 and less than 0.8, enhanced verification or manual review is required. A value of MFAM greater than or equal to 0.8 indicates high risk.

[0015] A system based on the aforementioned method for ETC application for trucks using multimodal data fusion and dynamic detection. include, The multimodal data access module is used to collect structured data, unstructured data, and real-time streaming data. The structured data includes applicant information, basic vehicle information, and delivery information. The unstructured data includes vehicle registration certificate images, vehicle front photos, and road transport permits. The real-time streaming data includes vehicle GPS trajectory, OBD device status, and traffic regulations pushed by the road network center. The feature extraction module is used to perform joint feature extraction on the multimodal data based on a deep learning model, and output indicators such as image modality confidence score, text modality logical consistency score, trajectory anomaly index, certificate image authenticity score and model recognition confidence. The automated cross-verification module is used to call multimodal feature fusion and cross-verification algorithms to perform multi-dimensional consistency verification of license plate consistency, load-vehicle matching, and certificate authenticity, and outputs a risk score based on a scenario adaptive weighting mechanism. The adaptive anomaly detection module is used to call the adaptive anomaly detection algorithm of multimodal fusion data based on the fusion features to dynamically perceive and assess the risk level of situations such as abnormal certificates, path deviation, and abnormal load. The abnormal collaborative handling module is used to report high-risk cases to the road network center law enforcement platform in real time via the MQTT protocol, generate standardized case reports, and complete joint inspection feedback and risk model adaptive updates.

[0016] The beneficial effects of this invention are: the method of this invention realizes data-driven consistency verification and automated risk control in the application process, and enhances the real-time adaptability and anomaly prevention and control level in complex scenarios such as cold chain transportation and hazardous chemical transportation. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is a schematic diagram of the system flow for dynamic detection based on multimodal data fusion in this invention; Figure 2 This is a schematic diagram of the collaborative process between the project and the road network center in this invention; Figure 3 This is a schematic diagram of the multimodal feature fusion dynamic intelligent control process of the present invention. Detailed Implementation

[0018] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0019] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0020] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0021] Secondly, the present invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of the present invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not according to the usual scale. Furthermore, the schematic diagrams are merely examples and should not limit the scope of protection of the present invention. In addition, actual fabrication should include three-dimensional spatial dimensions of length, width, and depth.

[0022] Example 1 Reference Figures 1-3 This embodiment provides a method for applying for ETC (Electronic Toll Collection) for trucks based on multimodal data fusion and dynamic detection, characterized in that: S1. Multimodal Data Acquisition: This step receives structured data, unstructured data, and real-time streaming data, and stores them in a unified format. The structured data includes applicant and vehicle information, the unstructured data includes document image data, and the real-time streaming data includes dynamic data pushed by in-vehicle equipment and the road network center. This step requires further explanation. Structured data is converted into JSON format through dynamic forms, while unstructured data is submitted via drag-and-drop upload. Real-time data includes GPS trajectory, OBD status information, and road traffic regulations, and is linked through a unified transaction ID for multimodal data linking.

[0023] 1.1 Real-time acquisition and dynamic verification of structured data The system captures user-inputted applicant information (ID number, address, emergency contact), basic vehicle information (vehicle model, license plate number, load capacity), delivery information (location selection / mailing address), and product selection results through a dynamic form engine.

[0024] The front end serializes form inputs into a structured JSON data stream in real time and performs field alignment and type conversion based on a predefined domain model (mapped to the backend MySQL database schema).

[0025] Output structured JSON: {"structured_info": {"applicant_info": {"identity_card":"ID card number","residence_address":"residential address","emergency_contacts": [{"name": "emergency contact person","relationship": "**","phone": "13***0"}]},"vehicle_info": {"vehicle_type": "vehicle type","license_plate": "license plate number","load_capacity_ton": load capacity,"vin": "vehicle frame number"},"delivery_info": {"delivery_method": "delivery method","service_network_id": "service network ID","shipping_address": service network address},"product_selection": {"product_id": "product ID","repayment_mode": "payment method"} 1.2 Unstructured Data Acquisition Users submit necessary unstructured files via a drag-and-drop batch upload component: scanned copies of the front and back of the vehicle registration certificate, a panoramic photo of the vehicle's front (including the license plate), and a road transport permit (required for certain vehicle types, such as container tractors). This unstructured data is then linked to the user's ID card, formatted as JSON, and output in the following format: {"unstructured_info": {"file_metadata": [{"file_type": "front view of vehicle registration certificate","original_filename": "image name of the front view of vehicle registration certificate","upload_timestamp": "upload time","file_size_bytes": image size,"mime_type": "image format","compression_algorithm": "compression method","compressed_size_bytes": compressed size,"file_fingerprint":"image fingerprint","storage_location": "storage location","ocr_status": "image status"},{"file_type": "back view of vehicle registration certificate","original_filename": "image name of the front view of vehicle registration certificate","upload_timestamp": "upload time","file_size_bytes": image size,"mime_type": "image format","compression_algorithm": "compression method","compressed_size_bytes": compressed size,"file_fingerprint": "image fingerprint","storage_location": {"file_type": "Vehicle front photo", "original_filename": "Image name of vehicle registration certificate front photo", "upload_timestamp": "Upload time", "file_size_bytes": Image size,"mime_type": "Image format", "compression_algorithm": "Compression method", "compressed_size_bytes": Compressed size,"file_fingerprint": "Image fingerprint", "storage_location": "Storage location", "ocr_status": "Image status"}, {"file_type": "Road transport permit", "original_filename": "Image name of vehicle registration certificate front photo", "upload_timestamp": "Upload time", "file_size_bytes": Image size,"mime_type": "Image format","compression_algorithm": "compression method","compressed_size_bytes": compressed size,"file_fingerprint": "image fingerprint","storage_location": "storage location","ocr_status": "image status"}],"processing_context": {"global_transaction_id": "global transaction number","upload_session_id": "upload session","storage_tier": "hot data storage","data_retention_policy": "automatic cleanup for 30 days","associated_vehicle": "associated vehicle"}}}, 1.3 Dynamic Access to Multi-Source Real-Time Streaming Data Data Source and Interface: Real-time data streams are received from in-vehicle GPS terminals, OBD devices, and other devices via a high-performance, low-latency HTTP / 2-based API interface, using encrypted (TLS 1.3) push notifications. The data is encapsulated using the Protobuf protocol and includes core fields such as latitude and longitude, instantaneous speed, precise timestamp, and unique device ID.

[0026] External industry regulation data: Through a subscription / callback mechanism, receive real-time updates on dynamic industry regulations pushed by authoritative institutions such as the Road Network Center (such as temporary traffic control areas, cross-provincial joint inspection rules, and changes in green channel regulations).

[0027] The output JSON data is: {"dynamic_info": {"gps_info": "gps data information","odb_info": "odb data information","road_policy": "road network regulations","declaration_route": "declaration route information"} This system constructs a unified data access layer to achieve automated acquisition, preprocessing, and temporary associated storage of multimodal data, providing standardized input for subsequent fusion feature extraction. The core lies in the collaborative processing mechanism for structured form data, unstructured image data, and real-time streaming data. The specific data processing procedure is as follows: S2. Multimodal Feature Extraction Step: Joint modeling is performed on the multimodal data using a deep learning model to extract high-dimensional feature parameters. It should be noted that this step... Image features are extracted using a ResNet-50 network to include license plate, vehicle type, and image clarity. Text features are extracted using BERT and LSTM models to include regional consistency and load-vehicle logical labels. Trajectory features are modeled using an LSTM network and combined with dynamic time warping and statistical anomaly analysis to generate a trajectory anomaly index.

[0028] 1. Upload of multimodal fusion data: A multi-head cross-modal attention mechanism is used to dynamically fuse image, text, and trajectory features, and outputs a fusion risk score, a multimodal matching score, and scene adaptation weights for policy adjustment.

[0029] Specifically, 2. Feature extraction from multimodal fusion data: Based on the above structured and unstructured uploaded data, the system uses a multimodal deep learning model to extract high-dimensional features and generate standardized feature fields required for subsequent verification and detection. The specific process is as follows: 2.1 Text Feature Extraction Input data: Fields from the structured_info database, including ID number, residential address, emergency contact person, vehicle type, and load capacity. Detailed data is as follows: {"structured_info": {"applicant_info": {"identity_card":"ID card number","residence_address":"residential address","emergency_contacts": [{"name": "emergency contact person","relationship": "**","phone": "13***0"}]},"vehicle_info": {"vehicle_type": "vehicle type","license_plate": "license plate number","load_capacity_ton": load capacity,"vin": "vehicle frame number"},"delivery_info": {"delivery_method": "delivery method","service_network_id": "service network ID","shipping_address": service network address},"product_selection": {"product_id": "product ID","repayment_mode": "payment method"} Feature extraction: [Text Modality Confidence Score]: Based on geographic consistency data, calculate the semantic match between the first 6 digits of the ID card (province, city, and district codes) and the residential address to generate a score. (Value range [0,1], the lower the value, the higher the consistency, used for subsequent MFCA / MFAM algorithms); The load-vehicle type logical label data is analyzed using a bidirectional LSTM to assess the reasonableness of the load (load_capacity_ton) and vehicle type (vehicle_type), outputting the probability of logical conflict (used for dynamic matching verification in the third part). Keyword compliance: Matching road transport permit necessity rules (e.g., container tractors require a road_transport_certificate) and outputting a compliance label.

[0030] 2.2 Image Feature Extraction Input data: Images of the front / back of the vehicle registration certificate, a front view of the vehicle, and an image of the road transport permit (storage path specified by storage_location) from unstructured_info.file_metadata. Detailed data is as follows: {"unstructured_info": {"file_metadata": [{"file_type": "front view of vehicle registration certificate","original_filename": "image name of the front view of vehicle registration certificate","upload_timestamp": "upload time","file_size_bytes": image size,"mime_type": "image format","compression_algorithm": "compression method","compressed_size_bytes": compressed size,"file_fingerprint":"image fingerprint","storage_location": "storage location","ocr_status": "image status"},{"file_type": "back view of vehicle registration certificate","original_filename": "image name of the front view of vehicle registration certificate","upload_timestamp": "upload time","file_size_bytes": image size,"mime_type": "image format","compression_algorithm": "compression method","compressed_size_bytes": compressed size,"file_fingerprint": "image fingerprint","storage_location": {"file_type": "Vehicle front photo", "original_filename": "Image name of vehicle registration certificate front photo", "upload_timestamp": "Upload time", "file_size_bytes": Image size,"mime_type": "Image format", "compression_algorithm": "Compression method", "compressed_size_bytes": Compressed size,"file_fingerprint": "Image fingerprint", "storage_location": "Storage location", "ocr_status": "Image status"}, {"file_type": "Road transport permit", "original_filename": "Image name of vehicle registration certificate front photo", "upload_timestamp": "Upload time", "file_size_bytes": Image size,"mime_type": "Image format","compression_algorithm": "compression method","compressed_size_bytes": compressed size,"file_fingerprint": "image fingerprint","storage_location": "storage location","ocr_status": "image status"}],"processing_context": {"global_transaction_id": "global transaction number","upload_session_id": "upload session","storage_tier": "hot data storage","data_retention_policy": "automatic cleanup for 30 days","associated_vehicle": "associated vehicle"}}}, Feature extraction: c_plate [Extracted License Plate Number]: License plate number recognition, extracting license plate characters from the front of the vehicle photo and generating text encoding c_plate (used for multimodal feature fusion and cross-validation); t_confidence [Vehicle type recognition confidence]: Extracted from the vehicle front photo and vehicle registration certificate, output vehicle type label (such as tractor / van) and confidence score t_confidence. The confidence score of vehicle type recognition ranges from 0 to 1. The higher the value, the more reliable the recognition result (used for multimodal feature fusion and cross-validation). [Confidence Score of Image Modality]: The data for the authenticity score of vehicle registration certificate and road transport certificate is used to detect forgery features (blurred edges, abnormal reflections) through a fine-tuned network, and outputs the image quality score S_image (value range [0,1], the higher the value, the greater the risk, reflecting the image quality or the credibility of feature extraction, such as: vehicle front photo clarity score (0-1), such as blurry image Simage=0.8, clear image Simage=0.1 (used for MFCA / MFAM algorithm); The matching degree between the certificate image and the standard template is determined by comparing the vehicle registration certificate and road transport certificate with the standard template, and combining the SSIM similarity and OCR field consistency. For example, if SSIM=0.1 and the OCR field matches perfectly, then Rdoc=0.1 (for MFCA / MFAM algorithm).

[0031] 2.3 Temporal Feature Extraction (LSTM-based Trajectory Modeling) Input data: GPS coordinates (latitude, longitude, timestamp) and OBD device status from real-time streaming data. Detailed data includes: {"dynamic_info": {"gps_info": "gps data information","odb_info": "odb data information", "road_policy": "road network regulations", "declaration_route": "declaration route information"} Feature extraction:

Track Anomaly Index

[0032] S3. Cross-verification and Risk Scoring: Based on scenario-specific parameter settings, the multimodal feature fusion and cross-verification algorithm is invoked to perform fusion verification on key indicators, generating a multimodal fusion consistency score and a global risk label. The key indicators include license plate consistency, load-vehicle type matching, and document authenticity. This step requires further explanation. A multi-head cross-modal attention mechanism is used to dynamically fuse image, text, and trajectory features, and outputs a fusion risk score, a multimodal matching score, and scene adaptation weights for policy adjustment.

[0033] The high-dimensional feature parameters include image quality confidence, text logical consistency score, trajectory anomaly index, matching degree between certificate image and template, and vehicle model recognition confidence.

[0034] 3. Multimodal feature fusion and cross-validation: 3.1 Input and Parameter Definition Based on multimodal fusion data feature extraction, and through the "multimodal feature fusion and cross-validation algorithm" combined with the dynamic weight allocation mechanism of business data, deep fusion and logical verification of multimodal feature data such as image and text features are performed to achieve high-precision, scenario-adaptive automated verification, ensuring the authenticity of ETC application data and business compliance, and reducing manual intervention.

[0035] The extracted multimodal fusion data includes structured data and unstructured data.

[0036] Structured data (structured_info) includes applicant information: ID number, residential address, emergency contact person; vehicle information: vehicle type, load capacity, license plate number; and delivery information: service outlet / mailing address, etc. Unstructured data (unstructured_info) includes the front and back photos of the vehicle license, the front view of the vehicle, and the road transport license (specific for container tractors); Business scenario flag (business_scenario): general freight, cold chain transportation, hazardous chemicals transportation, etc., which determines weight assignment and rule priorities.

[0037] 3.2 Multimodal Feature Fusion and Cross-Checking Scenarios and Algorithms License plate consistency verification in multimodal fusion features: The license plate number is the core identifier of a vehicle's identity, and its consistency directly affects the accuracy of ETC billing and vehicle tracking. If the license plate number filled in by the user does not match the actual license plate recognized in the front view of the vehicle, it may involve a cloned vehicle, false declaration, or human tampering, posing risks of toll evasion, smuggling, or illegal operation.

[0038] Identify license plate characters from the front view of the vehicle through ResNet-50, generate the code Cplate, and calculate the edit distance (Levenshtein distance) between the license plate number Tplate filled in by the user and Cplate.

[0039] Among them, license_plate: The license plate number filled in by the user; : The license plate number recognized from the front view of the vehicle (derived from the c_plate field in multimodal fusion data feature extraction); : Refers to the minimum number of single-character edit operations required to convert one string to another. The larger the edit distance value, the greater the difference between the two license plates.

[0040] : The edit distance between the license plate number filled in by the user and the license plate number recognized from the front view of the vehicle; This scenario is bounded by 2. If it is greater than 2, the return value is 1, indicating a risk that the license plate number filled in by the user does not match the actual license plate recognized in the front view of the vehicle; otherwise, the return value is 0, indicating no risk. For example, "Beijing A12345" vs "Beijing A12346" → difference of 1 character, which may be a cloned vehicle or a false application, and this interception meets the mandatory requirements of the Ministry of Transport for license plate consistency.

[0041] Load - vehicle type dynamic matching in multimodal fusion features: The legal load limits of different vehicle types vary significantly (e.g., the weight limit for tractors is 40 tons, and the weight limit for van trucks is 30 tons). Overloading not only violates the Road Traffic Safety Law but also causes road damage, fines, and insurance claim rejections. Precise matching can ensure compliant operation and avoid legal and financial risks.

[0042] in, load_capacity_ton: The actual load of the vehicle as input by the user; The legal load limit for this vehicle type (e.g., 40 tons for tractor units and 30 tons for box trucks). : Confidence of vehicle model recognition, ranging from 0 to 1. The higher the value, the more reliable the recognition result (derived from the feature extraction field t_confidence of multimodal fusion data). The formula calculates the deviation ratio between the actual load and the legal weight limit. If the limit is exceeded by 50%, a risk is triggered; this is based on the vehicle model recognition confidence level. If the value is less than 0.8, the vehicle model may be misjudged and requires manual verification.

[0043] This interception complies with the mandatory requirements of the Road Traffic Safety Law regarding load limits, avoiding the risks of fines, road damage, or insurance claim denials caused by overloading. By dynamically matching vehicle type and load, it ensures transportation compliance and reduces legal and financial risks.

[0044] Multimodal fusion feature verification of vehicle registration documents: The vehicle registration certificate is the core proof of a vehicle's legality, and forging documents is a common method of ETC fraud. By detecting the authenticity of images and comparing key fields, fake documents that have been altered, photoshopped, or have printing defects can be effectively identified.

[0045] OCR extracts the text (license plate number, vehicle identification number) from the vehicle registration certificate, calculates the similarity with the fields filled in by the user, and uses the SSIM algorithm to compare the local texture, brightness, and contrast of the vehicle registration certificate image with a standard template.

[0046] in, u_driving_image: A user-provided image of the driver's license; t_driving_image: Standard vehicle registration certificate template image; SSIM (Structural Similarity Index): Used to compare the similarity of local texture, brightness, contrast and other features between two images. Its value ranges from 0 to 1, and the closer the value is to 1, the higher the similarity.

[0047] Note: In this scenario, a threshold of 0.85 is used. If the SSIM value is less than 0.85, the return value is 1, indicating that the vehicle registration certificate image provided by the user differs significantly from the standard template, and there is a risk of forgery or tampering; otherwise, the return value is 0, indicating that the image is credible and there is no risk.

[0048] When the similarity is below 0.85, it is judged as high risk (such as partial image alteration, obscuring, or non-standard template), which may be fake material. This interception complies with the Ministry of Transport's mandatory requirements for the authenticity and consistency of vehicle registration certificates.

[0049] Multimodal feature fusion and cross-validation algorithm: Different transportation scenarios have different risk focuses (e.g., cold chain focuses on temperature control, hazardous chemicals focus on route). The dynamic weighting mechanism can adapt to business needs, balance efficiency and risk control, and avoid misjudgment by "one-size-fits-all" approaches.

[0050] The formula for the multimodal feature fusion and cross-validation algorithm is as follows: in, λ: Scenario weight vector; the weight for the general freight scenario is... It is 0.6. It is 0.2. The weight is 0.2 (focusing on certificate verification), and the weight for cold chain transportation scenarios is... It is 0.3. It is 0.2. The value is 0.5 (focusing on OBD temperature control abnormalities). The confidence score of the image modality (derived from the feature extraction field Simage of multimodal fusion data) reflects the image quality or the reliability of feature extraction. For example, the sharpness score of the front view of a car (0-1) is as follows: blurry image Simage=0.8, sharp image Simage=0.1. The confidence score of the text modality (derived from the feature extraction field Ptext of multimodal fusion data) measures the logical consistency or completeness of structured text, such as the consistency score (0-1) between the address field and the emergency contact's region, such as a perfect match Ptext=0.1; The matching degree between the certificate image and the standard template (derived from the feature extraction field Rdoc of multimodal fusion data), which combines SSIM similarity and OCR field consistency. For example, if SSIM=0.1 and the OCR field is a perfect match, then Rdoc=0.1. :k is a variable parameter, representing the license plate consistency verification of multimodal fusion features. Multimodal fusion features: Load-vehicle dynamic matching Multimodal fusion features for verifying the authenticity of certificates : is the sum of the products of the above three scenarios and the penalty coefficient, where the penalty coefficient is... The initial value is 0.2, and it is increased to 0.3 in high-risk scenarios, such as abnormal certificates or licenses. When MFCA is less than 0.5, it is a low risk, such as compliance - ordinary truck; When MFCA is greater than or equal to 0.5 and less than 0.7, enhanced verification or manual review is required; When MFCA is greater than or equal to 0.7, it is a high risk, such as rejection and freezing for forged certificates.

[0051] Business scenarios in the process of applying for truck ETC: Scenario 1: Compliance application for general freight Input information: General freight: λ = [0.6, 0.2, 0.2]; Vehicle type: Tractor (max(Ptype) = 0.9, load capacity: 30 tons (weight limit 30 tons); Driver's license: Clear and complete (Simage = 0.1, SSIM = 0.12); Confidence score of text modality, license plate number: The user fills in "Beijing A12345", and the OCR extraction is consistent, which is 0.1; The matching degree of the certificate image and the standard template is relatively high: which is 0.2; Calculated by multi - modal feature fusion and cross - verification algorithm: MFCA = 0.6×0.1 + 0.2×0.1 + 0.2×0.2 = 0.12 0.12 ≤ 0.7, and all verification marks pass. The judgment logic is automatic pass, and at the same time, asynchronously report to the road network center.

[0052] Scenario 2: Abnormal certificates for cold - chain transportation Input information: Cold - chain transportation: λ = [0.3, 0.2, 0.5]; Vehicle type: Cold - chain vehicle (max(Ptype) = 0.88), load capacity: 35 tons; Driver's license: Blurred and tampered (Simage = 0.8, SSIM = 0.72); Confidence score of text modality: The logic of the structured text filled in by the user is basically consistent, which is 0.2; The matching degree of the certificate image and the standard template: The certificate photo, such as the driver's license, has blurred and tampered, The matching degree of the certificate image and the standard template is relatively low, so Idoc is 0.7.

[0053] Calculated by multi - modal feature fusion and cross - verification algorithm: MFCA = 0.3×0.8 + 0.2×0.2 + 0.5×0.7 + 1×0.3 = 0.93 If the value is 0.93≥0.7 and the verification mark fails (verification of the authenticity of the certificate based on the multimodal fusion feature), the application is rejected and frozen, marked as "certificate abnormal". At the same time, the evidence chain of the tampered certificate image is reported to the road network center in real time.

[0054] The above multimodal feature fusion and cross-validation algorithm performs unified and intelligent pre-verification of the multimodal fusion data for ETC application of trucks. Through multi-rule collaboration and dynamic scoring, it realizes full-link intelligence from "data verification" to "risk decision-making", providing highly reliable and high-concurrency risk control support for ETC application.

[0055] S4. Anomaly Detection and Collaborative Handling: Based on the results of the multimodal fusion consistency score and global risk label, adaptive anomaly detection is performed, and standardized case reports are automatically generated for high-risk behaviors. These reports are then uploaded to the road network center in real time via a secure communication channel, and joint inspection results are received, achieving closed-loop risk control of the application process. This step requires further explanation. The adaptive anomaly detection includes calling an adaptive anomaly detection algorithm that integrates multimodal fusion data, fusing image modal confidence, text logical consistency, trajectory anomaly index, and anomaly event indicator function to calculate the final risk score, and classifying it into low-risk, review, or high-risk states based on thresholds.

[0056] Specifically, 4. Adaptive anomaly detection: Traditional static rules are ill-suited for handling complex and ever-changing anomaly scenarios (such as dynamic forgery of certificates and evasion of overload detection). An adaptive anomaly detection mechanism is needed, which combines real-time data to dynamically adjust dynamic rule strategies, enabling adaptive anomaly monitoring based on multimodal data fusion.

[0057] 4.1 Input and Parameter Definition The pre-resources for adaptive anomaly detection using multimodal fusion data include four parts: real-time data stream, multimodal feature data, business scenario identifiers, and dynamic parameters, detailed below. Real-time data stream includes vehicle GPS trajectory: latitude and longitude, speed, timestamp (Protobuf format, sampling frequency 1Hz), GPS trajectory (path variability). OBD device status (Trajectory Anomaly Index IGPS); OBD device status: engine speed, fuel consumption, temperature, fault codes (transmitted via TLS 1.3 encryption).

[0058] Multimodal feature data includes structured data: user-entered load (Winput), vehicle type (Vtype), license plate number (Tplate), address, and ID number; and unstructured data: vehicle registration certificate and front view photo (from which the authenticity score of the certificate is extracted). Vehicle type classification probability Ptype) Business scenario identifiers: general freight, cold chain transportation, hazardous chemical transportation, etc., which drive the priority of detection strategies.

[0059] Anomaly detection threshold for dynamic parameters: The scene weight vector is λ=[λ1,λ2,λ3]. For general freight: λ=[0.6,0.2,0.2] (focusing on certificate verification), and for special or cold chain transportation: λ=[0.3,0.2,0.5] (focusing on OBD temperature control anomalies).

[0060] Risk penalty coefficient for constraint verification =[ 1, 2, 3]: Penalty value for failed certificate, load, and path verification (default) =[0.2,0.2,0.2], with the risk level increased to 0.3 for high-risk scenarios.

[0061] Constraint thresholds: Load deviation limit ΔWlimit = 0.5, path difference threshold. = 200km.

[0062] 4.2 Adaptive Anomaly Detection Algorithm for Multimodal Fusion Data Based on the dynamic scenario of the integrated data for truck ETC application, an adaptive anomaly detection algorithm for multimodal fusion data is proposed.

[0063] The formula for the adaptive anomaly detection algorithm for multimodal fusion data is as follows: in, λ: Scenario weight vector; the weight for the general freight scenario is... It is 0.6. It is 0.2. The weight is 0.2 (focusing on certificate verification), and the weight for cold chain transportation scenarios is... It is 0.3. It is 0.2. The value is 0.5 (focusing on OBD temperature control abnormalities). : Confidence score of image modality (derived from feature extraction field of multimodal fusion data) This reflects the image quality or the reliability of feature extraction, such as the sharpness score of a car's front view (0-1), or a blurry image. =0.8, clear image =0.1; The confidence score of the text modality (derived from the feature extraction field Ptext of multimodal fusion data) measures the logical consistency or completeness of structured text, such as the consistency score (0-1) between the address field and the emergency contact's region, such as a perfect match Ptext=0.1; Track anomaly index (0-1) quantifies GPS / OBD data anomalies (such as route deviation or equipment failure); the higher the value, the higher the risk. 1, 2, 3: Penalty coefficient, an additional risk factor when verification fails (default 0.2, 0.3 in high-risk scenarios). If the certificates and licenses are inconsistent, a risk will be triggered, and the penalty coefficient will increase. *) is an indicator function. It returns 1 if the condition in parentheses is true, and 0 if it is false.

[0064] A certificate consistency score of less than 75% indicates a risk of triggering consistency issues (derived from the Rdoc feature extraction field of multimodal fusion data). The deviation of the actual load from the legal weight limit; if it exceeds the limit by 50%, a risk is triggered. and The maximum deviation between the declared route and the actual trajectory is compared. If it is less than the actual deviation, there is no risk; otherwise, a path deviation risk is triggered.

[0065] When MFAM is less than 0.5, it indicates low risk, such as for compliant applications; When MFAM is greater than or equal to 0.5 and less than 0.8, enhanced verification or manual review is required. When MFAM is greater than or equal to 0.8, it is considered high risk, such as abnormal dynamic paths, and the application will be rejected and frozen.

[0066] Anomaly monitoring scenarios during the ETC application process for trucks: Scenario 1: Cold chain transport vehicle declaration is normal Input information: image: =0.1; text: =0.15, load capacity Winput=40 tons (standard load limit Wlimit=30) Timing: =0.2, =180km.

[0067] Dynamic parameter loading: Cold chain scenario λ=[0.2,0.1,0.7], =[0.2,0.2,0.2].

[0068] Among them, constraint verification: Document verification: =0.85 (Pass, no penalty); Load matching: =0.33<0.5 (Pass, no penalty); Path verification: =180km<200km (Pass, no penalty).

[0069] Results of the adaptive anomaly detection algorithm for multimodal fusion data: The system's decision logic is to automatically approve the request, while simultaneously reporting it asynchronously to the road network center.

[0070] Scenario 2: Violations of hazardous chemical transportation routes Input information: Image: Simage=0.1, dangerous goods permit is valid; Text: Ptext=0.2; Timing: IGPS=0.6, OBD temperature exceeds the limit (T=55°C>Tmax).

[0071] Dynamic parameter loading: Hazardous chemicals scenario λ=[0.5,0.1,0.4], =[0.3,0.3,0.3].

[0072] Among them, constraint verification: Document verification: Rdoc = 0.4 × 0.95 + 0.3 × 0.90 + 0.3 × 0.92 = 0.926 (passed); Load matching: =0.2<0.5 (Pass); Route verification: Dpath=150km<200km (passed); Prohibited Area Detection: GPS track passes through the vicinity of the school for 0.8km (triggering additional penalty) path=0.3).

[0073] Results of the adaptive anomaly detection algorithm for multimodal fusion data: The system rejected and froze the account, marking it as "abnormal route + temperature control exceeding the standard". At the same time, the evidence chain of abnormal route and temperature control exceeding the standard was reported to the road network center in real time.

[0074] 5. Collaborative handling of anomalies Traditional ETC application systems rely on manual review and offline feedback in the anomaly handling process, making it difficult to achieve real-time risk prevention and cross-departmental collaboration. This system, by constructing a dual-channel mechanism of "real-time reporting and asynchronous feedback" and combining it with the global law enforcement resources of the road network center, enables rapid interception and closed-loop handling of high-risk behaviors, forming an intelligent anomaly collaborative handling system.

[0075] 5.1 Input Information for Collaborative Anomaly Handling The input information for collaborative handling of anomalies consists of high-risk case data, which includes three parts: anomaly type tags such as: tampering with certificates, GPS trajectory deviation, overloading, and route violation; Multimodal evidence chain: tampered certificate images, abnormal trajectory heatmaps, OBD device anomaly logs, risk scoring reports (MFCA / MFAM); Associated Transaction ID: Globally unique UUID v4, used to trace the complete data chain.

[0076] The above three types of data are reported through real-time and asynchronous reporting interfaces. Specifically, the real-time reporting interface is based on the MQTT protocol and supports high-concurrency real-time push of high-risk cases; the asynchronous feedback interface is based on a RESTful API and periodically pushes multimodal verification results (such as verification pass rate and risk score distribution). The reporting priority is allocated to transmission bandwidth and response time according to the risk level (low / medium / high), and sensitive data (such as license plate numbers and ID card numbers) are encrypted using the national cryptographic algorithm SM4.

[0077] 5.2 Collaborative Response Procedure for Abnormalities When the system determines high-risk behavior (e.g., MFAM ≥ 0.8) using an adaptive anomaly detection algorithm, the following process is automatically triggered: First, the evidence chain is encapsulated by extracting related data (structured forms, certificate images, GPS trajectories) from temporary storage and generating a standardized case report (JSON format); then, multimodal feature data is embedded: certificate authenticity score (S~image~), trajectory deviation (D~path~), and load deviation rate (ΔW); additional visual evidence is added: the tampered areas are marked through image segmentation (Mask R-CNN) to generate a trajectory heatmap (based on the Folium library).

[0078] Then comes encrypted transmission, with field-level encryption (SM4) applied to the case report, which is then pushed to the road network center law enforcement platform in real time via the MQTT protocol; transmission parameters: QoS=1 (at least one delivery), message queue Topic="etc / risk / emergency".

[0079] Finally, the status is synchronized. The system marks the case status as "reported" and freezes the ETC application permissions of the associated account. The front end prompts the user that "the case has entered the joint investigation process. Please cooperate with the subsequent investigation."

[0080] 5.3 Asynchronous reporting of multimodal verification results For all application requests (regardless of risk level), the system periodically (e.g., hourly) summarizes and verifies the results, and reports them to the road network center via a RESTful API.

[0081] The first step is data aggregation, which involves statistically analyzing the pass rate, average risk score, and high-frequency anomaly types for each scenario; and extracting typical case characteristics (such as the proportion of temperature control violations in "hazardous chemical route violations").

[0082] The second step is asynchronous push, with the data format conforming to the "Transportation Industry Standard" (JT / T 809-2019); the interface address is: POST / etc / audit / results, and the header contains an authentication token (JWT).

[0083] The third step is feedback reception, where the road network center returns the inspection results (such as updates to the banned list and suggestions for optimizing regulations); the system dynamically adjusts the risk model thresholds based on the feedback (such as increasing the ϕ coefficient for tampering with licenses).

[0084] 5.4 Feedback loop after collaborative processing After receiving the reported data, the road network center initiates a closed-loop joint handling process. For cases of fraudulent applications confirmed by the audit, the road network center returns a "case handled" signal and updates the banned list; the system automatically unfreezes the permissions of compliant users and permanently bans accounts confirmed to be fraudulent.

[0085] The road network center provides historical case data (such as high-frequency features of license tampering) to fine-tune the multimodal detection model; the dynamic rule engine automatically generates adaptive verification strategies based on changes in traffic rules (such as the addition of restricted areas).

[0086] In summary, this paper presents a truck ETC application system and method based on multimodal data fusion and dynamic detection. It deeply integrates multimodal data with core resources of the road network center, achieving a closed-loop process of data sharing, rule synchronization, and anomaly linkage. Based on a deep learning model, it extracts features from structured and unstructured data, and combines this with a "multimodal feature fusion and cross-validation algorithm" to achieve automated cross-validation (such as load and vehicle type matching, and document consistency verification), significantly reducing reliance on manual intervention. Simultaneously, the system introduces a dynamic rule anomaly detection mechanism, using an "adaptive anomaly detection algorithm for multimodal fusion data" to detect data anomalies (such as tampered fields, logically conflicting declared routes, and GPS trajectory differences), effectively resisting fraudulent activities. This improves the automation level and verification accuracy of the ETC application process, effectively resisting fraudulent activities such as tampered fields and logical conflicts. Furthermore, through the rapid access and secure integration of multi-source heterogeneous data (such as vehicle GPS and OBD devices), it enhances the real-time adaptability to complex scenarios (such as cross-regional transportation and special vehicle types), ensuring the efficiency and reliability of data transmission and verification, ultimately achieving a highly secure and high-coverage intelligent ETC application service.

[0087] This invention focuses on multimodal data fusion and dynamic rule collaboration technologies. By co-building an intelligent collaborative platform with the road network center, it deeply integrates multimodal data with the core resources of the road network center, achieving a closed-loop process of data sharing, rule synchronization, and anomaly linkage. Based on deep learning models, it extracts features from structured and unstructured data, and combines a "multimodal feature fusion and cross-validation algorithm" to achieve automated cross-validation (such as load and vehicle type matching, and certificate consistency verification), significantly reducing reliance on manual labor. At the same time, the system introduces a dynamic rule anomaly detection mechanism, using an "adaptive anomaly detection algorithm for multimodal fusion data" to detect data anomalies (such as tampered fields, logically conflicting declared routes, and GPS trajectory differences), effectively resisting fraudulent behavior. Based on real-time data (such as vehicle GPS trajectories and OBD device feedback), it automatically adjusts the verification strategy, supporting the evolution from static "one-size-fits-all" rules to scenario-driven dynamic rules, thereby improving the adaptability to complex scenarios.

[0088] A truck ETC application system and method based on multimodal data fusion and dynamic verification was developed. This system enables multimodal data cross-verification and consistency assurance in an intelligent transportation environment, efficient identification and blocking of fraudulent behavior through an adaptive anomaly detection mechanism, real-time scenario adaptability improvement driven by a dynamic rule engine, and rapid access and secure integration of multi-source heterogeneous data (such as vehicle GPS and OBD devices). The system detects high-risk behaviors such as tampering with licenses and abnormal GPS trajectories and reports them in real time, and automatically pushes case details (including multimodal evidence chains) to the road network center, thereby building a highly secure and high-coverage intelligent ETC application ecosystem.

[0089] Although the content of this application has been described in detail through the above examples, it should be understood that the above description should not be considered as a limitation of the invention. Therefore, the scope of protection of the invention is described as follows: This application protects all technologies related to multimodal feature fusion and cross-validation in the ETC application process for trucks; adaptive anomaly detection algorithms for multimodal fusion data in the ETC application process for trucks; multimodal data fusion and dynamic rule collaboration technologies in the ETC application process for trucks; and data sharing, rule synchronization, and anomaly linkage with core resources of the road network center in the ETC application process for trucks.

[0090] It should be understood that numerous specific implementation decisions can be made during the development of any practical implementation, such as in any engineering or design project. Such development efforts may be complex and time-consuming, but for those skilled in the art who benefit from this disclosure, the development effort will be a routine work of design, manufacturing, and production without requiring much experimentation.

[0091] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for applying for ETC (Electronic Toll Collection) for trucks based on multimodal data fusion and dynamic detection, characterized in that: S1. Multimodal data acquisition: Receives structured data, unstructured data and real-time streaming data, and stores them in a unified format. The structured data includes applicant and vehicle information, the unstructured data includes certificate image data, and the real-time streaming data includes dynamic data pushed by the vehicle equipment and the road network center. S2. Multimodal feature extraction step: Perform joint modeling on the multimodal data using a deep learning model to extract high-dimensional feature parameters; S3. Cross-verification and risk scoring: Based on the scenario-based parameter settings, the multimodal feature fusion and cross-verification algorithm is invoked to perform fusion verification on key indicators, generating a multimodal fusion consistency score and a global risk label. The key indicators include license plate consistency, load-vehicle matching, and authenticity of documents. S4. Anomaly Detection and Collaborative Handling: Based on the results of the multimodal fusion consistency score and global risk label, adaptive anomaly detection is performed, and standardized case reports are automatically generated for high-risk behaviors. The reports are then sent to the road network center in real time through a secure communication channel, and joint inspection results are received, thereby achieving closed-loop risk control of the application process.

2. The method for applying for ETC for trucks based on multimodal data fusion and dynamic detection according to claim 1, characterized in that: Image features are extracted using a ResNet-50 network to include license plate, vehicle type, and image clarity. Text features are extracted using BERT and LSTM models to include regional consistency and load-vehicle logical labels. Trajectory features are modeled using an LSTM network and combined with dynamic time warping and statistical anomaly analysis to generate a trajectory anomaly index.

3. The method for applying for ETC for trucks based on multimodal data fusion and dynamic detection according to claim 2, characterized in that: A multi-head cross-modal attention mechanism is used to dynamically fuse image, text, and trajectory features, and outputs a fusion risk score, a multimodal matching score, and scene adaptation weights for policy adjustment.

4. The method for applying for ETC for trucks based on multimodal data fusion and dynamic detection according to claim 3, characterized in that: The high-dimensional feature parameters include image quality confidence, text logical consistency score, trajectory anomaly index, matching degree between certificate image and template, and vehicle model recognition confidence.

5. The method for applying for ETC for trucks based on multimodal data fusion and dynamic detection according to claim 4, characterized in that: The adaptive anomaly detection includes calling an adaptive anomaly detection algorithm that integrates multimodal fusion data, fusing image modal confidence, text logical consistency, trajectory anomaly index, and anomaly event indicator function to calculate the final risk score, and classifying it into low-risk, review, or high-risk states based on thresholds.

6. The method for applying for ETC for trucks based on multimodal data fusion and dynamic detection according to claim 5, characterized in that: The Multimodal Feature Fusion and Cross-Validation (MFCA) algorithm formula is as follows: Where λ is the scene weight vector. Confidence score for image modality. Confidence score of the text modality To determine the matching degree between the certificate image and the standard template, In this context, k is a variable parameter, representing the license plate consistency verification of multimodal fusion features. Multimodal fusion features: Load-vehicle dynamic matching Multimodal fusion feature for certificate authenticity verification , For , , The sum of the products of the penalty coefficient, where This is the penalty coefficient.

7. The method for applying for ETC for trucks based on multimodal data fusion and dynamic detection according to claim 6, characterized in that: When MFCA is less than 0.5, it is considered low risk; When MFCA is greater than or equal to 0.5 and less than 0.7, enhanced verification or manual review is required. A MFCA of 0.7 or higher indicates a high risk.

8. The method for applying for ETC for trucks based on multimodal data fusion and dynamic detection according to claim 7, characterized in that: The adaptive anomaly detection algorithm for multimodal fusion data The formula is, Where: λ is the scene weight vector. Confidence score for image modality. Confidence score for text modality. This is a trajectory anomaly index. To score the consistency between licenses and permits, 1.

2. 3 is the penalty coefficient. *) is an indicator function. This represents the percentage deviation between the actual load and the legally mandated weight limit. To compare the maximum deviation between the declared route and the actual trajectory.

9. The method for applying for ETC for trucks based on multimodal data fusion and dynamic detection according to claim 8, characterized in that: When MFAM is less than 0.5, it is considered low risk; When MFAM is greater than or equal to 0.5 and less than 0.8, enhanced verification or manual review is required. A value of MFAM greater than or equal to 0.8 indicates high risk.

10. A system based on the truck ETC application method based on multimodal data fusion dynamic detection as described in any one of claims 1 to 9, characterized in that: include, The multimodal data access module is used to collect structured data, unstructured data, and real-time streaming data. The structured data includes applicant information, basic vehicle information, and delivery information. The unstructured data includes vehicle registration certificate images, vehicle front photos, and road transport permits. The real-time streaming data includes vehicle GPS trajectory, OBD device status, and traffic regulations pushed by the road network center. The feature extraction module is used to perform joint feature extraction on the multimodal data based on a deep learning model, and output indicators such as image modality confidence score, text modality logical consistency score, trajectory anomaly index, certificate image authenticity score and model recognition confidence. The automated cross-verification module is used to call multimodal feature fusion and cross-verification algorithms to perform multi-dimensional consistency verification of license plate consistency, load-vehicle matching, and certificate authenticity, and outputs a risk score based on a scenario adaptive weighting mechanism. The adaptive anomaly detection module is used to call the adaptive anomaly detection algorithm of multimodal fusion data based on the fusion features to dynamically perceive and assess the risk level of situations such as abnormal certificates, path deviation, and abnormal load. The abnormal collaborative handling module is used to report high-risk cases to the road network center law enforcement platform in real time via the MQTT protocol, generate standardized case reports, and complete joint inspection feedback and risk model adaptive updates.

Images