Illegal operating vehicle detection method, device and equipment and storage medium

Abstract

The invention discloses an illegal operating vehicle detection method, device and equipment and a storage medium. Comprising the following steps: extracting track data, ETC passing data, electronic checkpoint snapshot data, operation platform data and toll station transaction data of a vehicle to obtain multi-modal data of the vehicle; carrying out feature extraction based on the multi-modal data to obtain passing features, personnel features and qualification features of the vehicle; and inputting the passing characteristics, the personnel characteristics and the qualification characteristics into a pre-trained violation detection model to obtain a judgment result of whether the vehicle is an illegal operating vehicle. According to the invention, efficient fusion and joint feature extraction of the trajectory data and the visual data are realized. Compared with a traditional method using a single data source, the method can more comprehensively and accurately capture vehicle behavior characteristics, improves the recognition accuracy, removes privacy information through constructing a behavior characteristic extraction mode, and gives consideration to illegal operation recognition accuracy and privacy compliance.

Description

Technical Field

[0001] This application relates to the field of traffic data processing technology, and more specifically, to a method, apparatus, equipment, and storage medium for detecting illegally operating vehicles. Background Technology

[0002] With the rapid development of the transportation industry, the problem of illegal operating vehicles has become increasingly prominent, seriously affecting the market order of legally operating vehicles and posing a potential threat to passenger safety.

[0003] Traditional methods for detecting unlicensed vehicles primarily rely on the analysis of vehicle GPS trajectory data. They identify potential illegal operations by detecting spatiotemporal characteristics of vehicle trajectories, such as high-frequency short-distance movement and clustering in specific areas. However, this method has several limitations. For example, it can easily misclassify legitimate short-distance operations as illegal, leading to unreliable results. Secondly, the rules are rigid; traditional detection methods are based on fixed rules and cannot flexibly handle legitimate special circumstances, such as temporary changes in routes, resulting in a high false alarm rate. Finally, privacy protection issues are increasingly prominent; traditional methods involve the collection and analysis of large amounts of personal information, potentially leading to the leakage of vehicle owners' or drivers' privacy. Summary of the Invention

[0004] This application provides a method, apparatus, equipment, and storage medium for detecting illegally operated vehicles, thereby at least solving the technical problem in the related art of accurately, efficiently, and safely detecting illegally operated vehicles.

[0005] According to one aspect of the embodiments of this application, a method for detecting illegally operating vehicles is provided, comprising: Extract vehicle trajectory data, ETC passage data, electronic checkpoint capture data, operation platform data, and toll station transaction data to obtain multimodal vehicle data; Based on the multimodal data, feature extraction is performed to obtain vehicle traffic features, personnel features, and qualification features; The traffic characteristics, personnel characteristics, and qualification characteristics are input into a pre-trained violation detection model to obtain a judgment result on whether the vehicle is an illegal operating vehicle.

[0006] According to another aspect of the embodiments of this application, a detection device for illegally operating vehicles is provided, comprising: The data fusion module is used to extract vehicle trajectory data, ETC passage data, electronic checkpoint capture data, operation platform data, and toll station transaction data to obtain multimodal vehicle data. The feature extraction module is used to extract features based on the multimodal data to obtain vehicle traffic features, personnel features, and qualification features; The violation detection module is used to input the traffic characteristics, personnel characteristics, and qualification characteristics into a pre-trained violation detection model to obtain a judgment result on whether the vehicle is an illegal operating vehicle.

[0007] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described method for detecting illegally operating vehicles through the computer program.

[0008] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, which is configured to execute the above-described method for detecting illegally operating vehicles when it is run.

[0009] The technical solutions provided in this application embodiment may include the following beneficial effects: This application extracts vehicle trajectory data, ETC passage data, electronic checkpoint capture data, operation platform data, and toll station transaction data to obtain multimodal data. Feature extraction is then performed, extracting passage features, personnel features, and qualification features, which are input into a violation detection model for identification. This achieves efficient fusion and joint feature extraction of trajectory and visual data, enabling more comprehensive and accurate capture of vehicle behavior characteristics compared to traditional methods, especially demonstrating significant advantages in identifying illegally operated vehicles disguised as legitimate operations. Attached Figure Description

[0010] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of an optional method for detecting illegally operating vehicles according to an embodiment of this application; Figure 2 This is a schematic diagram of a vehicle multi-data fusion process according to an embodiment of this application; Figure 3 This is a schematic diagram of a feature extraction and comparison process for image information according to an embodiment of this application; Figure 4 This is a schematic diagram of a road matching method according to an embodiment of this application; Figure 5 This is a schematic diagram of a violation detection model training method according to an embodiment of this application; Figure 6 This is a schematic diagram of a dynamic correction method for a violation detection model according to an embodiment of this application; Figure 7This is a schematic diagram of a detection device for illegally operating vehicles according to an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0011] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0012] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0013] Traditional methods for determining whether a vehicle is operating illegally often rely on the past experience of law enforcement officers to set rules, which then need to be adjusted once illegal operators discover patterns. This application addresses the problem that, by extracting feature information and periodically revising the deep learning model based on user verification results, it avoids the issue of illegal operators cleverly camouflaging themselves to evade detection after discovering patterns.

[0014] Traditional methods rely too heavily on the past experience of law enforcement personnel. When new data dimensions are input, traditional methods still require law enforcement personnel to add query rules after the R&D staff has cleaned / standardized the data before it can be integrated with the existing data. This method uses the extracted feature values ​​as the sole input source. When new data dimensions are input, only the feature value extraction rules need to be modified to complete the data fusion.

[0015] Traditional solutions require full statistical analysis of GPS data over several consecutive days, which is computationally intensive. After a period of operation, manual intervention is needed to address the issue of inaccurate data caused by patterns observed by lawbreakers. This solution only requires similarity assessment of feature information, and the system can automatically correct the rules as the user operates normally.

[0016] By solving the aforementioned technical problems, this application can significantly reduce the workload of law enforcement personnel while ensuring high detection accuracy. Large-scale detection of vehicle data to determine whether there is illegal operation can be completed by machine assembly line, providing strong technical support for intelligent traffic management.

[0017] The method for detecting illegally operating vehicles according to embodiments of this application will be described in detail below with reference to the accompanying drawings. Figure 1 As shown, the method mainly includes the following steps: S101 extracts vehicle trajectory data, ETC passage data, electronic checkpoint capture data, operation platform data, and toll station transaction data to obtain multimodal vehicle data.

[0018] This application acquires vehicle trajectory data, ETC toll data, electronic checkpoint capture data, operation platform data, and toll station transaction data. The acquired multi-source data covers key fields such as vehicle location, timestamp, license plate number, and payment status. In one embodiment, to avoid the risk of original image data leakage, image data such as image and video data from the electronic checkpoint capture data are cached on a local server. A convolutional neural network-based image feature recognition model deployed on the local server is used to extract vehicle image feature data. In one embodiment, relevant information is first extracted from multiple data sources. These data sources include vehicle trajectory data, ETC toll data, electronic checkpoint capture data, operation platform data, and toll station transaction data. By using unified interfaces such as REST API, Kafka message queues, database sharing, and FTP / SFTP file reading, we can effectively connect to these data sources, ensuring data consistency and integrity. Each data source corresponds to a type of data acquisition device.

[0019] During the data acquisition process, all included image or video data is cached on a local server to prevent privacy data leakage.

[0020] Furthermore, the extracted trajectory data, ETC passage data, image data, operation platform data, and toll station transaction data are fused. To ensure data consistency and accuracy, the time format in the trajectory data, ETC passage data, and electronic checkpoint data is unified to Beijing time, accurate to the second. The geographical location data will be uniformly processed into latitude and longitude in floating-point format to facilitate geospatial analysis.

[0021] Furthermore, this application will extract common fields such as license plate number, license plate color, image address, image type, time, and latitude and longitude, which are crucial for vehicle identification and analysis. Through these common fields, data from different sources can be linked to form a complete multimodal vehicle dataset. For example... Figure 2 The diagram illustrates a data fusion method for extracting and fusing multi-source data. It acquires image data such as images and videos captured at checkpoints and caches these images locally. Next, it checks for the existence of multiple vehicle location record sources. If multiple sources exist, the data is merged based on license plate number, vehicle color, location, and image data. This merges GPS trajectory data, ETC toll records, electronic checkpoint capture information, and toll station transaction records from various sources. If no sources exist, license plate number fusion is skipped. Finally, the time format of the data is unified to Beijing time, and the latitude and longitude format of the location data is unified to floating-point numbers. The resulting fused multimodal data includes a sequence of vehicle location data in floating-point format and a unified data sequence. Common fields such as license plate number, license plate color, image address, image type, time, and latitude and longitude can be extracted from the fused multimodal data.

[0022] S102 performs feature extraction based on multimodal data to obtain vehicle traffic features, personnel features, and qualification features.

[0023] The traffic characteristics include the preset number of target checkpoints with the most traffic, the preset number of target toll stations with the most traffic, the preset number of target ETC gantries with the most traffic, the preset number of target roads with the most traffic, and the preset number of time periods with the most traffic; the personnel characteristics include the number of driver categories and the number of times each driver drives within the preset time period; the number of passenger categories within the preset time period; and the qualification characteristics include the number of complaints, the number of cases, and the type of operation.

[0024] According to one aspect of the embodiments of this application, feature extraction based on multimodal data is used to obtain the passage features of a vehicle, including: calculating the number of times a vehicle passes through each checkpoint based on multimodal data, and obtaining a preset number of target checkpoints with the most passages; calculating the number of times a vehicle passes through each toll station based on multimodal data, and obtaining a preset number of target toll stations with the most passages; calculating the number of times a vehicle passes through each ETC gantry based on multimodal data, and obtaining a preset number of target ETC gantries with the most passages; and querying a preset number of target roads with the most passages based on the target checkpoints, target toll stations, and target ETC gantries.

[0025] Specifically, multimodal data is analyzed to calculate the number of times each vehicle passes through each checkpoint, toll station, and ETC gantry. This step involves data aggregation and statistical analysis to identify the specific checkpoints, toll stations, and ETC gantries with the highest frequency of passage, locations that may be associated with illegal operations.

[0026] For example, suppose our target number is 5, meaning we want to find the top 5 facilities with the most passage times for each type of facility. We first analyze the checkpoint data, counting the number of times each vehicle passes through each checkpoint, and then select the 5 checkpoints with the most passage times as the target checkpoints. Similarly, we perform the same analysis on the toll station and ETC gantry data, respectively identifying the 5 toll stations and 5 ETC gantries with the most passage times.

[0027] Next, using information from these target checkpoints, toll stations, and ETC gantries, vehicle passage records are further analyzed to determine which roads connect these high-frequency locations. The number of times each vehicle passes through each road is counted again, and the top 5 roads with the highest number of passes are selected as target roads. These target roads are likely routes frequently used by unlicensed vehicles.

[0028] Because the detailed vehicle location data is too large and contains private information, in one embodiment of this application, the location information in the fused multimodal data is matched with nearby roads. Specifically, the vehicle location data sequence is extracted from the multimodal data obtained by fusing multi-source data, and it is determined whether there is a road within a preset range of each vehicle location. If there is no road within a preset range of n meters of the vehicle location, the location data is discarded. Conversely, if multiple roads exist nearby, the system calculates the relative distance between the vehicle location and each road and selects the road with the closest distance as the road to which the vehicle belongs. In this way, the location information can be effectively desensitized, protecting personal privacy while retaining the necessary information for feature extraction and analysis. This method ensures data security and privacy protection while also supporting subsequent feature extraction and data analysis.

[0029] like Figure 4 The diagram illustrates a road matching method. First, the system acquires the input vehicle location data. Then, it checks if a road exists within n meters of the vehicle. If one exists, the system iterates through the area to find the nearest road and determines the road information to which the vehicle belongs. If no road exists, the process terminates, and the vehicle location data is discarded. The entire process ends at the "End" node, aiming to determine the vehicle's current location or the nearest road.

[0030] This method not only identifies key locations that illegal vehicles may frequently pass through, but also tracks the specific routes they may take, thus providing valuable information for traffic management and law enforcement.

[0031] In one implementation, feature extraction is performed based on multimodal data to obtain vehicle traffic characteristics. The method also includes counting the number of times a vehicle passes through each traffic period based on the vehicle traffic time in the multimodal data, and obtaining a preset number of traffic periods with the most traffic.

[0032] Specifically, the system acquires the travel time corresponding to the vehicle's trajectory data and counts the activity frequency of vehicles during each travel period. By counting the number of times a vehicle travels during each travel period, peak travel times can be identified.

[0033] For example, assuming a preset quantity of 3, the goal is to identify the 3 time periods with the highest number of vehicle passages. Each "passage period" is defined as two hours. The number of vehicle passages within each passage period is counted. The three passage periods with the highest number of passages are then selected. For example, it might be found that vehicles pass most frequently between 7-8 AM, 12-1 PM, and 5-6 PM.

[0034] In one implementation, feature extraction is performed based on multimodal data to obtain the personnel features of the vehicle, including acquiring vehicle images from electronic checkpoint capture data; using a convolutional neural network model to extract driver and passenger facial features from the vehicle images; calculating the number of driver categories and the number of times each driver drives within a preset time period based on the driver facial features; and calculating the number of passenger categories within a preset time period based on the passenger facial features.

[0035] Specifically, the process begins by extracting vehicle personnel features from multimodal data sources, particularly vehicle images captured by electronic checkpoints. Using these images, a convolutional neural network (CNN) model is employed to extract facial features of drivers and passengers. Based on the extracted driver facial features, the number of different driver categories and the number of times each driver has driven are calculated within a preset time period. This is achieved by comparing the similarity of driver facial features captured at different time points, thus distinguishing different drivers. The similarity calculation uses the cosine similarity method to quantify the similarity between feature vectors. If the similarity exceeds a certain threshold, the two feature vectors are considered to represent the same driver.

[0036] Similarly, the number of passenger categories within a preset time period is calculated based on passenger facial features. This involves analyzing passenger facial features and grouping similar features into the same category based on similarity, thereby counting the number of different passengers.

[0037] like Figure 3The diagram illustrates a method for feature extraction and comparison of image information. First, vehicle image information is input, and then a convolutional neural network is used to extract image feature values. Next, the system calculates the cosine similarity between the extracted features and known features to determine if they match. If the similarity calculation result is "no," it indicates that the facial features belong to different people; if it is "yes," it indicates that the facial features belong to the same person. The system obtains feature information from the most recent n days, compares and classifies the similarity, and based on the similarity comparison results, further determines the number of driver and co-driver categories.

[0038] This method allows for the analysis of the number of drivers and passengers in a vehicle within a specific time period based on extracted vehicle image feature data, which helps in identifying illegally operating vehicles.

[0039] In one implementation, feature extraction is performed based on multimodal data to obtain the vehicle's qualification characteristics, including obtaining the number of complaints against the vehicle, the number of cases involving the vehicle, and extracting the vehicle's operation type based on the operational information in the multimodal data; and obtaining qualification characteristics based on the number of complaints, the number of cases, and the operation type.

[0040] Specifically, vehicle qualification characteristics are extracted by analyzing operational information from multimodal datasets. This begins with collecting relevant vehicle operational data, including the number of complaints against the vehicle, the number of cases the vehicle is involved in, and the vehicle's operational type (e.g., taxi, ride-hailing, bus). This data provides direct indicators of vehicle operational behavior.

[0041] Next, this operational information is used to construct the vehicle's qualification characteristics. Qualification characteristics are a quantitative representation of a vehicle's compliance and reputation, reflecting its performance and potential risk level during operation. For example, vehicles frequently involved in complaints or multiple cases may have a higher risk index, while the type of operation may affect the vehicle's regulatory requirements and compliance standards.

[0042] In at least one embodiment of this application, multi-dimensional feature extraction is performed based on multi-modal data obtained by fusing extracted multi-source data. By constructing a behavioral feature extraction method, privacy information in the original data is removed, and only data with the consent of the data source to be extracted and stored is retained. Under the premise of not storing original sensitive data (such as license plate number, face image, GPS details), only vehicle multi-modal feature data is retained, which takes into account both the accuracy of illegal operation identification and privacy compliance.

[0043] S103 inputs traffic characteristics, personnel characteristics, and qualification characteristics into a pre-trained violation detection model to obtain a judgment result on whether the vehicle is an illegal operating vehicle.

[0044] In one implementation, the method further includes first constructing a training dataset and training a violation detection model. This involves extracting traffic features, personnel features, and qualification features from multiple vehicles over historical time periods; adding labels to the extracted feature data to construct a training dataset; and training a violation detection model, which is a deep neural network, based on the training dataset.

[0045] Specifically, the process begins by extracting multiple vehicle traffic characteristics, personnel characteristics, and qualification characteristics from multimodal data over historical periods. Traffic characteristics may include the number of times and the time it takes for a vehicle to pass through a specific checkpoint, toll station, and ETC gantry; personnel characteristics involve the facial features of drivers and passengers; and qualification characteristics cover vehicle operational information, such as the number of complaints, the number of cases, and the type of operation.

[0046] After extracting these features, the feature data is labeled to construct a training dataset. The labeling process involves matching features with known illegal or legal operational behaviors, thus providing the model with a learning target. With the labeled training dataset, a deep neural network is used to train the violation detection model. Deep neural networks are powerful machine learning models capable of learning complex patterns and relationships from large amounts of data. During training, the model continuously adjusts its internal parameters to minimize the difference between the predicted results and the actual labels, thereby improving the accuracy of violation detection.

[0047] Ultimately, the trained model can be used to detect vehicle operation behavior in real time or in batches, automatically identifying possible illegal operating vehicles.

[0048] like Figure 5 The diagram illustrates a process for training a violation detection model, illustrating the interaction between feature extraction, data labeling, model training, and raw data. The diagram shows four main parts: feature extraction, data labeling, the model, and the raw data. The feature extraction part is responsible for extracting different types of information features from the raw data, including qualification information features (dimension A), access information features (dimension B), personnel information features (dimension C), and time features (dimension D), and then passing the extracted feature data to the data labeling part. The data labeling part labels the extracted feature data and stores the labeled data. Next, the model part trains the model using the labeled dataset and stores the trained model.

[0049] Furthermore, the traffic characteristics, personnel characteristics, and qualification characteristics are input into the trained violation detection model to obtain the judgment result of whether the vehicle is an illegal operating vehicle.

[0050] The violation detection model includes a qualification feature processing module, which consists of a fully connected layer sequence that uses ReLU activation and Dropout for regularization to process qualification features.

[0051] The traffic feature processing module, including an embedding layer, a bidirectional LSTM module, and a fully connected layer, is used to process traffic features. The embedding layer converts the sequence data into vectors, then the bidirectional LSTM module captures dependencies in the time series, and finally, the fully connected layer further extracts features.

[0052] The personnel feature processing module includes a fully connected layer sequence, which uses ReLU activation function and Dropout for regularization to process personnel features.

[0053] It also includes a classification module, consisting of a sequence of fully connected layers, regularized using ReLU activation and Dropout, to output the violation probability. It integrates all extracted features and performs classification through a multi-layer fully connected network. This module first maps the input features to a high-dimensional space, then further processes them through two fully connected layers and the ReLU activation function, finally outputting a violation probability value representing the likelihood of a vehicle engaging in a violation through the Sigmoid activation function.

[0054] Ultimately, the model outputs a probability value indicating the likelihood of a vehicle violating traffic rules, providing decision support for traffic management and law enforcement. This multimodal learning approach integrates data from different sources, improving the accuracy and reliability of violation detection.

[0055] In one implementation, after obtaining the determination result of whether a vehicle is an illegally operating vehicle, the method further includes sending the determination result to the user terminal; receiving feedback information from the user terminal; and dynamically correcting the parameters of the violation detection model based on the feedback information.

[0056] In this embodiment, the violation detection model not only identifies whether a vehicle is suspected of illegal operation, but also has the ability to interact with law enforcement personnel's user terminals, forming a closed-loop feedback and correction mechanism. Once the model completes the determination of the vehicle's illegal operation behavior and sends the result to the user terminal, such as a traffic management department or law enforcement agency, the user terminal can verify and evaluate the model's determination result according to the actual situation.

[0057] After receiving the judgment result, the user can provide feedback, which may include confirmation, correction, or additional contextual information. This feedback is crucial for the continuous optimization of the model. The system will receive this feedback and use it as a basis for adjusting the parameters of the violation detection model. By analyzing user feedback, the model can learn new patterns or correct previous errors, thereby dynamically adjusting its internal parameters. For example, based on this feedback, it may be discovered that the model is biased in identifying certain types of vehicles. Therefore, it may be decided to adjust the model's loss function to place greater emphasis on reducing the occurrence of such errors. At the same time, the dropout rate may be increased to reduce overfitting, or the network structure may be adjusted, which may require increasing or decreasing the number of feature layers, or changing the number of neurons in each layer to better capture vehicle features.

[0058] This dynamic correction mechanism based on user feedback enables the violation detection model to adapt to constantly changing traffic environments and patterns of illegal operations, while also better meeting the needs of actual law enforcement. Through this continuous learning and adaptation, the model can more intelligently and efficiently assist traffic management and law enforcement, improving the overall effectiveness of detecting illegally operating vehicles.

[0059] like Figure 6 The diagram illustrates the correction method for the violation detection model. First, feature values ​​are extracted. Next, the system determines whether the extracted feature values ​​indicate illegal operation. If so, it is recorded. Then, the system checks if the user has provided feedback or corrections to the recognition results. If the user has provided feedback or corrections, the system will correct the deep learning model to improve recognition accuracy. If the user has not provided feedback or corrections, the process will end directly.

[0060] This application extracts traffic features, personnel features, and qualification features from multimodal data, including vehicle trajectory data, ETC passage data, electronic checkpoint capture data, operation platform data, and toll station transaction data, and inputs these features into a violation detection model for identification. It achieves efficient fusion and joint feature extraction of trajectory data and visual data. Compared to traditional methods that use a single data source, this method can capture vehicle behavior features more comprehensively and accurately, especially showing significant advantages in identifying illegally operated vehicles disguised as legitimate ones.

[0061] Secondly, this method uses deep neural networks for identification based solely on the extracted features, enabling the analysis of illegal operations without storing sensitive data, thus avoiding privacy exposure issues at the source.

[0062] In one embodiment of this application, an edge-cloud collaborative software architecture is adopted to implement this solution. For example, vehicle trajectory data, ETC passage data, electronic checkpoint capture data, operation platform data, and toll station transaction data are collected at the edge. Data collected from various sensing data sources is received through a unified interface (such as REST API, Kafka message queue, database sharing, FTP / SFTP file reading). After fusing the time and location information of the multi-source data, multimodal data is obtained. A convolutional neural network deployed at the edge is used to extract features from the vehicle image data stored locally to obtain personnel feature data. The extracted personnel data, passage features, and qualification features are input into a violation detection model located in the cloud for the identification of illegally operating vehicles, achieving real-time detection at the second level. Deep learning and automatic decision-making mechanisms reduce human intervention and form an end-to-end closed-loop response. In terms of cross-departmental collaboration, this solution relies on deep learning and behavioral feature extraction to support joint modeling by multiple parties such as transportation, public security, and operators, breaking down data barriers while ensuring that the original data is "usable but not visible."

[0063] This solution supports modular architecture design, allowing for flexible adjustment of algorithm parameters and analysis dimensions based on illegal operation governance strategies. It enables adaptive business expansion and dynamically updates judgment parameters through corrections during use.

[0064] According to another aspect of the embodiments of this application, a detection device for illegally operating vehicles is also provided for implementing the above-described detection method for illegally operating vehicles. For example... Figure 7 As shown, the device includes: The data fusion module 701 is used to extract vehicle trajectory data, ETC passage data, electronic checkpoint capture data, operation platform data and toll station transaction data to obtain multimodal vehicle data. The feature extraction module 702 is used to extract features based on multimodal data to obtain vehicle traffic features, personnel features, and qualification features; The violation detection module 703 is used to input traffic characteristics, personnel characteristics, and qualification characteristics into a pre-trained violation detection model to obtain a judgment result on whether a vehicle is an illegal operating vehicle.

[0065] It should be noted that the above embodiments of the illegal vehicle detection device are only illustrated by the division of the above functional modules when executing the illegal vehicle detection method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the illegal vehicle detection device and the illegal vehicle detection method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0066] According to another aspect of the present application, an electronic device corresponding to the detection method for illegally operating vehicles provided in the foregoing embodiments is also provided, for performing the above-described detection method for illegally operating vehicles.

[0067] Please refer to Figure 8 This illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 8 As shown, the electronic device includes: a processor 800, a memory 801, a bus 802, and a communication interface 803. The processor 800, the communication interface 803, and the memory 801 are connected via the bus 802. The memory 801 stores a computer program that can run on the processor 800. When the processor 800 runs the computer program, it executes the method for detecting illegally operating vehicles provided in any of the foregoing embodiments of this application.

[0068] The memory 801 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 803 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0069] Bus 802 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. The memory 801 is used to store programs. After receiving execution instructions, the processor 800 executes the program. The method for detecting illegally operating vehicles disclosed in any of the aforementioned embodiments of this application can be applied to the processor 800, or implemented by the processor 800.

[0070] The processor 800 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 800 or by instructions in software form. The processor 800 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 801. Processor 800 reads the information in memory 801 and, in conjunction with its hardware, completes the steps of the above method.

[0071] The electronic device provided in this application embodiment and the method for detecting illegally operating vehicles provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0072] According to another aspect of the embodiments of this application, a computer-readable storage medium corresponding to the detection method for illegally operating vehicles provided in the foregoing embodiments is also provided, wherein a computer program (i.e., a program product) is stored thereon, and when the computer program is run by a processor, it executes the detection method for illegally operating vehicles provided in any of the foregoing embodiments.

[0073] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0074] The computer-readable storage medium provided in the above embodiments of this application and the method for detecting illegally operating vehicles provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application stored therein.

[0075] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0076] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

Claims

1. A method for detecting illegally operating vehicles, characterized in that, include: Extract vehicle trajectory data, ETC passage data, electronic checkpoint capture data, operation platform data, and toll station transaction data to obtain multimodal vehicle data; Based on the multimodal data, feature extraction is performed to obtain vehicle traffic features, personnel features, and qualification features; The traffic characteristics, personnel characteristics, and qualification characteristics are input into a pre-trained violation detection model to obtain a judgment result on whether the vehicle is an illegal operating vehicle.

2. The method according to claim 1, characterized in that, The vehicle multimodal data includes a sequence of vehicle location data, and also includes: Extract the vehicle location data sequence from the multimodal data. Determine whether a road exists within a preset range of each vehicle's location; If there are no roads within a preset range near the vehicle location data, discard that location data. If there is a road within a preset range near the vehicle's location data, the road closest to the vehicle's location will be taken as the road to which the vehicle belongs.

3. The method according to claim 1, characterized in that, The traffic characteristics include a preset number of target checkpoints with the most traffic, a preset number of target toll stations with the most traffic, a preset number of target ETC gantries with the most traffic, and a preset number of target roads with the most traffic. The feature extraction based on the multimodal data to obtain vehicle traffic features includes: Based on the multimodal data, the number of times a vehicle passes through each checkpoint is calculated, and a preset number of target checkpoints with the most passage times are obtained; Based on the multimodal data, the number of times a vehicle passes through each toll station is calculated, and a preset number of target toll stations with the most passage times are obtained; Based on the multimodal data, the number of times a vehicle passes through each ETC gantry is calculated, and a preset number of target ETC gantries with the most passage times are obtained; Based on the target checkpoint, target toll station, and target ETC gantry, query the preset number of target roads with the highest number of vehicle passages.

4. The method according to claim 1, characterized in that, Personnel characteristics include the number of driver categories within a preset time period and the number of times each driver drives. The feature extraction based on the multimodal data to obtain vehicle personnel characteristics includes: Obtain vehicle images from the electronic checkpoint capture data and cache the vehicle images on a local server; On a local server, a convolutional neural network model is used to extract the driver's facial features and the passenger's facial features from the vehicle images; Based on the driver's facial features, calculate the number of driver categories and the number of times each driver drives within a preset time period; The number of passenger categories within a preset time period is calculated based on the passenger's facial features.

5. The method according to claim 1, characterized in that, Also includes: Extract the traffic characteristics, personnel characteristics, and qualification characteristics of multiple vehicles from historical time periods; Label the extracted feature data to construct a training dataset; The violation detection model is trained based on the training dataset to obtain the violation detection model, which is a deep neural network.

6. The method according to any one of claims 1-5, characterized in that, The violation detection model includes: The qualification feature processing module includes a fully connected layer sequence, which is regularized using the ReLU activation function and Dropout to process qualification features; The access feature processing module includes an embedding layer, a bidirectional LSTM module, and a fully connected layer, which are used to process access features. The personnel feature processing module includes a fully connected layer sequence, which uses ReLU activation and Dropout for regularization to process personnel features. The classification module, consisting of fully connected layer sequences, is regularized using the ReLU activation function and Dropout to output the probability of violation.

7. The method according to claim 1, characterized in that, After obtaining the result of determining whether a vehicle is an illegally operating vehicle, the following also includes: The determination result is sent to the user terminal; Receive feedback information from the user terminal; The parameters of the violation detection model are dynamically adjusted based on the feedback information.

8. A detection device for illegally operating vehicles, characterized in that, include: The data fusion module is used to extract vehicle trajectory data, ETC passage data, electronic checkpoint capture data, operation platform data, and toll station transaction data to obtain multimodal vehicle data. The feature extraction module is used to extract features based on the multimodal data to obtain vehicle traffic features, personnel features, and qualification features; The violation detection module is used to input the traffic characteristics, personnel characteristics, and qualification characteristics into a pre-trained violation detection model to obtain a judgment result on whether the vehicle is an illegal operating vehicle.

9. An electronic device, characterized in that, It includes a processor and a memory storing program instructions, the processor being configured to execute, when executing the program instructions, the method for detecting illegally operating vehicles as described in any one of claims 1 to 7.

10. A computer-readable medium, characterized in that, It stores computer-readable instructions that are executed by a processor to implement the method for detecting illegally operating vehicles as described in any one of claims 1 to 7.

Images