A license plate detection method, device and electronic equipment

Abstract

The application relates to the technical field of vehicles. In order to solve the problem of how to identify whether a vehicle on a road has a license plate anomaly, the application provides a license plate detection method, a license plate detection device and electronic equipment. The method of the application comprises the following steps: acquiring vehicle on-road monitoring data and vehicle registration data, wherein the vehicle on-road monitoring data at least comprises video monitoring data of an intersection; identifying vehicle appearance features, license plate text features and vehicle attribute features of the vehicle on the road according to the vehicle on-road monitoring data; and performing license plate detection according to the vehicle appearance features, the license plate text features and the vehicle attribute features in combination with the vehicle registration data to determine whether there is a license plate anomaly. According to the method of the application, the hardware equipment requirement for license plate detection can be reduced, the difficulty of license plate detection implementation can be reduced, the coverage range of license plate detection can be improved, and the accuracy of license plate anomaly detection can be improved.

Description

Technical Field

[0001] This application relates to the field of vehicles, and in particular to a method, apparatus and electronic device for license plate detection. Background Technology

[0002] License plates, also known as vehicle registration plates, are plates affixed to the front and rear of vehicles. They are typically made of aluminum, sheet metal, plastic, or paper. The license plate contains the vehicle's registration number, registration region, and other relevant information. The license plate serves as a unique identifier and registration document for the vehicle; its primary function is to identify the vehicle's location and to locate the owner and registration details.

[0003] License plates are like a vehicle's identity card. To prevent vehicles from using counterfeit license plates or impersonating others on the road, a license plate detection method is needed to identify whether a vehicle on the road has an abnormal license plate. Summary of the Invention

[0004] To address the issue of how to identify whether a vehicle on the road has an abnormal license plate, this application provides a license plate detection method, device, and electronic device. This application also provides a computer program product and a computer-readable storage medium.

[0005] The embodiments of this application adopt the following technical solutions: In a first aspect, this application provides a license plate detection method, which is applied to an electronic device and includes: Acquire on-road vehicle monitoring data and vehicle registration data. On-road vehicle monitoring data shall include at least video surveillance data from intersections. Based on on-road vehicle monitoring data, identify the vehicle appearance features, license plate text features, and vehicle attribute features of on-road vehicles; Based on vehicle appearance features, license plate text features, and vehicle attribute features, combined with vehicle registration data, license plate detection is performed to determine whether there are any license plate anomalies.

[0006] According to the first approach, using intersection surveillance video for license plate anomaly detection can reduce the hardware requirements for license plate detection, lower the difficulty of implementing license plate detection, and increase the coverage of license plate detection.

[0007] According to the method in the first aspect, license plate detection based on the vehicle appearance features, license plate text features, and vehicle attribute features of vehicles on the road can improve the accuracy of license plate anomaly detection.

[0008] In one implementation of the first aspect, based on on-road vehicle monitoring data, the vehicle's appearance features, license plate text features, and vehicle attribute features are identified, including: Perform video frame extraction on video surveillance data to obtain the extracted vehicle images; The extracted vehicle images are cropped to obtain the regions of interest related to the vehicles on the road; Perform feature recognition on the region of interest to obtain vehicle appearance features and vehicle attribute features; Optical character recognition is performed on the region of interest to obtain the text features of the license plate.

[0009] In one implementation of the first aspect, the method further includes: The first-stage annotation results are obtained by annotating based on vehicle appearance features, license plate text features, and vehicle attribute features. The first-stage annotation results were reviewed, and vehicle sample data was obtained.

[0010] In one implementation of the first aspect, the annotation results of the first stage are reviewed, and vehicle sample data is obtained, including: The first-stage labeling results were reviewed based on the anonymized real-name information to obtain vehicle sample data.

[0011] In one implementation of the first aspect, license plate detection is performed based on vehicle appearance features, license plate text features, and vehicle attribute features, combined with vehicle registration data, to determine whether there are any license plate anomalies, including: Generate vehicle appearance feature vector, license plate text feature vector, and vehicle attribute feature vector based on vehicle appearance features, license plate text features, and vehicle attribute features, respectively. Based on the vehicle appearance feature vector, license plate text feature vector, and vehicle attribute feature vector, determine whether there is an abnormal license plate.

[0012] In one implementation of the first aspect, determining whether there is a license plate anomaly based on the vehicle appearance feature vector, the license plate text feature vector, and the vehicle attribute feature vector includes: The vehicle appearance feature vector, license plate text feature vector, and vehicle attribute feature vector are jointly fused and encoded across modalities to generate a unique vehicle identity representation. Based on the vehicle's unique identification characteristics, determine whether there is any abnormality in the license plate.

[0013] In one implementation of the first aspect, determining whether there is a license plate anomaly based on the vehicle appearance feature vector, the license plate text feature vector, and the vehicle attribute feature vector includes: The de-identified real-name information is jointly encoded with the license plate text feature vector to generate a license plate identity fusion vector. Based on the vehicle appearance feature vector, license plate identity fusion vector, and vehicle attribute feature vector, determine whether there is an abnormal license plate.

[0014] In one implementation of the first aspect, determining whether there is an abnormal license plate based on the vehicle's unique identifier includes: This system utilizes vector retrieval libraries, relational databases, and graph databases to detect license plate anomalies and determine the presence of such anomalies. The vector retrieval library is used to store unique vehicle identification information; Relational databases are used to store structured data from vehicle appearance features, license plate identity fusion, and vehicle attribute features; Graph databases are used to store the spatiotemporal relationships between vehicles on the road, their locations, and the times they are located.

[0015] Secondly, this application provides an electronic device, which includes a memory and a processor; The processor executes instructions stored in memory to cause the electronic device to perform the method as described in the first aspect.

[0016] Thirdly, this application provides a computer program product containing instructions that, when executed by a computing device system, cause a cluster of computing devices to perform the method as described in the first aspect.

[0017] Fourthly, this application provides a computer-readable storage medium including computer program instructions, which, when executed by a computer system, cause the computer system to perform the method as described in the first aspect. Attached Figure Description

[0018] Figure 1 The diagram shown is a schematic diagram of an electronic device structure according to an embodiment of this application; Figure 2 The diagram shown is a schematic diagram of a license plate detection device according to an embodiment of this application; Figure 3 The diagram shown is a schematic flowchart of a license plate detection method according to an embodiment of this application. Figure 4 The diagram shown is a schematic diagram of a knowledge graph structure according to an embodiment of this application; Figure 5 The diagram shown is a schematic diagram of a license plate anomaly detection logic according to an embodiment of this application; Figure 6 The diagram shown is a logical architecture diagram of a license plate detection system according to an embodiment of this application; Figure 7 The diagram shown is a schematic diagram of a license plate detection process according to an embodiment of this application; Figure 8 This is a schematic diagram of an electronic device structure according to an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] The terminology used in the implementation section of this application is for the purpose of explaining specific embodiments of this application only, and is not intended to limit this application.

[0021] To address the issue of identifying license plate anomalies on vehicles on the road, one feasible technical solution is to capture license plate images with a camera, use Optical Character Recognition (OCR) to identify the license plate number, and then compare it with the vehicle registration database to determine if the license plate number exists. However, this solution only relies on the license plate text to confirm the validity of the license plate content and cannot identify whether the vehicle displaying the license plate matches the registered vehicle.

[0022] To address the issue of identifying license plate anomalies in vehicles on the road, another feasible technical solution is to track vehicle trajectories within the road network system and identify instances where the same license plate appears simultaneously in different locations. For example, in highway networks, this involves relying on hardware such as Electronic Toll Collection (ETC) gantries, Radio Frequency Identification (RFID) electronic license plates, and dedicated checkpoint cameras to collaboratively determine vehicle trajectories. However, this solution is costly to deploy and only covers closed road networks in specific scenarios like highways and toll stations, failing to cover the broader landscape of urban intersections, residential areas, and industrial parks.

[0023] To address the above situation, one embodiment of this application provides a license plate detection method. In this method, for license plates of vehicles on the road, not only is the text content of the license plate identified, but also the vehicle's appearance information and vehicle attribute information; the method then compares this information with the appearance and attribute information of registered vehicles to confirm whether the vehicle is using another vehicle's license plate.

[0024] The method described in this application is applied to electronic devices. This application does not impose specific limitations on the electronic devices to which the method is applied. For example, the electronic device may be a mobile phone, tablet computer, personal digital assistant (PDA), desktop computer, laptop computer, notebook computer, ultra-mobile personal computer (UMPC), handheld computer, netbook, etc. This application does not impose any special limitations on the specific form of the aforementioned electronic devices.

[0025] Figure 1 The diagram shown is a schematic diagram of an electronic device structure according to an embodiment of this application.

[0026] The method provided in any embodiment of this application can be applied to... Figure 1 In the electronic device 100 shown.

[0027] Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, interface 130, power management module 141, antenna 1, communication module 150, audio module 170, sensor module 180, camera 193, display screen 194, etc.

[0028] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0029] The processor 110 may be an on-chip device (SOC) or other architecture. The processor 110 may include a central processing unit (CPU) and may further include other types of processors.

[0030] Processor 110 may include one or more processing units. For example, the processing units of processor 110 may include any combination of one or more of the following: Central Processing Unit (CPU), Digital Signal Processor (DSP), Microcontroller Unit (MCU), Digital Signal Processor (DSP), Application Processor (AP), Graphics Processing Unit (GPU), Neural-network Processing Units (NPU), Image Signal Processing (ISP), Modem Processor, Controller, Video Codec, and Baseband Processor. Processing units of processor 110 may also include other processing units besides those described above.

[0031] In processor 110, different processing units can be independent devices or integrated into one or more processors. The controller can generate operation control signals based on the instruction opcode and timing signals to control instruction fetching and execution.

[0032] The processor may also include necessary hardware accelerators or logic processing hardware circuitry, such as an ASIC, or one or more integrated circuits for controlling the execution of the program in this application. Furthermore, the processor may have the capability to operate one or more software programs, which may be stored in a storage medium.

[0033] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0034] In some embodiments, processor 110 may include one or more interfaces.

[0035] The interfaces of processor 110 may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0036] Interface 130 is used to provide an interface for external access of electronic device 100 to processor 110.

[0037] In one embodiment, interface 130 is an interface of processor 110. In another embodiment, interface 130 includes an interface conversion module for converting one type of interface to another. One end of the interface conversion module provides external access for electronic device 100, and the other end of the interface conversion module is connected to the interface of processor 110.

[0038] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0039] Internal memory 121 can be used to store computer executable program code, which includes instructions.

[0040] The internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function (such as sound playback, image playback, etc.). The data storage area may store data created during the use of the electronic device 100.

[0041] Internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of electronic device 100 by running instructions stored in internal memory 121 and / or instructions stored in memory disposed in the processor.

[0042] The internal memory 121 may be a read-only memory (ROM), other types of static storage devices that can store static information and instructions, random access memory (RAM), or other types of dynamic storage devices that can store information and instructions. It may also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), a magnetic disk storage medium, or other magnetic storage devices. Alternatively, it may be any computer-readable medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer.

[0043] Processor 110 and internal memory 121 can be combined into a single processing device, or more commonly they are separate components.

[0044] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0045] The power management module 141 manages the power supply to various components of the electronic device 100. In one embodiment, the power management module 141 includes a charging management module and a battery. The charging management module receives charging input from a charger.

[0046] The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may also be located in the same device.

[0047] The wireless communication function of electronic device 100 can be realized through antenna 1, communication module 150, etc.

[0048] Antenna 1 is used to transmit and receive electromagnetic wave signals. Antenna 1 may include one or more physical antennas. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization.

[0049] The communication module 150 may be one or more devices integrating at least one communication processing module. The communication module 150 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for use on the electronic device 100. The communication module 150 can also provide wireless communication solutions, including wireless local area networks (WLANs) (such as Wireless Fidelity (Wi-Fi) networks), Bluetooth (BT), Global Navigation Satellite System (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies, for use on the electronic device 100.

[0050] The display screen 194 is used to display images, videos, etc. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0051] Camera 193 is used to capture still images or videos. In some embodiments, electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0052] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0053] In some embodiments, the electronic device 100 further includes a speaker, a microphone, etc. The electronic device 100 can implement audio functions through the audio module 170, the speaker, the microphone, and an application processor, such as music playback and voice control.

[0054] The sensor module 180 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc.

[0055] In some embodiments, the electronic device 100 also includes buttons, indicators, etc.

[0056] For example, buttons include vehicle start button, volume buttons, air conditioning control buttons, etc. Buttons can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0057] Indicators can be indicator lights, used to indicate vehicle status such as speed, charging status, and battery level; they can also be used to indicate messages such as fault alarms and low battery alarms.

[0058] In addition to the aforementioned components, the electronic device runs an operating system. For example, iOS. ® Operating system, Android ® Operating system, Windows ® Operating systems, such as vehicle operating systems, can be used to install and run applications. The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture.

[0059] Furthermore, in one embodiment, the method of this application embodiment can be applied to an electronic device. This electronic device may refer in whole or in part to electronic device 100.

[0060] In another embodiment, the method of this application can be applied to a system composed of multiple electronic devices, where each electronic device in the system executes a portion of the method steps. Any electronic device in the system can refer to electronic device 100 in whole or in part.

[0061] In order to implement the license plate detection method proposed in the embodiments of this application, an embodiment of this application also proposes a license plate detection device.

[0062] Figure 2 The diagram shown is a schematic diagram of a license plate detection device according to an embodiment of this application.

[0063] like Figure 2 As shown, the license plate detection device 200 includes a data acquisition module 201, a recognition module 202, and a detection module 203.

[0064] The data acquisition module 201 is used to acquire on-road vehicle monitoring data and vehicle registration data.

[0065] The identification module 202 is used to process the on-road vehicle monitoring data acquired by the data acquisition module 201, and to identify the vehicle appearance features, license plate text features and vehicle attribute features of the on-road vehicles.

[0066] Specifically, vehicle appearance features are used to describe the appearance of the vehicle. License plate text features are used to describe the content of the license plate text. Vehicle attribute features are used to describe the basic inherent attributes of the vehicle. For example, in one embodiment, vehicle appearance features include: overall vehicle appearance (body color, body outline), front appearance (grille, headlights, etc.), side appearance (windows, sunroof, etc.), rear appearance (taillights, exhaust, etc.), and added / modified visible features (rear spoiler, roof rack, etc.); vehicle attribute features include: vehicle brand, model category, etc.

[0067] The detection module 203 is used to detect license plates based on the recognition results output by the recognition module 202 and the vehicle registration data acquired by the data acquisition module 201, and to determine whether there are any license plate anomalies.

[0068] Specifically, in one embodiment, license plate anomalies include cloned plates, fake plates, and plates that have been swapped.

[0069] The license plate detection device of this application embodiment is applied to an electronic device, which can refer to electronic device 100.

[0070] This application does not limit the specific method of implementing the license plate detection device. Those skilled in the art can design the implementation method of the device according to the actual situation.

[0071] For example, in one embodiment, the device is installed on the electronic device in hardware (e.g., a functional chip). In another embodiment, the device is installed in the operating system of the electronic device as software code. Yet another embodiment, the device is installed on the electronic device as a combination of hardware and software.

[0072] Furthermore, in one embodiment, the device is mounted on an electronic device.

[0073] In another embodiment, the device is mounted on multiple electronic devices, which together form a complete device structure. Different electronic devices may install the same functional modules, or different electronic devices may install different functional modules.

[0074] In the description of the embodiments of this application, for the sake of convenience, the device is described by dividing it into various modules according to its functions. The division of each module is only a logical functional division. When implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware.

[0075] Specifically, the apparatus proposed in this application can be fully or partially integrated onto a single physical entity (e.g., a GPU or other type of processor), or it can be physically separated. These modules can be implemented entirely in software via processing element calls; entirely in hardware; or some modules can be implemented in software via processing element calls, while others are implemented in hardware. For example, the detection module can be a separate processing element or integrated into a chip in an electronic device. The implementation of other modules is similar. Furthermore, these modules can be fully or partially integrated together or implemented independently. During implementation, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0076] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs). Alternatively, these modules can be integrated together as a system-on-a-chip (SOC).

[0077] Specifically, in one embodiment, the instruction calculation module 202 is implemented in the vehicle's controller (e.g., an electronic control unit (ECU)).

[0078] Figure 3 The diagram shown is a schematic flowchart of a license plate detection method according to an embodiment of this application.

[0079] In one embodiment, the electronic device performs as follows Figure 3 The following process is shown to achieve license plate detection.

[0080] S300 acquires on-road vehicle monitoring data and vehicle registration data.

[0081] In one embodiment, the electronic device is equipped with Figure 2 The license plate detection device 200 shown is executed by the data acquisition module 201 in step S300.

[0082] In one embodiment, the on-road vehicle monitoring data acquired by S300 includes at least video surveillance data of vehicles driving on the road. The vehicle registration data includes at least the motor vehicle registration data.

[0083] The vehicle registration data includes at least the text of the license plate and a description of the vehicle associated with that license plate.

[0084] Video surveillance data of vehicles on the road must include at least the vehicle image with the license plate.

[0085] To expand the coverage of license plate detection, in one embodiment, the video surveillance data of vehicles driving on the road includes video surveillance data from intersection monitoring.

[0086] According to the method in the embodiments of this application, using intersection surveillance video to conduct license plate anomaly detection can reduce the hardware requirements for license plate detection, reduce the difficulty of implementing license plate detection, and increase the coverage of license plate detection.

[0087] Furthermore, considering that there may be gaps in equipment distribution and / or missing images in urban intersection monitoring, in one embodiment, the on-road vehicle monitoring data acquired by S300 also includes other vehicle driving monitoring data, such as images of checkpoint / electronic police vehicles, vehicle passing records from roadside sensing devices, etc.

[0088] The method according to the embodiments of this application makes full use of the vehicle-to-everything intersection monitoring video stream, without relying on dedicated hardware, and can break through scene limitations to achieve full coverage of license plate detection in urban roads, parks, residential areas, and checkpoints.

[0089] Furthermore, considering that the vehicle registration data may contain insufficient vehicle description information, in one embodiment, the vehicle registration data acquired by S300 also includes basic vehicle file data. Specifically, in one embodiment, the basic vehicle file data includes detailed descriptions of the vehicle's appearance and attributes.

[0090] Specifically, in one embodiment, the data acquisition module 201 adopts a standardized multi-source data interface design to access urban intersection surveillance video streams, checkpoint / electronic police vehicle images, roadside sensing device vehicle passage records, vehicle basic file data, and motor vehicle registration data in real time; it completes heterogeneous data normalization processing through unified data protocols and format standardization, and achieves accurate time synchronization of multi-source data by incorporating a timestamp alignment mechanism; it adopts a streaming access + incremental consumption architecture to ensure real-time transmission and low-latency storage of video streams and vehicle passage data; at the same time, it performs data anonymization and access control on vehicle owner privacy, license plate sensitive fields, and identity information, and builds a full-dimensional secure data foundation required for license plate counterfeiting detection under the premise of compliance.

[0091] According to the method of this application embodiment, based on the vehicle-to-everything (V2X) interconnected perception architecture, license plate anomaly detection is carried out using the full range of ordinary intersection surveillance videos in the city. It does not require the construction mode of relying on dedicated high-definition checkpoints, ETC, RFID and other exclusive hardware. It directly reuses the networked ordinary intersection surveillance resources in the city, covering the entire scenario of main and secondary roads, branch roads, parks and communities. It can eliminate the monitoring blind spots of fixed checkpoints and realize seamless license plate supervision throughout the city.

[0092] In scenarios where it is necessary to determine whether a vehicle on the road is a registered vehicle with a license plate, the image features are weak in their expressive power. Comparisons based on single image features (e.g., by extracting shallow features such as vehicle color, brand, and model for comparison) cannot distinguish between similar models or vehicles of the same color and model, resulting in low accuracy in identifying counterfeit license plates.

[0093] Therefore, S301 is executed after S300.

[0094] S301 identifies the vehicle appearance features, license plate text features, and vehicle attribute features of vehicles on the road based on on-road vehicle monitoring data.

[0095] In one embodiment, the electronic device is equipped with Figure 2 The license plate detection device 200 shown is executed by the recognition module 202 in step S301.

[0096] The method according to the embodiments of this application detects license plates based on the vehicle appearance features, license plate text features, and vehicle attribute features of vehicles on the road, which can improve the accuracy of license plate anomaly detection.

[0097] In one embodiment, in S301, video frames are extracted from the video surveillance data of the vehicle driving on the road to obtain a framed vehicle image containing the license plate. Image recognition is then performed on the framed vehicle image to obtain vehicle appearance features, license plate text features, and vehicle attribute features.

[0098] In one embodiment, a differentiated frame-dropping mode that adaptively matches the scene state of the intersection is adopted for video surveillance data of urban intersections. Specifically, a fixed-interval low-frequency frame-dropping mode (e.g., once every 1 second) is used for regular intersections with smooth traffic flow and simple road conditions; for key intersections with dense traffic flow, complex intersections, and frequent illegal parking and lane changes, as well as complex environmental scenes such as nighttime, rain, fog, and backlighting, high-density continuous frame-dropping mode (e.g., once every 0.2 seconds) is enabled.

[0099] In one embodiment, in order to balance computational overhead and target capture integrity, and improve subsequent recognition efficiency and accuracy, the vehicle images are cropped before image recognition is performed on the extracted vehicle images to obtain image regions (Regions of Interest, ROIs) related to the vehicles, and background interference such as pedestrians, green plants, and buildings is filtered out.

[0100] Specifically, in one embodiment, a video parsing model (such as YOLOv8) is used to crop the extracted vehicle images.

[0101] Furthermore, in order to perform license plate detection based on the spatiotemporal attributes of the vehicle, in one embodiment, the vehicle is located while the extracted vehicle image is being cropped, and the vehicle coordinate region is output.

[0102] In one embodiment, during image recognition of the extracted vehicle images, vehicle features are identified to obtain vehicle appearance features and vehicle attribute features. Furthermore, during image recognition of the extracted vehicle images, license plate features are also identified to obtain license plate text features.

[0103] In one embodiment, after obtaining the vehicle feature recognition result and the license plate feature recognition result, the vehicle is further labeled based on the vehicle appearance feature, the license plate text feature and the vehicle attribute feature to obtain vehicle sample data.

[0104] Specifically, in one embodiment, a two-stage annotation is employed.

[0105] The first stage involves annotation based on vehicle appearance features, license plate text features, and vehicle attribute features to obtain the first stage annotation results. For example, it completes the annotation of multi-dimensional attributes such as license plate, vehicle location, brand, model, color, body damage, and appearance condition.

[0106] The second stage reviews the annotation results from the first stage, corrects errors, and adds annotations to problematic samples to form an iterative data loop, ensuring that the annotation accuracy rate is no less than 99%, and providing high-quality sample support for subsequent vehicle feature extraction and accurate detection of counterfeit license plates.

[0107] Furthermore, in one embodiment, during the second-stage review process, anonymized real-name information (such as the vehicle owner's surname, vehicle registration brand, registration color, and model year derived from the VIN code) is imported to assist in determining whether the captured vehicle matches the registration information, improving the accuracy of labeling, and reserving strong-association tags for subsequent vehicle identity verification. During the second-stage review process, the labeling results from the first stage are reviewed based on the anonymized real-name information to obtain vehicle sample data.

[0108] According to the method in this application, a two-stage autonomous annotation system is adopted. The first stage of automated annotation fully utilizes vehicle attribute information (vehicle brand, model, color, body damage, appearance condition, etc.) to complete multi-dimensional attributes and generate labels, rather than relying solely on a single license plate or appearance. The second stage involves review, error correction, and supplementary annotation of difficult samples, and introduces de-identified real-name information to assist in verification, forming a closed loop of data-annotation-training-inference-iteration. The two-stage autonomous annotation system does not rely on third-party datasets and can continuously optimize the model's generalization ability.

[0109] According to the method in the embodiments of this application, a two-stage annotation is adopted to construct an autonomous intelligent annotation system that is adapted to complex monitoring of ordinary urban intersections and is oriented towards multimodal vector fusion. This is different from the simple vehicle passage annotation mode of highway audit and effectively improves the accuracy and precision of the annotation data.

[0110] The method according to the embodiments of this application can achieve high-quality annotation of vehicle appearance, attributes, and status in multiple dimensions, forming a high-quality vehicle dataset that can be iterated autonomously.

[0111] In one embodiment, during the image recognition process of the extracted vehicle images, the cropped vehicle ROI region is used to perform license plate recognition using an optical character recognition model (based on secondary lightweighting and feature optimization of the PaddleOCR architecture) customized and optimized for the intersection scene. Through small target enhancement, distortion correction, environmental noise reduction and adaptation to complex intersection environments such as backlight, night, rain, tilt, dirt, and blur, the license plate text features are stably obtained.

[0112] Specifically, in one embodiment, the specific implementation of secondary lightweighting and feature optimization based on the PaddleOCR architecture includes: In terms of secondary lightweighting, a structured secondary lightweighting transformation is implemented on top of the PaddleOCR basic model architecture. Structured channel pruning and network layer simplification are performed on the feature extraction backbone network to remove redundant convolutional channels and invalid network branches in general text recognition tasks, while retaining the core feature pathways adapted to license plate character recognition. At the same time, the conventional standard convolution operator is replaced with a depthwise separable convolution operator, and operator fusion and 8-bit integer quantization fixed-point compression are implemented on the inference computation link. While maintaining the accuracy of license plate recognition without degradation, the number of model parameters and computation is compressed, reducing the single-frame inference latency, and adapting to application scenarios such as concurrent multi-channel video streams at urban intersections and real-time deployment of low-computing-power edge devices.

[0113] In terms of feature optimization, model-specific feature optimization is carried out for complex intersection monitoring conditions. A license plate-specific feature enhancement sub-branch is added to the shallow feature layer of PaddleOCR to enhance the inherent features of the license plate outline, character stroke texture, and background color, while suppressing invalid feature responses such as vehicle paint reflection, background interference, and window decoration. At the same time, a small target feature upsampling enhancement and perspective distortion adaptive feature correction mechanism are embedded, enabling all models to autonomously learn and adapt to the feature distribution of real intersection scenes such as backlight overexposure, low light at night, rain and fog, license plate tilt distortion, and surface dirt blur. Robust recognition is achieved by relying on the model's endogenous feature expression capabilities, without relying on additional pre-processing traditional image rules, and the standardized license plate text information is stably output.

[0114] According to the method in the embodiments of this application, based on a combination of video analysis model and optical character recognition model, robustness in complex environments can be improved. The software algorithm capability can make up for the inherent deficiencies of ordinary surveillance shooting angle, lighting and image quality, and solve the problem of low accuracy of vehicle detection and license plate recognition in intersection scenarios such as backlight, night, blur, occlusion and tilt.

[0115] S302, based on the vehicle's appearance features, license plate text features, and vehicle attribute features, combined with vehicle registration data, performs license plate detection to determine whether there are any license plate anomalies.

[0116] In one embodiment, the electronic device is equipped with Figure 2 The license plate detection device 200 shown is executed by the detection module 203 in step S301.

[0117] In one embodiment, in S302, three types of high-dimensional feature vectors are generated based on vehicle appearance features, license plate text features, and vehicle attribute features: 1. Vehicle appearance feature vector (generated by a deep feature extraction network through global feature encoding of vehicle appearance features, without relying on manually defined local component features); 2. License plate text feature vector (generated by vectorizing the identified license plate character information using a text encoding model); 3. Vehicle attribute feature vector (generated by structured encoding of basic inherent attributes such as vehicle brand, model category, and body color).

[0118] In S302, the vehicle appearance feature vector, license plate text feature vector, and vehicle attribute feature vector are jointly fused and encoded across modalities to form a unified high-dimensional feature representation (unique vehicle identity representation).

[0119] Specifically, in one embodiment, the three heterogeneous modal features—vehicle appearance feature vector, license plate text feature vector, and vehicle attribute feature vector—are first processed through a modality-specific mapping network to achieve dimensional alignment and feature normalization, eliminating feature space differences between image visual modality, text semantic modality, and structured attribute modality. Then, an attention-weighted fusion mechanism is used to perform cross-modal association encoding on the three types of vectors, adaptively allocating the weights of each modality feature in the vehicle identity representation, strengthening highly discriminative and effective features, and weakening redundant and irrelevant features. After multi-layer feature interaction fusion, a global high-dimensional fusion feature vector with fixed dimensions and unified semantics is generated to construct a unique vehicle identity representation.

[0120] According to the method in the embodiments of this application, based on the unique identity representation of a vehicle, deep semantic association and information complementarity of different modal features can be achieved. It can accurately depict the subtle feature differences between similar vehicles of the same model and color, highly simulated cloned vehicles, and vehicles with multiple license plates, providing a reliable unified feature basis for subsequent similarity measurement, spatiotemporal trajectory verification, and intelligent judgment of violation types.

[0121] According to the method in the embodiments of this application, the vehicle appearance, license plate text, and vehicle attributes are uniformly vectorized, and a multimodal vector fusion detection mechanism is constructed to realize joint judgment of multi-dimensional features, which greatly improves the accuracy of license plate anomaly recognition.

[0122] Furthermore, in one embodiment, the desensitized real-name information (such as registered color, brand model, and the administrative region to which the vehicle owner belongs) is jointly encoded with the license plate text feature vector to generate an enhanced "license plate identity fusion vector." This vector, along with the vehicle appearance feature vector and the vehicle attribute feature vector, participates in the similarity determination. The method according to the embodiments of this application can effectively solve the problem of identifying cloned vehicles that "use real license plates but have highly similar appearances," improving legal validity and business credibility.

[0123] In one embodiment, in S302, it is determined whether there are appearance conflicts and attribute discrepancies. Appearance conflicts include situations where the appearance of the vehicle is inconsistent with the appearance of the vehicle registered with the license plate, or multiple vehicles with inconsistent appearances for the same license plate. Attribute discrepancies include situations where the attributes of the vehicle are inconsistent with the attributes of the vehicle registered with the license plate, or multiple vehicles with inconsistent attributes for the same license plate.

[0124] Furthermore, in one embodiment, in S302, it is also determined whether there are multiple license plates for one vehicle and spatiotemporal conflicts. Multiple license plates for one vehicle includes multiple different license plates for the same vehicle. Spatiotemporal conflicts include the same license plate appearing in different locations simultaneously (or within a short time interval).

[0125] In one embodiment, in S302, a vector retrieval library (such as FAISS), a relational database (such as MySQL), and a graph database (such as Neo4j) are constructed. License plate anomaly detection is performed based on these libraries. The vector retrieval library stores the unique identity representation of each vehicle; the relational database stores structured data from vehicle appearance features, license plate identity fusion, and vehicle attribute features; and the graph database stores the spatiotemporal relationships between vehicles on the road, their locations, and location times.

[0126] Specifically, the unified high-dimensional feature vector (a unique vehicle identifier) ​​obtained through multimodal fusion is stored in a vector retrieval library (such as FAISS). Millisecond-level fast retrieval of similar vehicles is achieved through vector indexing, quantization compression, and batch retrieval optimization. Structured data such as license plate number, vehicle type, and color are stored in a relational database (such as MySQL). The spatiotemporal relationship between vehicles, vehicle locations, and location times is stored in a graph database (such as Neo4j) to construct a knowledge graph for cross-regional trajectory correlation and spatiotemporal conflict reasoning.

[0127] The method according to the embodiments of this application, based on a hybrid storage engine of vector retrieval library and graph database, enables millisecond-level retrieval of hundreds of millions of data points, supports concurrent deployment of thousands of cameras and city-level large-scale deployment, and improves the real-time performance and scalability of the system.

[0128] Furthermore, in one embodiment, a six-level knowledge graph is constructed in the graph database, encompassing provinces, cities, districts, areas, intersections, and devices, to achieve accurate identification and tracking of license plate clones across regions and intersections.

[0129] Figure 4 The diagram shown is a schematic diagram of a knowledge graph structure according to an embodiment of this application.

[0130] like Figure 4 As shown, a certain knowledge graph includes a structure of province-city-district-region-intersection-equipment. Specifically, in this knowledge graph, a region includes intersection A and intersection B, intersection A includes equipment A1 and equipment A2, and intersection B includes equipment B1.

[0131] Furthermore, in one embodiment, to address issues such as inconsistent data standards across different regions, missing data fields, or insufficient monitoring coverage of some nodes, the knowledge graph in the graph database is dynamically downgraded according to the actual scenario (e.g., province-city-district-intersection-device or city-district-intersection-device). During the graph construction process, multi-source data is adaptively fused and data is completed, including: Unified modeling of heterogeneous data: For the differences in the format of monitoring data from different provinces and cities, the standardized data access layer performs field mapping and semantic alignment, and uniformly transforms it into a graph-recognizable node and relationship structure; Intelligent completion of missing fields: Based on historical data distribution and road segment association rules, the missing key information such as time, location, and device number is inferred and completed; fields that cannot be completed are marked as "low confidence" nodes, reducing their weight in spatiotemporal conflict judgment; Blind spot supplementation mechanism: When surveillance video data of a certain intersection or road segment is missing, the system automatically connects to supplementary data sources such as roadside sensing devices (such as radar, geomagnetic sensors, RFID readers), vehicle passing records of adjacent intersections, and shared sensing information returned by social vehicles to build "virtual nodes" to fill monitoring blind spots and ensure trajectory continuity and inference integrity.

[0132] For example, such as Figure 4 As shown, when the road segment monitoring video data of device B1 is missing, a virtual node is constructed for device B1 to fill in the blind spot.

[0133] According to the method in the embodiments of this application, a six-level knowledge graph of province-city-district-region-intersection-device is constructed, and the regions, intersections and monitoring devices at each level are hierarchically associated and networked; the passage time and location information of the same license plate can be aggregated across regions and intersections, and physical reachability logic is performed by using the spatial distance and time difference between intersections, thereby realizing full-domain spatiotemporal intelligent reasoning and identifying hidden license plate clone behavior that is unlikely to occur in different places in a short period of time.

[0134] Furthermore, in one embodiment, in the graph database, the anonymized real-name information is additionally used as an "identity node" and associated with vehicle nodes, license plate nodes, and trajectory nodes to form a four-dimensional joint reasoning graph structure of "person, vehicle, license plate, and trajectory". When multiple instances of license plate cloning or changing occur, multiple sets of vehicles and license plates under the same owner can be quickly aggregated through the identity node, enabling cross-case suspect association and source tracing analysis.

[0135] According to the method of this application embodiment, by introducing desensitized real-name information, it is integrated into multimodal feature vectors and knowledge graph identity nodes to achieve four-dimensional joint verification of people, vehicles, license plates, and trajectories, and to achieve deep association and full-link verification of real-name system.

[0136] According to the method in the embodiments of this application, a real-name information desensitization and fusion mechanism is adopted to deeply integrate the desensitized vehicle owner identity, vehicle registration information, license plate text vector, and graph database identity nodes to achieve four-dimensional joint verification of human, vehicle, and license plate trajectories, which significantly improves the ability to identify cloned vehicles and legal effectiveness.

[0137] The method according to the embodiments of this application simultaneously supports the detection of three types of license plate anomalies: cloned plates, fake plates, and replaced plates, covering the entire scenario of intelligent transportation vehicle identity verification.

[0138] Furthermore, in one embodiment, in S302, a comprehensive evaluation is conducted based on the judgment results of multiple license plates for one vehicle, spatiotemporal conflicts, appearance conflicts, and attribute discrepancies, combined with user behavior, to further confirm whether there is an abnormal license plate.

[0139] Specifically, in S302, based on the judgment results of multiple license plates for one vehicle, spatiotemporal conflicts, appearance conflicts, and attribute discrepancies, risk point detection is performed in conjunction with user behavior (user's historical trajectory, violations, associated persons, and regional clustering) to obtain a risk point score. The risk point score is then used to determine whether there is an abnormal license plate.

[0140] For example, Figure 5 The diagram shown is a schematic diagram of the license plate anomaly detection logic according to an embodiment of this application.

[0141] based on Figure 4 The knowledge graph structure shown, in a scenario, such as Figure 5 As shown, a vehicle matching vehicle X (with the same appearance / attribute characteristics) was identified, and its license plate A appeared at intersection A at time T1; furthermore, a vehicle matching vehicle X (with the same appearance / attribute characteristics) was identified, and its license plate B appeared at intersection B at time T2. Based on the identification results, it was determined that there was a case of one vehicle with multiple license plates and a spatiotemporal conflict.

[0142] Based on vehicle X, license plate A, and license plate B, the corresponding identity nodes (anonymized real-name information) are invoked, and further, the corresponding user behavior is invoked. Risk point detection is performed by combining the results of determining multiple license plates for one vehicle and spatiotemporal conflicts with user behavior. Based on the results of determining multiple license plates for one vehicle and spatiotemporal conflicts, the risk point detection results, and the identity nodes, a warning for license plate cloning / replacement is issued.

[0143] According to the method in the embodiments of this application, a comprehensive risk point detection mechanism is adopted, which integrates license plate detection results, user behavior characteristics and vehicle characteristics to construct a multi-dimensional risk scoring model, thereby realizing proactive abnormal behavior detection and making up for the passivity and lag of traditional rule-based judgment.

[0144] Furthermore, in one embodiment, in S302, an early warning is issued after determining that there is an abnormal license plate.

[0145] Specifically, in one embodiment, a preliminary warning work order is generated, which includes the license plate number, vehicle image, capture location, warning time, warning type, judgment basis, historical trajectory, and vehicle owner information.

[0146] In one embodiment, a graded early warning is performed based on the risk point score in S302.

[0147] Specifically, in one embodiment, a license plate is determined to be abnormal if any of the following conditions are met, and a comprehensive assessment and classification-based early warning are conducted by combining historical trajectory, traffic violation behavior, related personnel, and regional clustering: 1. If the appearance feature vectors of vehicles with the same license plate have a very low similarity, it is suspected of being a cloned license plate; 2. If the same vehicle's appearance feature vector matches multiple different license plates, it is suspected of being a fake license plate or a license plate swap. 3. The same license plate appears at multiple intersections that are physically inaccessible within a short period of time, resulting in a conflict in its temporal and spatial trajectory; 4. The vehicle model and body color identified on-site are clearly inconsistent with the information registered and filed with the vehicle management office; 5. By comparing and tracing the vehicle's historical travel trajectory, the suspicion level is upgraded if the trajectory frequently and abnormally drifts and the vehicle is often found in sensitive areas; 6. Link vehicle's historical traffic violation records, and increase the weight of analysis for vehicles with long-term and repeated violations, hit-and-runs, and high-frequency violations; 7. By leveraging the relationships between people, vehicles, and roads, and linking information on vehicle owners and suspects involved in the case, accurate analysis and identification of vehicles involved in the case can be achieved; 8. Based on cluster analysis, feature clustering is performed on intersections with high incidence of violations, key areas, and high-frequency suspicious vehicle groups to achieve classified and graded management and early warning of regional crime-prone locations and suspected groups.

[0148] 9. Comprehensive Risk Detection: This system jointly analyzes license plate recognition results, vehicle feature vectors, and associated user behavior data (such as historical traffic violations, high-frequency traffic periods, abnormal driving trajectories, and nighttime activity patterns) to calculate a comprehensive risk score using a risk scoring model. When the risk score exceeds a set threshold, a focused warning is triggered regardless of whether the aforementioned rules are met, enabling proactive detection of concealed license plate clones / fake plates / plate swapping behaviors that exhibit "abnormal behavior but matching features."

[0149] Furthermore, in one embodiment, a multi-level verification and review process is added, which can access multiple third-party data sources such as other roadside sensing devices, data from surrounding checkpoints, and shared information from nearby vehicles to cross-verify the initial warning results and correct the suspicion level. After the verification is passed, a formal warning work order is generated and pushed to the business terminal in real time to support traffic management personnel in rapid verification and accurate handling.

[0150] The following example illustrates the implementation process of a license plate detection method according to an embodiment of this application.

[0151] In one embodiment, a license plate detection system is constructed based on a hierarchical structure.

[0152] Figure 6The diagram shown is a logical architecture diagram of a license plate detection system according to an embodiment of this application.

[0153] like Figure 6 As shown, the license plate detection system includes a data access layer 610. The functions of the data access layer 610 can be referred to the description in S300 (data acquisition module 201). Specifically, the data access layer 610 is used to acquire intersection surveillance video, vehicle images, roadside perception data, vehicle registration data, real-name information (anonymized real-name information), and user behavior data (e.g., traffic violation records, user trajectory records).

[0154] The license plate detection system also includes a blind spot supplementation layer 611. The blind spot supplementation layer 611 processes the data acquired by the data access layer 610. The function of the blind spot supplementation layer 611 can be referenced from the process of adaptive fusion of multi-source data and data completion described in S302 during the knowledge graph construction process. Specifically, the blind spot supplementation layer 611 performs heterogeneous unification, missing data completion, and virtual node creation operations on the data acquired by the data access layer 610.

[0155] The license plate detection system also includes a processing and annotation layer 620. The functions of the processing and annotation layer 620 can be found in the description of S301 (recognition module 202). Specifically, the processing and annotation layer 620 performs vehicle and license plate recognition on the data acquired by the data access layer 610 to obtain vehicle appearance features, license plate text features, and vehicle attribute features. The processing and annotation layer 620 also performs two-stage annotation based on the vehicle appearance features, license plate text features, and vehicle attribute features to obtain vehicle sample data. Furthermore, during the second stage of the two-stage annotation process, the processing and annotation layer 620 performs auxiliary verification based on the anonymized real-name information.

[0156] The license plate detection system also includes a multimodal vector layer 621. Multimodal vector layer 621 generates vehicle appearance feature vectors, license plate text feature vectors, and vehicle attribute feature vectors based on vehicle appearance features, license plate text features, and vehicle attribute features, respectively. Multimodal vector layer 621 also generates an identity fusion vector (license plate identity fusion vector) based on the license plate text features and anonymized real-name information. Multimodal vector layer 621 further performs vector fusion (attention fusion) on the vehicle appearance feature vector, license plate text feature vector, vehicle attribute feature vector, and identity fusion vector to generate a unique vehicle identity representation. The functions of multimodal vector layer 621 can be found in the description of S302.

[0157] The license plate detection system also includes a hybrid storage layer 630. The hybrid storage layer 630 stores a graph database, a relational database, and a vector database. The vector database stores the unique vehicle identity representation generated by the multimodal vector layer 621. The relational database stores structured data from vehicle appearance features, license plate identity fusion, and vehicle attribute features. The graph database stores a knowledge graph, which stores identity nodes that describe the trajectory of a person (user), vehicle, and license plate. Specifically, identity nodes describe the spatiotemporal relationships between user information (anonymized real-name information), vehicle information (vehicle features), license plate information (license plate features), and trajectory information (vehicle / license plate location data and location time). Furthermore, the hybrid storage layer 630 also performs downgraded completion of the knowledge graph based on the processing results of the blind spot completion layer 611. The function of the hybrid storage layer 630 can be referred to in the description of S302.

[0158] The license plate detection system also includes a decision and warning layer 640. This layer determines whether license plate anomalies exist based on data from the hybrid storage layer 630 and issues warnings accordingly. Specifically, the decision and warning layer 640 determines whether there are multiple license plates for one vehicle, spatiotemporal conflicts, attribute discrepancies, or appearance conflicts. Based on the decision results for multiple license plates for one vehicle, spatiotemporal conflicts, attribute discrepancies, and appearance conflicts, the decision and warning layer 640 also performs risk point detection in conjunction with user behavior, and issues tiered warnings based on the risk point detection results. The functions of the decision and warning layer 640 can be found in the description of S302.

[0159] Figure 7 The diagram shown is a schematic diagram of a license plate detection process according to an embodiment of this application.

[0160] In one embodiment, the electronic device performs Figure 7 The following process is shown to achieve abnormal license plate detection and early warning.

[0161] S700, multi-source data access, the execution of S700 can be referred to S300.

[0162] S701, video frame extraction, ROI cropping.

[0163] S702, Vehicle Recognition.

[0164] S703, license plate recognition.

[0165] The execution of S701~S703 can refer to S301.

[0166] During the execution of S701~S703, S704 is executed to determine whether there are any missing items (data missing, monitoring node missing).

[0167] If a missing value exists, execute S705 to complete the missing value. If no missing value exists, skip S705.

[0168] After completing S701~S703, execute S706 to perform two-stage annotation to obtain vehicle sample data. In the second stage of annotation, import the de-identified real-name information for real-name auxiliary verification.

[0169] Building upon S706, S707 is executed, performing multimodal vector fusion. Specifically, it generates vehicle appearance feature vectors, license plate text feature vectors, vehicle attribute feature vectors, and identity fusion vectors, and then performs vector fusion to generate a unique vehicle identity representation.

[0170] Based on S707, S708 is implemented, using hybrid storage. Data is stored in graph databases, relational databases, and vector retrieval libraries.

[0171] Building upon S708, S709 is executed, implementing a five-dimensional anomaly detection system. This system includes detections for spatiotemporal conflicts, multiple license plates for a single vehicle, attribute discrepancies, and appearance conflicts. Additionally, risk point detection is performed based on user behavior, building upon these detections. In S709, a risk point score is generated based on the final risk point detection results.

[0172] Following S709, S710 is executed for risk point scoring. If the risk point score in S709 exceeds a preset threshold, a corresponding tiered warning is issued based on the risk point score (S711). If the risk point score in S709 does not exceed the preset threshold, it is determined that there is no license plate anomaly, and this round of license plate detection ends.

[0173] According to the method in the embodiments of this application, the entire process from data access, annotation, feature extraction, association, retrieval to early warning and iteration is fully automated, which can realize a closed loop of full-process automation, thereby reducing reliance on manual labor and improving the speed of early warning response.

[0174] An embodiment of this application also proposes an electronic device. This electronic device is used to execute the method flow or part of the method flow described in the embodiments of this application.

[0175] Figure 8 This is a schematic diagram of an electronic device structure according to an embodiment of this application.

[0176] like Figure 8 As shown, the electronic device 2500 includes a memory 2502 for storing computer program instructions and a processor 2501 for executing the program instructions. When the computer program instructions are executed by the processor 2501, the electronic device 2500 is triggered to execute the method steps performed by the electronic device as described in the embodiments of this application.

[0177] Specifically, in one embodiment of this application, the aforementioned one or more computer programs are stored in the aforementioned memory 2502. The aforementioned one or more computer programs include instructions that, when executed by the aforementioned electronic device 2500, cause the aforementioned electronic device 2500 to perform the method steps described in the embodiments of this application.

[0178] It is understood that the structural description of the electronic device 2500 in this application does not constitute a specific limitation on the electronic device 2500. In other embodiments of this application, the electronic device 2500 may include other components besides the processor 2501 and the memory 2502.

[0179] In one embodiment, electronic device 2500 may refer to electronic device 100, wherein processor 2501 may refer to processor 110, and memory 2502 may refer to internal memory 121.

[0180] Processor 2501 and memory 2502 can be combined into a single processing device, but more commonly they are separate components.

[0181] An embodiment of this application also provides an electronic chip. This electronic chip is used to execute the method flow or part of the method flow described in the embodiments of this application.

[0182] Specifically, the electronic chip includes a processor for executing program instructions. When the computer program instructions are executed by the processor, the electronic chip is triggered to perform the steps described in the embodiments of this application. The processor of the electronic chip can refer to the processor of the above-described electronic device.

[0183] Optionally, the devices, apparatuses, and modules described in the embodiments of this application may be implemented by computer chips or physical entities, or by products with certain functions.

[0184] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media containing computer-usable program code.

[0185] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0186] Specifically, one embodiment of this application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to execute the method provided in the embodiment of this application.

[0187] An embodiment of this application also provides a computer program product, which includes a computer program that, when run on a computer, causes the computer to perform the method provided in the embodiment of this application.

[0188] The embodiments described in this application are described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0189] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0190] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0191] It should also be noted that in the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, or the existence of B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0192] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0193] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0194] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0195] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments of this application can be implemented using electronic hardware, computer software, or a combination of electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0196] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0197] The above description is merely a specific embodiment of this application. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application. The protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for detecting license plates, characterized in that, The method is applied to an electronic device, and the method includes: Acquire on-road vehicle monitoring data and vehicle registration data, wherein the on-road vehicle monitoring data includes at least video surveillance data from intersection monitoring; Based on the on-road vehicle monitoring data, identify the vehicle appearance features, license plate text features, and vehicle attribute features of the on-road vehicles. Based on the vehicle's appearance features, license plate text features, and vehicle attribute features, and in conjunction with the vehicle registration data, license plate detection is performed to determine whether there are any license plate anomalies.

2. The method according to claim 1, characterized in that, The step of identifying the vehicle appearance features, license plate text features, and vehicle attribute features of vehicles on the road based on the on-road vehicle monitoring data includes: The video surveillance data is subjected to frame extraction to obtain the extracted vehicle images; The extracted vehicle images are cropped to obtain the region of interest related to the vehicle on the road. Feature recognition is performed on the region of interest to obtain the vehicle's appearance features and vehicle attribute features; Optical character recognition is performed on the region of interest to obtain the text features of the license plate.

3. The method according to claim 1, characterized in that, The method further includes: The first-stage annotation results are obtained by annotating the vehicle's appearance features, license plate text features, and vehicle attribute features. The annotation results of the first stage are reviewed to obtain vehicle sample data.

4. The method according to claim 3, characterized in that, The step of reviewing the annotation results of the first stage and obtaining vehicle sample data includes: The first-stage labeling results are reviewed based on the anonymized real-name information to obtain the vehicle sample data.

5. The method according to any one of claims 1-4, characterized in that, The step of detecting license plates based on the vehicle's appearance features, license plate text features, and vehicle attribute features, combined with the vehicle registration data, to determine whether there are any license plate anomalies includes: Based on the vehicle appearance features, the license plate text features, and the vehicle attribute features, a vehicle appearance feature vector, a license plate text feature vector, and a vehicle attribute feature vector are generated respectively. Based on the vehicle appearance feature vector, the license plate text feature vector, and the vehicle attribute feature vector, determine whether there is an abnormal license plate.

6. The method according to claim 5, characterized in that, The step of determining whether there is a license plate anomaly based on the vehicle appearance feature vector, the license plate text feature vector, and the vehicle attribute feature vector includes: The vehicle appearance feature vector, the license plate text feature vector, and the vehicle attribute feature vector are subjected to cross-modal joint fusion encoding to generate a unique vehicle identity representation. Based on the vehicle's unique identification characteristics, determine whether there is any license plate anomaly.

7. The method according to claim 6, characterized in that, The step of determining whether there is a license plate anomaly based on the vehicle appearance feature vector, the license plate text feature vector, and the vehicle attribute feature vector includes: The desensitized real-name information is jointly encoded with the license plate text feature vector to generate a license plate identity fusion vector. Based on the vehicle appearance feature vector, the license plate identity fusion vector, and the vehicle attribute feature vector, determine whether there is an abnormal license plate.

8. The method according to claim 6, characterized in that, The step of determining whether there is an abnormal license plate based on the vehicle's unique identification feature includes: This system utilizes vector retrieval libraries, relational databases, and graph databases to detect license plate anomalies and determine the presence of such anomalies. The vector retrieval library is used to store unique vehicle identification representations; The relational database is used to store structured data from the vehicle's appearance features, license plate identity fusion, and vehicle attribute features; The graph database is used to store the spatiotemporal relationship between the vehicles on the road, their locations, and the times they were located.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor; The processor is configured to execute instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by a computer system, perform the method as described in any one of claims 1-8.

Images