Unmanned aerial vehicle system for illegal parking judgment

Abstract

The invention relates to an unmanned aerial vehicle system for illegal parking judgment. The unmanned aerial vehicle system comprises an edge calculation unit; the edge calculation unit is used for loading the deep learning model from the local storage of the unmanned aerial vehicle in the starting process of the unmanned aerial vehicle; performing parking space detection, vehicle detection and license plate detection on the obtained video stream image frame through a deep learning model in the process of executing the traffic inspection task by the unmanned aerial vehicle; and according to results of vehicle detection, license plate detection and parking space detection, a vehicle parking determination result containing a closed-loop evidence chain is calculated. Through the system, the image analysis task which is to be executed by the ground server is executed by the unmanned aerial vehicle, the video does not need to be transmitted back in real time, the transmission cost and the network fluctuation influence are reduced, a closed-loop evidence chain is obtained based on the detection of the vehicle, the license plate and the parking space, and the direct development of subsequent illegal parking processing work is facilitated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicles, in particular to an unmanned aerial vehicle system for illegal parking judgment. BACKGROUND

[0002] In recent years, unmanned aerial vehicles (UAV) have gradually been applied to urban traffic patrol tasks due to their strong maneuverability, wide perspective, and flexible deployment. However, current illegal parking identification systems based on unmanned aerial vehicle platforms mostly only realize video acquisition and transmission functions, i.e., they are still in the primary stage of "flying + shooting + transmission", and their core analysis process still relies on centralized processing by ground servers (such as patents CN119625996A and CN119399652A), resulting in high overall response delay, large communication bandwidth occupation, and difficulty in meeting real-time requirements. In other words, the existing technical solutions essentially regard unmanned aerial vehicles as "flying cameras" rather than "aerial intelligent agents", and fail to effectively integrate the aerial advantages of unmanned aerial vehicle platforms with the decision-making capabilities of artificial intelligence, resulting in insufficient overall intelligence of the system and difficulty in meeting the needs of modern urban governance.

[0003] Currently, there is no effective solution to how to improve the intelligent decision-making capability of unmanned aerial vehicles in illegal parking patrol tasks in the related art. SUMMARY

[0004] Embodiments of the present application provide an unmanned aerial vehicle system for illegal parking judgment to at least solve the problem of how to improve the intelligent decision-making capability of unmanned aerial vehicles in illegal parking patrol tasks in the related art.

[0005] In a first aspect, embodiments of the present application provide an unmanned aerial vehicle system for illegal parking judgment, which includes an edge computing unit. The edge computing unit is configured to load a deep learning model from a local storage of the unmanned aerial vehicle during a starting process of the unmanned aerial vehicle. The edge computing unit is configured to perform parking space detection, vehicle detection, and license plate detection on the acquired video stream image frames through the deep learning model during a traffic patrol task performed by the unmanned aerial vehicle. The edge computing unit is configured to calculate a vehicle parking judgment result containing a closed-loop evidence chain according to the results of the vehicle detection, the license plate detection, and the parking space detection.

[0006] In some embodiments, the edge computing unit is configured to perform cross-modal correlation matching on motor vehicles and license plates according to motor vehicle information obtained through the vehicle detection and license plate information obtained through the license plate detection, to obtain an identity binding relationship between the motor vehicles and the license plates.

[0007] In some embodiments, the edge computing unit is configured to determine a parking state of the motor vehicle according to the motor vehicle information and the parking space information, by using spatial geometric relationship rules and traffic semantic rules, wherein the traffic semantic rules include whether the motor vehicle is located in a parking space, whether the motor vehicle is parked beyond a line, and whether the motor vehicle invades a no-parking area.

[0008] In some embodiments, the edge computing unit includes a target tracker. The target tracker is configured to track the motor vehicle from each video stream image frame, ensure continuity of the motor vehicle in the video stream sequence, and obtain a vehicle ID track. The edge computing unit is configured to perform cross-modal association matching according to the identity binding relationship of the motor vehicle, the parking state, and the vehicle ID track, and obtain a parking determination result of the motor vehicle, wherein the parking determination result includes a closed-loop evidence chain of vehicle ID-plate number-parking state.

[0009] In some embodiments, the edge computing unit is configured to generate a structured event record according to the parking determination result of the motor vehicle, and aggregate all structured event records to generate a standardized report file in JSON format after the traffic inspection task is completed. According to the parking determination result and the video stream image frame, a labeled video stream is generated.

[0010] In some embodiments, the edge computing unit is configured to generate a structured event record according to the closed-loop evidence chain of vehicle ID-plate number-parking state included in the parking determination result. wherein t is a timestamp, GPS t is a geographic location, id j is a vehicle ID, plate k is a license plate number, status is a parking state, confidence is a detection confidence, and triggered rule is a parking determination rule number.

[0011] In some embodiments, the edge computing unit is configured to perform multi-layer visual information superposition on the video stream image frame according to the closed-loop evidence chain of vehicle ID-plate number-parking state included in the parking determination result, and generate a labeled video stream, wherein the multi-layer visual information superposition includes parking space annotation with a specific color contour line, boundary box coloring of the vehicle according to the parking state, display of vehicle ID, license plate number, and parking state by using information labels, and display of statistical information in a corner of the picture.

[0012] In some embodiments, the edge computing unit is configured to load the deep learning model M_parking, the deep learning model M_car, and the deep learning model M_plate from the local storage of the UAV during the starting process of the UAV, wherein the deep learning model M_parking is used for detecting a ground parking area, the deep learning model M_car is used for detecting a motor vehicle, and the deep learning model M_plate is used for locating a license plate and extracting visual features thereof.

[0013] In some embodiments, the edge computing unit is configured to perform parking space detection on the acquired video stream image frames by using the deep learning model M_parking to obtain a set of parking space information during the process in which the UAV performs the traffic inspection task. The edge computing unit is configured to perform motor vehicle detection on the acquired video stream image frames by using the deep learning model M_car to obtain a set of motor vehicle information during the process in which the UAV performs the traffic inspection task. The edge computing unit is configured to perform license plate detection on the acquired video stream image frames by using the deep learning model M_plate to obtain a set of license plate information during the process in which the UAV performs the traffic inspection task.

[0014] In some embodiments, the UAV system includes a camera module, a positioning module, a power management module, and a wireless communication module. The camera module is configured to collect video stream images of the UAV during the traffic inspection process. The positioning module is configured to acquire position information of the UAV during the traffic inspection process. The power management module is configured to manage power supply of the UAV. The wireless communication module is configured to transmit the structured event records, the standardized report files, and the labeled video streams generated by the edge computing unit to a ground terminal.

[0015] Compared with the related art, the unmanned aerial vehicle system for illegal parking judgment provided by the embodiment of the application comprises an edge computing unit; the edge computing unit is configured to load a deep learning model from a local storage of the unmanned aerial vehicle during a starting process of the unmanned aerial vehicle; during a process in which the unmanned aerial vehicle performs a traffic inspection task, the deep learning model is used to perform parking space detection, vehicle detection and license plate detection on acquired video stream image frames; and a vehicle parking judgment result containing a closed loop evidence chain is calculated according to the results of the vehicle detection, the license plate detection and the parking space detection, so that the image analysis task that should be performed by a ground server is performed by the edge computing unit of the unmanned aerial vehicle, thereby eliminating the need to return the collected video to the ground in real time, reducing the influence of transmission cost and network fluctuation, and performing separate detection on the vehicle, the license plate and the parking space, so that the closed loop evidence chain about the vehicle parking can be obtained on the unmanned aerial vehicle, facilitating the direct development of subsequent illegal parking processing work, and solving the problem of how to improve the intelligent decision-making capability of the unmanned aerial vehicle in the illegal parking inspection task. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are included to provide a further understanding of the application, constitute a part of the application and serve to explain the application together with the specification. The illustrative embodiments of the application and their description serve to explain the application and do not limit the application in any manner. In the drawings: Figure 1 FIG. 1 is a structural block diagram of an unmanned aerial vehicle system for illegal parking judgment according to an embodiment of the application; Figure 2 FIG. 2 is a flowchart of starting the unmanned aerial vehicle system according to an embodiment of the application; Figure 3 FIG. 3 is a flowchart of model detection according to an embodiment of the application; Figure 4 FIG. 4 is a flowchart of cross-modal correlation matching according to an embodiment of the application; Figure 5 FIG. 5 is a flowchart of logical reasoning judgment according to an embodiment of the application; Figure 6 FIG. 6 is a flowchart of labeled video stream generation according to an embodiment of the application; Figure 7 FIG. 7 is a whole structural block diagram of the unmanned aerial vehicle system for illegal parking judgment according to an embodiment of the application; Figure 8 FIG. 8 is a schematic diagram of the internal structure of an electronic device according to an embodiment of the application. DETAILED DESCRIPTION

[0017] In order to make the purposes, technical solutions, and advantages of the present application clearer, the present application is described and explained below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application and not to limit the present application. Based on the embodiments provided by the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort fall within the scope of the present application.

[0018] It is obvious that the accompanying drawings in the following description are only some examples or embodiments of the present application, and for those of ordinary skill in the art, the present application can be applied to other similar scenarios without creative effort based on the accompanying drawings. In addition, it can be understood that although the efforts made in the development process can be complex and lengthy, for those of ordinary skill in the art related to the content disclosed in the present application, some design, manufacture or production changes based on the technical content disclosed in the present application are only routine technical means and should not be understood as insufficient disclosure of the content disclosed in the present application.

[0019] In the present application, the phrase "embodiments" means that the specific features, structures or properties described in conjunction with the embodiments can be included in at least one embodiment of the present application. The appearance of this phrase at various places in the specification does not necessarily mean the same embodiment, nor is it an independent or alternative embodiment to other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in the present application can be combined with other embodiments without conflict.

[0020] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0021] This application provides a drone system for determining illegal parking. Figure 1 This is a structural block diagram of a drone system for determining illegal parking according to an embodiment of this application, as shown below. Figure 1 As shown, the drone system includes an edge computing unit; An edge computing unit is used to load a deep learning model from the drone's local storage during the drone's startup process. This deep learning model is preferably a lightweight model. Specifically, the edge computing unit is used to load deep learning models M_parking, M_car, and M_plate from the drone's local storage during the drone's startup process. Among them, deep learning model M_parking is used to detect ground parking space areas, deep learning model M_car is used to detect motor vehicles, and deep learning model M_plate is used to locate license plates and extract their visual features.

[0022] It should be noted that activating the drone system used for illegal parking detection and performing initialization operations ensures that the system has complete models, algorithms, parameters, and communication capabilities before entering the perception phase, laying the foundation for subsequent high-precision illegal parking detection. Figure 2 This is a schematic diagram of the process of starting the unmanned aerial vehicle system according to an embodiment of this application, such as... Figure 2As shown, the preferred initialization operation procedure is as follows: (1) Model loading and deployment: After the system starts, three pre-trained and optimized lightweight deep learning models are loaded from local storage: M_parking is used to detect ground parking space areas, M_car is used to detect motor vehicles, and M_plate is used to locate license plates and extract their visual features. All models adopt the TensorRT format to improve inference efficiency and adapt to the computing resource limitations of the UAV platform.

[0023] (2) Initialization of the target tracker: The target tracking module is initialized at the same time. This module is based on the improved DeepSORT algorithm and includes a Kalman filter for predicting the vehicle's motion state and an appearance feature extraction network for handling occlusion and ID switching problems to ensure the continuity of vehicle identity in the video sequence.

[0024] (3) System parameter configuration: Load the preset set of judgment parameters, including cross-modal matching threshold, parking space overlap rate threshold, and line crossing ratio threshold. These parameters can be flexibly configured according to different urban road specifications to improve the system's adaptability.

[0025] (4) Establishment of communication link: Start the wireless communication module to establish a stable data transmission channel with the ground control terminal for the real-time transmission of subsequent structured reports and labeled videos, so as to ensure the timeliness and reliability of law enforcement information.

[0026] (5) Hardware resource check: Perform self-check on GPU memory, storage space and power status to confirm that the system has the ability to run continuously and avoid interruption in the middle of the task.

[0027] The edge computing unit is used to perform parking space detection, vehicle detection, and license plate detection on the acquired video stream image frames during the traffic inspection mission of the drone; Specifically, the edge computing unit is used to detect parking spaces in the video stream image frames acquired by the drone during the traffic inspection task, through the deep learning model M_parking, to obtain a set of parking space information. Preferably, Figure 3 This is a schematic diagram of the model detection process according to an embodiment of this application, such as... Figure 3 As shown, parking space detection and structured representation are performed. The M_parking model is used to analyze the current video stream image frames. Perform inference and output a set of parking space information from the image:

[0028] in, N represents the four boundary points of the bounding box of the i-th parking space.p This indicates the number of parking spaces in the video frame. Additionally, it includes the bounding box for each parking space. Assign type label This is used to distinguish between parallel, perpendicular, or angled parking spaces, providing a basis for subsequent analysis of parking regulations.

[0029] Specifically, the edge computing unit is used to detect motor vehicles in the video stream image frames acquired by the UAV during the traffic inspection task, through the deep learning model M_car, to obtain a set of motor vehicle information. Preferably, such as Figure 3 As shown, vehicle detection is linked to identity. The M_car model detects vehicles on the same video frame and outputs a set of vehicle information in the image. , where N v This indicates the number of motor vehicles in a video frame. This represents the vehicle bounding box of the j-th locomotive. Indicates category confidence, id j This represents the vehicle ID of a motor vehicle. The vehicle ID remains stable throughout the video sequence, supporting cross-frame behavior analysis.

[0030] Specifically, the edge computing unit is used to detect license plates on the acquired video stream image frames through the deep learning model M_plate during the traffic inspection task performed by the drone, and obtain a set of license plate information.

[0031] Preferably, such as Figure 3 As shown, license plate detection and feature extraction are performed. The M-plate model is used to locate the license plate region, and the output is a set of license plate information from the image. , where N l This indicates the number of license plates in the video frame. This represents the bounding box of the k-th license plate. Indicates category label, This represents the depth visual feature vector. This feature vector is used for subsequent cross-modal matching with the vehicle's appearance.

[0032] It should be noted that the three detection tasks are completed at the same timestamp, ensuring strict alignment of the perception results in the time dimension. This avoids information misalignment caused by processing delays and provides high-quality input for multimodal fusion. Unlike the existing serial cascaded architecture of "detecting vehicles first, then cropping the image to detect license plates," the parallel synchronous perception mechanism of this application breaks through the limitations of traditional serial detection, effectively avoiding error accumulation and processing delays caused by cascaded processing, and significantly improving the robustness and accuracy of the system in dynamic flight scenarios. In other words, the M_parking model, M_car model, and M_plate model synchronously acquire information on three key targets—parking spaces, vehicles, and license plates—from a single frame image, avoiding error accumulation caused by cascaded processing.

[0033] The edge computing unit is used to calculate the vehicle parking determination result containing a closed-loop evidence chain based on the results of vehicle detection, license plate detection, and parking space detection.

[0034] Specifically, the edge computing unit is used to perform cross-modal association matching between motor vehicles and license plates based on the motor vehicle information obtained from vehicle detection and the license plate information obtained from license plate detection, so as to obtain the identity binding relationship between motor vehicles and license plates.

[0035] Preferably, Figure 4 This is a schematic diagram of the cross-modal association matching process according to an embodiment of this application, such as... Figure 4 As shown, the steps for cross-modal association matching of vehicle and license plate numbers are as follows: To achieve identity binding between vehicle ID and license plate number, a cross-modal association mechanism is constructed. For each detected vehicle v j With license plate k Calculate its overall matching score:

[0036] In this system, IOU is a function that measures the spatial overlap between the bounding boxes of the two vehicles, CosSim is a function that measures the semantic consistency between the vehicle's appearance features and the license plate features, and α is a balancing weight coefficient. If the matching score exceeds a preset threshold and the license plate is the optimal match for the vehicle, then an association between the vehicle and the license plate is established. Compared to matching methods that rely solely on spatial location (such as IoU, which are prone to mismatches when the vehicle is partially occluded or has severe perspective distortion), or matching methods that rely solely on OCR recognition (which have unstable recognition rates due to limitations in image resolution and lighting conditions), the cross-modal association matching in this application takes into account both geometric constraints and semantic consistency, forming a complementary verification mechanism. This mechanism effectively overcomes the identity mismatch problem caused by perspective distortion, partial occlusion, and other factors from an aerial perspective.

[0037] It should be noted that existing UAV systems can only output vague information such as "a vehicle is illegally parked at a certain location," meaning that the information output to the ground terminal does not include the mapping relationship between vehicle ID and license plate number. In other words, the subsequent judgment result lacking license plate information is not traceable and must rely on manual secondary verification at the ground terminal, which greatly reduces the practical value of the automated system and makes it difficult to achieve a closed loop of "automatic identification → automatic archiving → automatic illegal parking processing." However, the embodiment of this application is the first to deploy the entire process of multi-task detection, cross-modal matching, and rule reasoning on the edge computing unit of the UAV, realizing the aerial intelligent decision-making capability of "discovery upon judgment, judgment upon output," filling the technological gap in the field of edge intelligent illegal parking judgment by UAVs.

[0038] Specifically, the edge computing unit is used to determine the parking status of a motor vehicle based on the vehicle information obtained from vehicle detection and the parking space information obtained from parking space detection, through spatial geometric relationship rules and traffic semantic rules. The traffic semantic rules include whether the vehicle is within a parking space, whether it is parked beyond the line, and whether it has entered a no-parking zone.

[0039] Specifically, the edge computing unit includes a target tracker for tracking motor vehicles from various video stream image frames to ensure the continuity of motor vehicles in the video stream sequence and obtain vehicle ID trajectories. The edge computing unit then performs cross-modal association matching based on the identity binding relationship of the motor vehicle, the parking status of the vehicle, and the vehicle ID trajectory to obtain the parking determination result of the motor vehicle. The parking determination result includes a closed-loop evidence chain of vehicle ID-license plate number-parking status.

[0040] Preferably, Figure 5 This is a flowchart illustrating the logical reasoning and determination process according to embodiments of this application, such as... Figure 5 As shown, an interpretable illegal parking detection engine is constructed based on spatial geometric relationship rules and traffic semantic rules. First, it determines whether the vehicle is within a parking space:

[0041] If the vehicle boundary box of a motor vehicle With parking space boundary box The overlap rates are all below the threshold. If so, it is determined to be parking outside the designated space. Secondly, it is determined whether the parking crosses the line:

[0042] If the proportion of a motor vehicle's area extending beyond the parking space boundary exceeds a threshold... If the area is too large, it is considered irregular parking, and Area is the area calculation function. Finally, by combining the no-parking zone data provided by the geographic information system, it is determined whether the vehicle has entered the no-parking zone:

[0043] in, This area is designated as a no-parking zone. Based on the combined results of the three rules, the system outputs the vehicle's parking status (located within a parking space, parked beyond the line, or encroaching on a no-parking zone) via a logical OR operation. This is further combined with the vehicle ID trajectory to arrive at the final parking determination (e.g., vehicles whose vehicle ID trajectories stop or change in the video stream can be identified as temporary passenger pick-up / drop-off vehicles, meaning temporary parking is not investigated), ensuring the comprehensiveness and traceability of the determination logic.

[0044] It should be noted that existing ground terminals, after receiving video data collected by drones, rely on end-to-end deep learning models to directly output illegal parking labels, lacking intermediate reasoning steps; that is, their judgment logic is hidden within the neural network parameters, making it difficult to explain "why it was judged as illegal parking." In other words, in controversial scenarios (such as partial crossing of the line, temporary passenger pick-up and drop-off, etc.), it is difficult to provide verifiable judgment criteria; leading to judgment results that are easily questioned and failing to meet the basic requirements of transparency, defensibility, and compliance. In contrast, the embodiments of this application introduce an interpretable judgment engine based on spatial geometry and traffic rules in the judgment of the drone's edge computing unit, transforming "model-driven" into "rule + model collaborative driving," converting semantics such as "no parking areas" and "parking regulations" in road traffic rules into computable logical rules. This achieves, for the first time, the alignment of algorithmic judgment with semantic rules, filling the design gap in compliance and interpretability of drone illegal parking judgment.

[0045] The edge computing unit is used to generate structured event records based on the parking determination results of motor vehicles, and after the traffic inspection task is completed, it summarizes all structured event records and generates a standardized report file in JSON format; it also generates annotated video streams based on the parking determination results and video stream image frames.

[0046] Specifically, the edge computing unit is used to generate structured event records based on the parking determination results. Where t is the timestamp, GPS t For geographical location, id j For vehicle ID, plate k The license plate number is the vehicle's license plate number, status is the parking status, confidence is the detection confidence level, and triggered rule is the number of the parking determination rule that was triggered.

[0047] It should be noted that existing drones used for illegal parking detection only output image screenshots or simple text descriptions. Illegal parking events lack key metadata such as timestamps, GPS coordinates, confidence levels, and judgment rule numbers. This necessitates manual intervention in subsequent processes such as data entry and evidence archiving. In contrast, the embodiment of this application completes the entire closed-loop processing from perception and analysis to report generation on the drone itself, directly outputting structured event records containing time, location, vehicle ID, license plate number, confidence level, and judgment criteria. This avoids the offline "collection-transmission-post-processing" mode of traditional solutions, achieving real-time enforcement capabilities of "aerial detection, aerial judgment, and aerial recording," greatly improving the efficiency and response speed of illegal parking supervision. The generated structured data can be directly connected to urban management platforms, possessing high practical value.

[0048] Specifically, Figure 6 This is a schematic diagram illustrating the process of generating an annotated video stream according to an embodiment of this application, such as... Figure 6 As shown, the edge computing unit is used to overlay multi-layered visual information on video stream image frames based on the closed-loop evidence chain of vehicle ID-license plate number-parking status contained in the parking determination result, and generate an annotated video stream. The multi-layered visual information overlay includes parking spaces marked with outlines of specific colors, vehicle bounding boxes distinguished by coloring according to parking status, displaying vehicle ID, license plate number and parking status using information tags, and displaying statistical information in the corners of the screen.

[0049] It should be noted that after the mission is completed, the system summarizes all event records and generates a standardized report file. The report stores structured data in JSON format and also generates a readable text report, including an event summary and the basis for judgment. The report and annotated video are then transmitted back to the ground control terminal. After data transmission is complete, system resources are released, relevant modules are shut down, and the mission process ends.

[0050] It should be further noted that Table 1 is an example table illustrating the relevant evaluation indicators of the experiment, and Table 2 is an example table comparing the experimental effects between the UAV system of this application embodiment and the basic illegal parking judgment system.

[0051] Table 1

[0052] Table 2

[0053] As can be seen, the unmanned aerial vehicle system provided in this application has achieved good results in various test evaluation indicators.

[0054] In some of these embodiments, Figure 7 This is an overall structural block diagram of a drone system for determining illegal parking according to an embodiment of this application, as shown below.Figure 7 As shown, the drone system also includes a camera module, a positioning module, a power management module, and a wireless communication module; The camera module is used to capture video stream images of the drone during traffic inspection. The positioning module is used to obtain the location information of the drone during traffic inspection. The power management module is used to manage the power supply of the drone; The wireless communication module is used to transmit the structured event logs, standardized report files, and labeled video streams generated by the edge computing unit back to the ground terminal.

[0055] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0056] This embodiment provides an electronic device, including a memory and a processor, which can be used to build the unmanned aerial vehicle system provided in the above embodiment.

[0057] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0058] Optionally, the electronic device may further include a processor, memory, network interface, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements the algorithm steps executed by the edge computing unit in the UAV system. It should be noted that specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0059] In one embodiment, Figure 8 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application, such as... Figure 8 As shown, an electronic device is provided, which can be a server, and its internal structure diagram can be as follows. Figure 8As shown, the electronic device includes a processor, a network interface, internal memory, and non-volatile memory connected via an internal bus. The non-volatile memory stores the operating system, computer programs, and a database. The processor provides computing and control capabilities, the network interface communicates with external terminals via a network, the internal memory provides an environment for the operating system and computer programs to run, the computer programs are executed by the processor to implement the algorithm steps executed by the edge computing unit in the UAV system, and the database stores data.

[0060] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0061] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0062] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0063] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A drone system for determining illegal parking, characterized in that, The unmanned aerial vehicle system includes an edge computing unit; The edge computing unit is used to load a deep learning model from the drone's local storage during the drone's startup process. The edge computing unit is used to perform parking space detection, vehicle detection and license plate detection on the acquired video stream image frames through the deep learning model during the traffic inspection task performed by the UAV. The edge computing unit is used to calculate a vehicle parking determination result containing a closed-loop evidence chain based on the results of vehicle detection, license plate detection, and parking space detection.

2. The system according to claim 1, characterized in that, The edge computing unit is used to perform cross-modal association matching between the motor vehicle and the license plate based on the motor vehicle information obtained from the vehicle detection and the license plate information obtained from the license plate detection, so as to obtain the identity binding relationship between the motor vehicle and the license plate.

3. The system according to claim 2, characterized in that, The edge computing unit is used to determine the parking status of a motor vehicle based on the vehicle information obtained from vehicle detection and the parking space information obtained from parking space detection, through spatial geometric relationship rules and traffic semantic rules. The traffic semantic rules include whether the vehicle is within a parking space, whether it is parked beyond the line, and whether it has entered a no-parking zone.

4. The system according to claim 3, characterized in that, The edge computing unit includes a target tracker; The target tracker is used to track motor vehicles from each video stream image frame, ensuring the continuity of motor vehicles in the video stream sequence and obtaining the vehicle ID trajectory; The edge computing unit is used to perform cross-modal association matching based on the identity binding relationship of the motor vehicle, the parking status of the vehicle and the vehicle ID trajectory to obtain the parking determination result of the motor vehicle, wherein the parking determination result includes a closed-loop evidence chain of vehicle ID-license plate number-parking status.

5. The system according to claim 1, characterized in that, The edge computing unit is used to generate structured event records based on the parking determination results of motor vehicles, and after the traffic inspection task is completed, to summarize all structured event records and generate a standardized report file in JSON format. Based on the parking determination result and the video stream image frames, an annotated video stream is generated.

6. The system according to claim 5, characterized in that, The edge computing unit is used to generate structured event records based on the closed-loop evidence chain of vehicle ID, license plate number, and parking status contained in the parking determination result. Where t is the timestamp, GPS t For geographical location, id j For vehicle ID, plate k The license plate number is the vehicle's license plate number, status is the parking status, confidence is the detection confidence level, and triggered rule is the number of the parking determination rule that was triggered.

7. The system according to claim 5, characterized in that, The edge computing unit is used to perform multi-layer visualization information overlay on the video stream image frames based on the closed-loop evidence chain of vehicle ID-license plate number-parking status contained in the parking determination result, and generate an annotated video stream. The multi-layer visualization information overlay includes parking spaces marked with outlines of specific colors, vehicle bounding boxes distinguished by color according to parking status, displaying vehicle ID, license plate number and parking status using information tags, and displaying statistical information in the corner of the screen.

8. The system according to claim 1, characterized in that, The edge computing unit is used to load deep learning models M_parking, M_car, and M_plate from the local storage of the drone during the drone startup process. The deep learning model M_parking is used to detect ground parking space areas, the deep learning model M_car is used to detect motor vehicles, and the deep learning model M_plate is used to locate license plates and extract their visual features.

9. The system according to claim 8, characterized in that, The edge computing unit is used to detect parking spaces by using the deep learning model M_parking to obtain a set of parking space information during the process of the UAV performing traffic inspection tasks. The edge computing unit is used to detect motor vehicles by using the deep learning model M_car to obtain a set of motor vehicle information during the process of the UAV performing traffic inspection tasks. The edge computing unit is used to perform license plate detection on the acquired video stream image frames through the deep learning model M_plate during the traffic inspection task performed by the UAV, and obtain a set of license plate information.

10. The system according to claim 1, characterized in that, The unmanned aerial vehicle (UAV) system includes a camera module, a positioning module, a power management module, and a wireless communication module. The camera module is used to capture video stream images of the drone during traffic inspection. The positioning module is used to obtain the location information of the drone during traffic inspection. The power management module is used to manage the power supply of the drone; The wireless communication module is used to transmit the structured event records, standardized report files, and labeled video streams generated by the edge computing unit back to the ground terminal.

Images