Unmanned aerial vehicle inspection method, unmanned aerial vehicle, cloud platform and unmanned aerial vehicle inspection system

Abstract

Embodiments of the present application provide a UAV inspection method, a UAV, a cloud platform and a UAV inspection system, and relate to the technical field of UAVs. The UAV flies along a preset flight path, collects image data of an inspection area in real time, analyzes the image data according to a preset target feature library, generates an alarm signal including a target identification label, a position coordinate and a timestamp when a target scene meeting preset conditions is identified, matches a corresponding automatic disposal process from a preset disposal rule library according to the target identification label and the position coordinate in the alarm signal, autonomously performs one or more on-site disposal actions for the target scene according to the automatic disposal process, collects evidence data, returns the evidence data to the cloud platform, and resumes execution of the preset flight path. Thus, the disposal process corresponding to the target scene is automatically matched and executed, and the standardization degree and response efficiency of inspection disposal are improved.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and more specifically, to a UAV inspection method, a UAV, a cloud platform, and a UAV inspection system. Background Technology

[0002] In recent years, drone inspection technology has been widely used in urban governance, ecological and environmental protection, and infrastructure monitoring. The current mainstream drone inspection system usually adopts an image acquisition plus back-end recognition working mode: the drone flies along a preset route and collects images, and transmits the image data back to the back-end server in real time. Back-end personnel perform target recognition or use algorithms to assist in recognition. After a suspicious target is found, the drone is then manually controlled to perform actions such as issuing a warning, hovering, and collecting evidence.

[0003] However, when existing technical solutions identify a target scene and require on-site handling, the drones usually need to make temporary judgments based on their experience and issue control commands item by item, depending on the specific actions to be performed, the order in which they should be performed, and the parameters of each action to be configured. Different personnel have different judgment standards and operating habits, resulting in a lack of uniformity and standardization in the handling actions. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a drone inspection method, a drone, a cloud platform, and a drone inspection system to at least partially improve the above-mentioned problems.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows: In a first aspect, embodiments of the present invention provide a method for inspecting unmanned aerial vehicles (UAVs), applied to UAVs, the method comprising: It flies along a preset route and collects image data of the inspection area in real time. The image data is analyzed according to a preset target feature library. When a target scene that meets the preset conditions is identified, an alarm signal including a target identification label, location coordinates and a timestamp is generated. Based on the target identification tag and location coordinates in the alarm signal, the corresponding automated handling process is matched from the preset handling rule base; According to the automated handling process, it autonomously executes one or more on-site handling actions for the target scenario and collects evidence data; The evidence data is transmitted back to the cloud platform, and the preset flight path is resumed.

[0006] Optionally, the target feature library includes at least one preset target recognition label and target feature data corresponding to each target recognition label; the analysis of the image data according to the preset target feature library includes: The visual algorithm model is invoked to extract image features from the image data; The image features are compared with the target feature data in the target feature library; If a match is successful, the corresponding target identification label and identification confidence level will be output. When the recognition confidence level meets the preset threshold condition, it is determined that a target scene that meets the preset condition has been identified.

[0007] Optionally, the processing rule base includes at least one mapping relationship between target identification tags and automated processing procedures; Each of the automated processing procedures is composed of at least one basic execution module combined according to a preset execution sequence; The basic execution module includes at least one or more of the following: a public warning module, a coordinate positioning module, a circumferential inspection module, a fixed-point hovering module, a flight control module, and a photographic evidence collection module.

[0008] Optionally, the step of matching the corresponding automated handling process from a preset handling rule base based on the target identification tag and location coordinates in the alarm signal includes: Based on the location coordinates in the alarm signal, determine the control area where the target scene is located, and read the pre-configured handling strategy of the control area; If the handling strategy allows handling, then the corresponding automated handling process is matched from the handling rule base according to the target identification tag in the alarm signal.

[0009] Optionally, the step of autonomously executing one or more on-site handling actions for the target scenario according to the automated handling process, and collecting evidence data, includes: In response to the automated handling process, the flight mode is switched from cruise mode to handling mode, the current route is interrupted and the aircraft flies to a position above the coordinates. According to the preset module combinations and execution sequence in the automated handling process, the parameter configurations corresponding to each basic execution module are loaded sequentially, and the corresponding on-site handling actions are executed; wherein: If the automated processing procedure includes a fixed-point hovering module, then the fixed-point hovering is executed; If the automated processing procedure includes a flight control module, the fuselage orientation and / or gimbal orientation are adjusted according to preset angle parameters or the offset of the target scene in the image. If the automated handling process includes a warning module, then the onboard loudspeaker will be invoked to play a warning message; If the automated processing procedure includes a shooting and evidence collection module, then the airborne camera device is controlled to perform multi-angle and multi-scale shooting to collect image data and / or video data; If the automated processing procedure includes a surround inspection module, then the system will perform a surround flight around the target scene and collect panoramic data. If the automated processing procedure includes a coordinate positioning module, then the positioning coordinates of the target scene are acquired and recorded; Once all modules have been executed, switch the flight mode back to cruise mode.

[0010] Optionally, before the step of transmitting the evidence data back to the cloud platform and resuming the execution of the preset flight route, the method further includes: The evidence data is subjected to quality verification; the quality verification includes at least integrity verification and clarity verification. If the verification is successful, the evidence data will be sent back to the cloud platform, and the preset flight path will be resumed. If the verification fails, repeat each of the on-site handling actions and re-collect evidence data until the verification passes or the preset maximum number of retries is reached.

[0011] Secondly, embodiments of the present invention provide a drone inspection method applied to a cloud platform, the method comprising: Receive image data of the inspection area collected in real time by the drone; The image data is analyzed according to a preset target feature library. When a target scene that meets the preset conditions is identified, an alarm signal including a target identification label, location coordinates and a timestamp is generated. Based on the target identification tag and location coordinates in the alarm signal, the corresponding automated handling process is matched from the preset handling rule base; The automated processing procedure is then sent to the drone.

[0012] Thirdly, embodiments of the present invention provide an unmanned aerial vehicle (UAV) including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the method as described in any of the first aspects.

[0013] Fourthly, embodiments of the present invention provide a cloud platform, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the method described in the second aspect.

[0014] Fifthly, embodiments of the present invention provide an unmanned aerial vehicle (UAV) inspection system, including a UAV and a cloud platform; The drone flies along a preset route and collects and transmits image data of the inspection area back to the cloud platform in real time. The cloud platform analyzes the image data according to a preset target feature library. When a target scene that meets the preset conditions is identified, an alarm signal is generated, including a target identification label, location coordinates, and a timestamp. The cloud platform matches the corresponding automated handling process from a preset handling rule library based on the target identification tag and location coordinates in the alarm signal, and sends it to the drone. The drone autonomously executes one or more on-site handling actions for the target scenario according to the automated handling process, and collects evidence data. The drone transmits the evidence data back to the cloud platform and resumes executing the preset flight path.

[0015] This invention provides a drone inspection method, drone, cloud platform, and drone inspection system. By pre-constructing a target feature library and a handling rule library, a mapping relationship is established between target identification tags and automated handling processes. This enables the drone to automatically match and execute the corresponding handling process after identifying a target scene, eliminating the need for back-end personnel to make temporary judgments based on experience and issue control commands item by item. This effectively avoids the problem of inconsistent and non-standard handling actions caused by differences in human judgment standards and operating habits.

[0016] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a drone inspection method provided in an embodiment of the present invention. Figure 2 A flowchart illustrating the target recognition process provided in an embodiment of the present invention; Figure 3 A flowchart illustrating a quality verification process provided in an embodiment of the present invention; Figure 4 This is another flowchart illustrating a drone inspection method provided in an embodiment of the present invention; Figure 5 This is a schematic structural block diagram of an unmanned aerial vehicle (UAV) inspection system provided in an embodiment of the present invention.

[0019] Icons: 10 - Drone inspection system; 100 - Drone; 200 - Cloud platform. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0021] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0022] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0023] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0024] In existing technologies, drone inspection systems typically employ an image acquisition plus backend recognition working mode. The drone is only responsible for collecting images along its flight path and transmitting the image data back to a backend server or cloud platform in real time. Backend personnel then perform target recognition or use algorithms to assist in the recognition process. Upon detecting a suspicious target, backend personnel need to make temporary judgments based on the recognition results regarding which actions should be taken, in what order, and how to configure the parameters for each action. They then issue control commands to the drone item by item: first, let the drone fly over; then, let it hover; then, adjust the angle; then, issue a verbal warning; then, take a picture… Each step requires manual intervention and the issuance of individual commands, which the drone receives and responds to one by one.

[0025] This working model has at least two shortcomings: First, the selection and arrangement of handling actions rely entirely on the personal experience and subjective judgment of the back-end personnel. Different personnel have different operating standards and habits, which often leads to inconsistent and non-standard handling actions in the same scenario, making it difficult to form a standardized closed-loop supervision process. Second, the serial interaction method from the completion of identification to the issuance of instructions item by item and the drone responding item by item makes the handling process time-consuming. Especially in scenarios that require the continuous execution of multiple actions such as hovering, zoom shooting, surround evidence collection, and coordinate locking, the connection between actions needs to rely on manual triggering item by item, which prolongs the entire handling process and makes it difficult to complete the closed-loop response in a short time.

[0026] Based on the above, embodiments of the present invention provide a drone inspection method, a drone, a cloud platform, and a drone inspection system, which enable the drone to automatically match the corresponding handling process and execute it autonomously after identifying a target, without waiting for back-end personnel to issue instructions item by item, and without relying on the experience judgment of back-end personnel, thereby improving the standardization and response efficiency of inspection and handling.

[0027] The following is an exemplary description of the UAV inspection method provided in the embodiments of the present invention. This UAV inspection method is applied to UAVs, see [link to documentation]. Figure 1 The method includes, for example Figure 1 The following steps are described: S11: Flies along a preset route and collects image data of the inspection area in real time.

[0028] Before performing an inspection mission, the drone is pre-loaded with planned flight path data. This data includes a series of parameters such as waypoint coordinates, flight altitude, and flight speed. After takeoff, the drone flies autonomously along the preset path, with its onboard camera continuously capturing images of the ground or target area in real time. The image data can include visible light images, infrared images, or a fusion of both, depending on the needs of the inspection scenario. For example, in urban management scenarios, a high-definition visible light camera can be used to acquire images; in nighttime or heavily obstructed scenarios, an infrared camera can be used to assist in identification.

[0029] The image data acquisition process described above is continuous, and each frame of image data acquired is sent to the subsequent recognition stage for processing.

[0030] S12: Analyze the image data according to the preset target feature library, and when a target scene that meets the preset conditions is identified, generate an alarm signal including target identification label, location coordinates and timestamp.

[0031] The UAV's onboard processing unit calls upon the target feature library stored in the onboard storage unit to perform real-time analysis of the acquired image data.

[0032] The target feature library is pre-built and contains at least one target identification label and the target feature data corresponding to each label. The target identification label is a name given to the type of target scene to be of interest, such as "sewage pipe," "illegal construction," "garbage dump," "wild swimming," and "livestock shed." The target feature data describes the visual features of this type of target in the image and can include color features, texture features, shape features, and contour features. The target feature library can be constructed by first collecting a large number of labeled sample images of various target scenes, and then training the sample images using machine learning algorithms (such as deep learning neural networks) to extract high-dimensional feature vectors for various targets, forming a feature database for subsequent comparison.

[0033] The image data is analyzed and identified based on a pre-set target feature library. Upon successful identification, the airborne processing unit generates a structured alarm signal. The alarm signal includes at least the following three fields: target identification tag (indicating the type of scene identified, such as "sewage pipe"), location coordinates (indicating the current geographical location of the target, obtained in real time by the airborne positioning module), and timestamp (indicating the time when the identification occurred).

[0034] Optionally, the alarm signal may also include a screenshot and / or information such as identification confidence level for subsequent review and verification.

[0035] S13: Based on the target identification tag and location coordinates in the alarm signal, match the corresponding automated handling process from the preset handling rule base.

[0036] After generating an alarm signal, the onboard processing unit of the UAV reads the target identification tag and location coordinates in the alarm signal, and then uses the target identification tag as an index to query the preset handling rule library.

[0037] The handling rule base is a set of rules pre-built and stored in the onboard storage unit. The handling rule base contains at least one mapping relationship between target identification tags and automated handling processes. That is, the handling rule base pre-defines the correspondence of "if target X is identified, then process Y is executed".

[0038] S14: In accordance with the automated handling process, autonomously execute one or more on-site handling actions for the target scenario and collect evidence data.

[0039] After matching the corresponding automated handling procedure, the UAV's onboard processing unit switches the flight mode from cruise to handling mode, interrupts the current flight path, and flies to a position above the coordinates indicated by the alarm signal. The process of flying to the target location can be completed autonomously by the flight control system based on real-time positioning data.

[0040] The automated processing flow consists of at least one basic execution module combined according to a preset execution sequence. The basic execution module is a standardized encapsulation of the basic maneuver capabilities of the UAV, and each module corresponds to a set of independently callable action commands and parameter configurations.

[0041] Throughout the process, the image and / or video data collected by the drone is temporarily stored in the onboard storage unit as evidence. The evidence data may include panoramic images, detailed images, surround video of the target scene, and corresponding positioning coordinates and timestamp information.

[0042] S16: Send the evidence data back to the cloud platform and resume the execution of the preset flight path.

[0043] After completing all modules in the automated handling process, the drone will switch its flight mode from handling state back to cruise state and resume the interrupted preset route to continue completing the subsequent inspection tasks.

[0044] Meanwhile, the drones transmit the collected evidence data back to the cloud platform. After receiving the evidence data, the cloud platform associates it with the corresponding alarm signals and stores it to form a complete inspection work order, which can be viewed, reviewed, and archived by the back-end management personnel.

[0045] It should be noted that the execution order of "returning evidence-gathering data" and "restoring the preset flight route" can be flexibly configured by the system according to the actual situation. In one optional implementation, the data can be compressed, packaged, and returned before restoring the flight route to ensure that the processing results have been saved. In another optional implementation, given the widespread use of high-speed communication technologies such as 5G, the data return time is extremely short, and data return can also be performed simultaneously with the restoration of the flight route, with the two processes running in parallel without interference. This invention does not strictly limit the timing relationship between the two processes, as long as data reporting is completed within a reasonable time before or after the resumption of flight.

[0046] This method forms a complete closed-loop operation process, from "flying along a preset route to collect images" to "target recognition," "matching of handling rules," "automated process execution," and "evidence collection and feedback to resume patrol." After identifying a target, the UAV can automatically match and execute the handling process corresponding to the target scenario without waiting for instructions from back-end personnel or relying on their experience. This effectively improves the standardization and response efficiency of inspection and handling.

[0047] The target feature library can be pre-stored in the onboard storage unit of the UAV, including at least one preset target identification tag and the target feature data corresponding to each target identification tag.

[0048] The target identification labels are scene type names customized by managers based on actual inspection needs. For example, in a livestock and poultry breeding pollution inspection scenario, labels such as "breeding sheds," "sewage pipes," "sewage," and "open-air manure" can be defined; in an urban management scenario, labels such as "illegal construction," "garbage dumps," and "street vending" can be defined; and in an emergency inspection scenario, labels such as "fire spots," "smoke," and "water accumulation" can be defined. The specific categories and number of labels can be flexibly configured according to the application scenario, and this invention does not limit this.

[0049] Target feature data is a dataset describing the visual features of a corresponding target scene. In this embodiment of the invention, the target feature data is pre-extracted based on a deep learning algorithm. Specifically, it is constructed as follows: a large number of image samples containing various target scenes are collected, labeled by professionals, and then a deep learning neural network (such as a convolutional neural network) is used to train the labeled samples to extract high-dimensional feature vectors for various targets. After training, the feature vectors corresponding to various targets are stored in an onboard storage unit, forming a target feature library. The target feature library can be updated periodically to adapt to changes in the visual features of targets under different seasons and lighting conditions.

[0050] See Figure 2 , Figure 2 This is a flowchart illustrating the target recognition process provided in an embodiment of the present invention. The analysis of image data based on a preset target feature library includes: S121: Call the visual algorithm model to extract image features from the image data.

[0051] The system invokes a pre-deployed visual algorithm model on the onboard processing unit to extract image features from the current frame's image data. The visual algorithm model can be a lightweight deep learning model, ensuring recognition accuracy while taking into account the limited computing resources of the onboard unit.

[0052] S122: Compare the image features with the target feature data in the target feature library.

[0053] The extracted image features are compared one by one with the target feature data in the target feature library. The comparison method can be to calculate the similarity between the image feature vector and each target feature vector (such as cosine similarity or Euclidean distance). If the similarity corresponding to a certain target exceeds a preset matching threshold, the match is considered successful.

[0054] S123: If the match is successful, output the corresponding target identification label and identification confidence level.

[0055] If a match is successful, the corresponding target identification label and identification confidence score will be output. The identification confidence score can be mapped based on the specific numerical value of the similarity; the higher the similarity, the higher the confidence score.

[0056] S124: When the recognition confidence level meets the preset threshold condition, it is determined that the target scene that meets the preset condition has been identified.

[0057] The preset conditions are: the same target scene is stably identified in three consecutive frames, and the recognition confidence of each frame is not less than 85%. Using continuous multi-frame confirmation instead of single-frame confirmation can effectively avoid momentary misidentification caused by accidental factors and improve the reliability of target recognition.

[0058] It should also be noted that, in order to facilitate the deployment and continuous optimization of the visual algorithm model, the construction and updating of the target feature library can be completed in the cloud and then periodically distributed to each drone terminal via wireless communication link.

[0059] In one possible implementation, the handling rule base includes at least one mapping relationship between target identification tags and automated handling processes; each automated handling process is composed of at least one basic execution module combined according to a preset execution sequence; the basic execution module includes at least one or more of the following: a verbal warning module, a coordinate positioning module, a surround inspection module, a fixed-point hovering module, a flight control module, and a photographic evidence collection module.

[0060] The handling rule base can be pre-stored in the drone's onboard storage unit, and its core function is to realize the automated mapping of "target scenario and standardized handling process".

[0061] The disposal rule base contains at least one mapping relationship between target identification tags and automated disposal processes. For example, the target identification tag "illegal construction" is mapped to automated disposal process A; the target identification tag "garbage dump" is mapped to automated disposal process B; and the target identification tag "wild swimming" is mapped to automated disposal process C.

[0062] Each automated processing flow consists of at least one basic execution module combined according to a preset execution sequence. The basic execution module is a standardized encapsulation of a single action capability of the UAV. In this embodiment of the invention, the following six basic execution modules are predefined, each corresponding to a set of configurable execution parameters: (1) Warning Module: Used to play warning messages through the onboard speaker. Configuration parameters include: voice text content, volume (0-100% stepless adjustment), number of plays, loop interval, etc. Multiple sets of differentiated broadcast texts can be preset for different types of scenarios, such as "Swimming is prohibited here, please get ashore immediately" or "You have entered an area with illegal construction, please cooperate with the handling", etc.

[0063] (2) Coordinate positioning module: used to acquire and record the precise spatial coordinates of the target scene. Configuration parameters include: positioning accuracy requirements, fusion positioning strategy (such as BeiDou / GPS dual-mode fusion), coordinate saving format, etc.

[0064] (3) Orbital Inspection Module: Used to control the UAV to perform orbital flight around the target scene. Configuration parameters include: orbital radius, number of orbital layers, orbital direction, flight speed, shooting frequency, etc.

[0065] (4) Fixed-point hovering module: used to control the UAV to hover above the target position. Configuration parameters include: hovering time, hovering altitude, position holding accuracy, etc.

[0066] (5) Flight control module: used to adjust the UAV's fuselage orientation and gimbal orientation. Configuration parameters include: yaw angle, gimbal yaw angle, gimbal pitch angle, and adjustment speed.

[0067] (6) Shooting and Evidence Collection Module: Used to control the airborne camera device to perform shooting tasks. Configuration parameters include: shooting order (panorama first, then detail or vice versa), zoom magnification sequence (such as 1×, 3×, 5×), number of shots, shooting interval, image resolution, video duration, etc.

[0068] The above-described division and parameter configuration of the basic execution modules are merely illustrative examples. Those skilled in the art can make appropriate adjustments to the granularity, number, and parameters of the modules according to the needs of actual application scenarios, without departing from the basic concept of the present invention.

[0069] The automated processing workflow is constructed as follows: For each target scenario, managers select applicable modules from the aforementioned basic execution modules based on regulatory requirements, and arrange them sequentially according to the execution order to form a complete work process. After the workflow is constructed, it is bound to the corresponding target identification tag, forming a mapping record in the rule base.

[0070] For example, in the case of "illegal construction", since it is necessary to obtain clear information on the structure and location of the illegal construction, the process can be configured as "flight control module (adjusting the fuselage and gimbal to center the main body of the illegal construction), fixed-point hovering module (maintaining a stable shooting platform), shooting and evidence collection module (multi-level zoom shooting of panoramic and detailed images), coordinate positioning module (locking and recording the coordinates of the illegal construction)", forming an automated handling process A.

[0071] For example, in the "garbage dump" scenario, since the main task is to record the size and location of the garbage dump, the process can be configured as "fixed-point hovering module (hovering directly above the garbage dump), shooting and evidence collection module (shooting panoramic and detailed images), coordinate positioning module (locking the coordinates of the garbage dump)", forming an automated disposal process B.

[0072] For example, in the case of "wild swimming", since the main task is to drive people away and record the situation on site, the process can be configured as "shouting warning module (playing warning voice to ask people to come ashore) and surrounding inspection module (circling and flying to take panoramic pictures to record the situation on site)", forming an automated handling process C.

[0073] In one possible implementation, step S13 may include: S131: Based on the location coordinates in the alarm signal, determine the control area where the target scene is located, and read the pre-configured handling strategy of the control area.

[0074] S132: If the handling policy is to allow handling, then match the corresponding automated handling process from the handling rule base according to the target identification tag in the alarm signal.

[0075] After matching the target identification tag to the corresponding automated handling process, the drone also needs to determine the controlled area based on its location coordinates. The onboard storage unit pre-stores electronic fence data, which divides the inspection area into different controlled areas. Different types of controlled areas are pre-configured with different handling strategies: for example, the handling strategy configured for "alarm areas" is "allowed handling," meaning that subsequent handling actions should be triggered when a target scene is identified in this area; the handling strategy configured for "legal areas" is "prohibited handling," meaning that no subsequent handling actions need to be triggered when a target scene is identified in this area, to avoid generating invalid alarms and unnecessary handling actions in compliant areas (such as planned compliant breeding areas or legal operation areas). Based on the location coordinates in the alarm signal and the pre-stored electronic fence data, the drone determines the controlled area where the target scene is located and reads the pre-configured handling strategy for that area. If the handling strategy is "allowed handling," the subsequent matching and distribution steps are executed; if the handling strategy is "prohibited handling," the current handling process is terminated, no subsequent actions are executed, and the drone continues to fly along its original route.

[0076] Through the aforementioned area judgment mechanism, embodiments of the present invention can effectively block invalid alarms within legal and compliant areas, preventing drones from performing actions such as broadcasting, hovering, and evidence collection in areas where no intervention is required, thereby saving drone energy consumption and storage resources, while reducing interference with normal activities in compliant areas.

[0077] After the onboard processing unit matches the corresponding automated handling procedure from the handling rule base and approves the handling based on the area determination, the UAV enters the execution phase. In one possible implementation, S14 may include: S141: In response to the automated handling process, switch the flight mode from cruise to handling mode, interrupt the current route and fly above the position coordinates.

[0078] S142: Based on the preset module combinations and execution sequence in the automated handling process, load the parameter configurations corresponding to each basic execution module sequentially and execute the corresponding on-site handling actions; wherein: If the automated processing flow includes a fixed-point hovering module, then execute the fixed-point hovering. If the automated handling process includes a flight control module, the fuselage orientation and / or gimbal orientation are adjusted according to preset angle parameters or the offset of the target scene in the image. If the automated handling process includes a loudspeaker warning module, then the onboard loudspeaker will be invoked to play the warning message. If the automated processing procedure includes a shooting and evidence collection module, then the airborne camera device is controlled to perform multi-angle and multi-scale shooting to collect image data and / or video data; If the automated processing flow includes a surround inspection module, then it will perform a surround flight around the target scene and collect panoramic data; If the automated processing flow includes a coordinate positioning module, then the positioning coordinates of the target scene are acquired and recorded.

[0079] S143: After all modules have been executed, switch the flight mode back to cruise mode.

[0080] First, the drone switches its flight mode from cruise to emergency response. In cruise mode, the drone flies smoothly along a preset route with waypoints and speed. In emergency response mode, the drone uses the target location as a navigation point, interrupts its current flight path, adjusts its course, and flies to a position above the coordinates indicated in the warning signal. During flight, the onboard positioning module continuously provides real-time position feedback to ensure the drone accurately reaches the target location.

[0081] Upon reaching the target location, the drone, according to the pre-set module combinations and execution sequence in the automated handling process, sequentially loads the parameter configurations corresponding to each basic execution module, driving the relevant systems to execute the corresponding on-site handling actions. The execution methods of each basic execution module are as follows: Execution of the hovering module: After receiving the hovering command, the flight control system controls the UAV to maintain a hovering state directly above the target location. During hovering, the flight control system uses multi-source sensor data, including the onboard barometer, GPS / BeiDou positioning module, and visual odometry, to perform fusion positioning, resisting environmental wind disturbances and ensuring that the position deviation remains within a preset allowable range. The hovering time is preset by process parameters and can be set to a fixed duration (e.g., 15 seconds, 30 seconds, etc.) or can be set to continue until a trigger signal from a subsequent module is received. Hovering provides a reliable platform guarantee for subsequent actions that require high platform stability, such as zoom shooting and precise voice communication.

[0082] Flight control module execution: The flight control module is used to fine-tune the orientation of the drone's fuselage and gimbal, aiming to lock the target scene in the center of the frame and provide the best framing angle for subsequent evidence collection. There are two adjustment methods: one is adjustment based on preset angle parameters, which directly loads preset values ​​such as yaw angle, gimbal yaw angle, and gimbal pitch angle from the process, simultaneously adjusting the fuselage and gimbal orientation to center the target in the frame; the other is closed-loop adjustment based on the actual offset of the target scene in the frame, which uses image recognition algorithms to detect the pixel position of the target in the frame in real time, calculates its offset from the frame center, and then generates corresponding adjustment commands to rotate the gimbal or fuselage towards the target until the target is centered in the frame. These two adjustment methods can be used in combination: first, load the preset angles for coarse adjustment, and then fine-tune based on the frame offset to ensure final positioning accuracy.

[0083] The execution of the public address warning module: The onboard processing unit retrieves the matching voice text from the pre-stored voice library based on the target recognition tag, and then performs speech synthesis or loads a pre-recorded audio file through the audio processing module. Subsequently, the onboard loudspeaker begins to play the warning voice in a loop. Playback parameters (such as volume, number of loops, etc.) are determined by the process configuration. Different warning content is played for different scenarios, such as "Deep water, danger, do not swim, please get ashore immediately" for wild swimming scenarios, and "Open burning is prohibited, please extinguish the fire source immediately" for open burning scenarios. During the public address system, the pan-tilt camera device can simultaneously capture on-site video, recording the personnel's reactions and the on-site situation, forming a complete chain of evidence for the handling of the incident.

[0084] The execution of the evidence collection module: The onboard processing unit sends shooting control commands to the gimbal system, controlling the onboard camera to perform multi-angle, multi-scale shooting. The shooting process proceeds according to the preset shooting sequence: First, the camera is controlled to capture a panoramic image of the target scene and its surrounding environment at a wide-angle or low zoom level (e.g., 1×), recording the overall environmental background of the target; then, the optical zoom is gradually increased (e.g., switching to 3×, 5×), capturing mid-range images and detailed feature images of the target subject layer by layer (e.g., the diameter of the sewage outlet pipe, structural nodes of illegal buildings, components of the garbage dump, etc.). While capturing each image, auxiliary information such as the positioning coordinates, gimbal angle, and zoom level at the time of shooting is simultaneously recorded, forming a complete evidence data package together with the image itself. In addition to still images, the evidence collection module can also control the camera to record a short video to dynamically record the actual situation of the target scene.

[0085] The orbital inspection module is used to comprehensively collect information about the surrounding environment of the target scene. The flight control system and gimbal system work together to control the drone to orbit around the target location at a preset radius (e.g., 5 meters, 10 meters, 20 meters, etc.). During the orbit, the gimbal camera always points towards the target center, continuously collecting panoramic video or capturing multiple orbital images at a fixed distance. The number of orbital inspection layers (i.e., the number of orbits) and the direction (clockwise or counterclockwise) can both be preset.

[0086] Execution of the coordinate positioning module: The coordinate positioning module is used to acquire high-precision spatial coordinates of the target scene. The airborne positioning module integrates dual-mode positioning signals from the BeiDou Navigation Satellite System and the GPS Navigation Satellite System for joint calculation, and combines clues provided by airborne visual images (such as the pixel coordinates of the target in the image and its corresponding field of view information) to perform visual-assisted correction on the positioning results, ultimately generating accurate spatial coordinates of the target scene. The acquired coordinates are associated with and stored with target identification tags, timestamps, and other information as part of the evidence data for subsequent precise positioning and spatial analysis.

[0087] During execution, the aforementioned modules can also be executed in parallel or interleaved depending on the configuration of the automated handling process. For example, during a perimeter inspection, the image capture module can be invoked simultaneously to take photos at intervals; during a verbal warning process, the image capture module can be invoked simultaneously to record video.

[0088] After each module completes its execution sequentially or collaboratively according to a preset time sequence, the airborne processing unit collects the image data, video data, coordinate data, audio recordings, etc., acquired by each module into a unified evidence collection data package, which is then temporarily stored in the airborne storage unit.

[0089] Once all modules have completed their execution, the onboard processing unit switches the drone's flight mode back to cruise mode, preparing to resume executing the preset route.

[0090] During actual inspections, factors such as changes in ambient light, drone motion, and gimbal focusing delays can lead to issues with the collected evidence data (especially images and videos), including blurriness, overexposure, underexposure, misalignment of targets, and incomplete content. Directly transmitting and archiving evidence data with these defects would render it ineffective, wasting storage and transmission resources. Furthermore, it could necessitate deploying drones for reshoots, resulting in additional manpower and resources, severely impacting inspection efficiency and hindering the formation of a closed-loop regulatory system.

[0091] Therefore, this embodiment of the invention includes a quality verification step before the evidence data is transmitted back, whereby the onboard processing unit performs real-time automatic quality checks on the collected evidence data. See also Figure 3 , Figure 3 This is a flowchart illustrating a quality verification process provided in an embodiment of the present invention. Before the steps of transmitting the evidence data back to the cloud platform and resuming the execution of the preset flight route, the method further includes: S15: Perform quality verification on the evidence data; the quality verification includes at least integrity verification and clarity verification. If the verification is successful, proceed to step S16; if the verification fails, re-execute each on-site handling action and re-collect evidence data until the verification is successful or the preset maximum number of retries is reached.

[0092] Quality verification includes at least two aspects: integrity verification and sharpness verification.

[0093] The integrity check is used to verify whether all preset evidence collection items have been completed. The airborne processing unit reads the configuration parameters of the image capture and evidence collection module and the surround inspection module in the automated processing flow, obtains the preset list of evidence collection items (such as 1 panoramic image, 3 detailed images, 1 surround video segment, 1 coordinate record, etc.), and then iterates through and checks whether the evidence collection data packet contains all the items listed, and whether the file size of each item is greater than zero (excluding empty files). If any item is missing, the integrity check fails.

[0094] Sharpness verification is used to check whether the acquired images and videos are sharp and usable. The onboard processing unit automatically evaluates the image data using a sharpness evaluation algorithm. This algorithm calculates a sharpness score based on parameters such as the image's grayscale gradient, edge strength, and frequency domain energy. When the sharpness score is below a preset threshold, the image is considered blurry. Common sharpness evaluation metrics include, but are not limited to, image grayscale variance method, Brenner gradient method, Tenengrad gradient method, and Laplacian variance method. For video data, several keyframes can be extracted from the video stream, and sharpness is evaluated for each frame. If the sharpness of all or most keyframes is satisfactory, the video is considered sharp. Furthermore, sharpness verification can also include detection of exposure levels (e.g., whether there are large areas of overexposure or underexposure) and target centering detection (e.g., whether the target is in the center of the frame).

[0095] If both the integrity and clarity checks are passed, it means that the quality of the evidence data meets the standards. The drone will then transmit the evidence data back to the cloud platform and resume executing the preset flight path.

[0096] If either the integrity check or the sharpness check fails, it indicates a quality defect in the evidence data. The drone will then re-execute at least some of the on-site handling actions and re-collect the evidence data before performing the quality check again. For example, if only the sharpness fails, only the image capture module will be re-executed to re-capture the target image at the hovering position, without needing to re-execute the hovering module and flight control module, thus saving time. If the integrity fails (e.g., missing surround video), the surround inspection module will be re-executed to capture panoramic footage. If the re-captured footage is still unsatisfactory, the retry scope will be expanded, and all on-site handling actions will be re-executed.

[0097] The retry process continues until the verification is successful. To prevent infinite loops caused by objective environmental factors (such as persistent heavy fog, target obstruction, etc.), this embodiment of the invention also sets a preset upper limit for the number of retryes (e.g., 3 times). When the upper limit is reached and the verification still fails, the UAV stops retrying, and sends the collected evidence data (regardless of whether it is successful) along with the retry record back to the cloud platform, marking the status information "quality verification failed" in the returned data, which is then manually reviewed by the backend administrators. This ensures automatic closed-loop operation in most cases, while also reserving a channel for manual intervention in extreme situations.

[0098] Through the aforementioned quality verification and automatic retry mechanism, the embodiments of the present invention can effectively ensure the standardization and availability of evidence data, avoid invalid evidence collection due to shooting quality issues, and significantly improve the success rate of evidence collection and the completeness of the regulatory loop.

[0099] Furthermore, this embodiment of the invention also provides a drone inspection method applied to a cloud platform, see [link to relevant documentation]. Figure 4 The method includes: S21: Receives image data of the inspection area collected in real time by the drone.

[0100] S22: Analyze the image data according to the preset target feature library, and when a target scene that meets the preset conditions is identified, generate an alarm signal including target identification label, location coordinates and timestamp.

[0101] S23: Based on the target identification tag and location coordinates in the alarm signal, match the corresponding automated handling process from the preset handling rule base.

[0102] S24: Deploy automated processing procedures to drones.

[0103] Unlike the aforementioned schemes where the onboard drone independently completes identification, matching, and execution, the identification and matching steps of this drone inspection method are executed by the cloud platform, making it suitable for inspection scenarios with limited onboard computing power or requiring centralized cloud management.

[0104] During inspection missions, the drone flies along a pre-set route, collecting real-time image data of the inspected area using onboard cameras. This data is then transmitted back to a cloud platform via wireless communication links (such as 4G, 5G, or a dedicated data link). The cloud platform continuously receives the image data stream from the drone through a data receiving interface. Unlike onboard solutions, in this method, the drone is only responsible for data collection and transmission, without performing onboard identification processing.

[0105] After receiving image data, the cloud platform invokes a visual algorithm model deployed in the cloud to extract image features. These features are then compared with target feature data in a target feature library, outputting the corresponding target recognition label and recognition confidence level. When the recognition confidence level meets a preset threshold, the target scene that meets the preset conditions is determined to have been identified.

[0106] The composition of the target feature library and the specific implementation methods for image feature extraction and comparison are consistent with those described above and will not be repeated here. It should be noted that cloud platforms have more abundant computing resources than onboard devices, enabling the deployment of larger-scale, higher-precision visual models, and can utilize GPU clusters for parallel acceleration of inference.

[0107] Upon successful identification, the cloud platform generates a structured alarm signal. The alarm signal includes at least the following three fields: target identification label, location coordinates, and timestamp.

[0108] In one embodiment of the present invention, after identifying the target, the cloud platform can also determine the handling strategy for the controlled area where the target scene is located based on the location coordinates in the alarm signal and the electronic fence data pre-stored in the cloud platform. If the handling strategy is to prohibit handling, the cloud platform terminates the process and does not issue any handling instructions.

[0109] The cloud platform uses the target identification tag in the alarm signal as a query index to match the corresponding automated handling process from the handling rule base stored in the cloud platform's storage system.

[0110] The cloud platform sends the matched automated handling process to the corresponding drone via wireless communication link in one go. The sent content includes: a module list, the execution sequence of each module, and the parameter configuration of each module. After receiving the complete automated handling process, the drone autonomously performs on-site handling actions according to the preset module combinations and execution sequence in the process, and collects evidence data. After the handling is completed, the drone sends the evidence data back to the cloud platform for archiving.

[0111] Furthermore, embodiments of the present invention also provide an unmanned aerial vehicle (UAV) inspection system, see [link to relevant documentation]. Figure 5 The drone inspection system 10 includes a drone 100 and a cloud platform 200.

[0112] The drone 100 flies along a preset route, collecting and transmitting image data of the inspection area back to the cloud platform 200 in real time.

[0113] The cloud platform 200 analyzes image data based on a preset target feature library. When a target scene that meets the preset conditions is identified, an alarm signal is generated, including the target identification label, location coordinates, and timestamp.

[0114] Based on the target identification tag and location coordinates in the alarm signal, the cloud platform 200 matches the corresponding automated handling process from the preset handling rule library and sends it to the drone 100.

[0115] The drone 100 autonomously executes one or more on-site handling actions for the target scenario according to the automated handling process, and collects evidence data.

[0116] The drone 100 will transmit the evidence data back to the cloud platform 200 and resume executing the preset flight path.

[0117] In summary, the UAV inspection method, UAV, cloud platform, and UAV inspection system provided by this invention establish a mapping relationship between target identification tags and automated handling processes by pre-constructing a target feature library and a handling rule library. This enables the UAV to automatically match and execute the corresponding handling process after identifying a target scene, eliminating the need for back-end personnel to make temporary judgments based on experience and issue control commands item by item. This effectively avoids the problems of inconsistent and non-standard handling actions caused by differences in human judgment standards and operating habits. By setting up quality verification and automatic retry mechanisms on the airborne end, the standardization and availability of evidence data are ensured. The configuration of control areas and handling strategies can block invalid alarms within legal and compliant areas, saving UAV energy consumption and communication resources.

[0118] In the embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0119] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0120] If the functionality is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a computer-readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0121] The above are merely preferred embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0122] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for unmanned aerial vehicle (UAV) inspection, characterized in that, Applied to drones, the method includes: It flies along a preset route and collects image data of the inspection area in real time. The image data is analyzed according to a preset target feature library. When a target scene that meets the preset conditions is identified, an alarm signal including a target identification label, location coordinates and a timestamp is generated. Based on the target identification tag and location coordinates in the alarm signal, the corresponding automated handling process is matched from the preset handling rule base; According to the automated handling process, it autonomously executes one or more on-site handling actions for the target scenario and collects evidence data; The evidence data is transmitted back to the cloud platform, and the preset flight path is resumed.

2. The method according to claim 1, characterized in that, The target feature library includes at least one preset target identification label and target feature data corresponding to each target identification label; The step of analyzing the image data according to a preset target feature library includes: The visual algorithm model is invoked to extract image features from the image data; The image features are compared with the target feature data in the target feature library; If a match is successful, the corresponding target identification label and identification confidence level will be output. When the recognition confidence level meets the preset threshold condition, it is determined that a target scene that meets the preset condition has been identified.

3. The method according to claim 1, characterized in that, The processing rule base includes at least one mapping relationship between target identification tags and automated processing procedures; Each of the automated processing procedures is composed of at least one basic execution module combined according to a preset execution sequence; The basic execution module includes at least one or more of the following: a public warning module, a coordinate positioning module, a circumferential inspection module, a fixed-point hovering module, a flight control module, and a photographic evidence collection module.

4. The method according to claim 1, characterized in that, The step of matching the corresponding automated handling process from a preset handling rule base based on the target identification tag and location coordinates in the alarm signal includes: Based on the location coordinates in the alarm signal, determine the control area where the target scene is located, and read the pre-configured handling strategy of the control area; If the handling strategy allows handling, then the corresponding automated handling process is matched from the handling rule base according to the target identification tag in the alarm signal.

5. The method according to claim 1, characterized in that, The automated handling process autonomously executes one or more on-site handling actions for the target scenario and collects evidence data, including: In response to the automated handling process, the flight mode is switched from cruise mode to handling mode, the current route is interrupted and the aircraft flies to a position above the coordinates. According to the preset module combinations and execution sequence in the automated handling process, the parameter configurations corresponding to each basic execution module are loaded sequentially, and the corresponding on-site handling actions are executed; wherein: If the automated processing procedure includes a fixed-point hovering module, then the fixed-point hovering is executed; If the automated processing procedure includes a flight control module, the fuselage orientation and / or gimbal orientation are adjusted according to preset angle parameters or the offset of the target scene in the image. If the automated handling process includes a warning module, then the onboard loudspeaker will be invoked to play a warning message; If the automated processing procedure includes a shooting and evidence collection module, then the airborne camera device is controlled to perform multi-angle and multi-scale shooting to collect image data and / or video data; If the automated processing procedure includes a surround inspection module, then the system will perform a surround flight around the target scene and collect panoramic data. If the automated processing procedure includes a coordinate positioning module, then the positioning coordinates of the target scene are acquired and recorded; Once all modules have been executed, switch the flight mode back to cruise mode.

6. The method according to claim 1, characterized in that, Before the step of transmitting the evidence data back to the cloud platform and resuming the execution of the preset flight route, the method further includes: The evidence data is subjected to quality verification; the quality verification includes at least integrity verification and clarity verification. If the verification is successful, the evidence data will be sent back to the cloud platform, and the preset flight path will be resumed. If the verification fails, repeat each of the on-site handling actions and re-collect evidence data until the verification passes or the preset maximum number of retries is reached.

7. A method for unmanned aerial vehicle (UAV) inspection, characterized in that, Applied to a cloud platform, the method includes: Receive image data of the inspection area collected in real time by the drone; The image data is analyzed according to a preset target feature library. When a target scene that meets the preset conditions is identified, an alarm signal including a target identification label, location coordinates and a timestamp is generated. Based on the target identification tag and location coordinates in the alarm signal, the corresponding automated handling process is matched from the preset handling rule base; The automated processing procedure is then sent to the drone.

8. A drone, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 6.

9. A cloud platform, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in claim 7.

10. A drone inspection system, characterized in that, Including drones and cloud platforms; The drone flies along a preset route and collects and transmits image data of the inspection area back to the cloud platform in real time. The cloud platform analyzes the image data according to a preset target feature library. When a target scene that meets the preset conditions is identified, an alarm signal is generated, including a target identification label, location coordinates, and a timestamp. The cloud platform matches the corresponding automated handling process from a preset handling rule library based on the target identification tag and location coordinates in the alarm signal, and sends it to the drone. The drone autonomously executes one or more on-site handling actions for the target scenario according to the automated handling process, and collects evidence data. The drone transmits the evidence data back to the cloud platform and resumes executing the preset flight path.

Images