Automatic routing inspection system without air route

Abstract

The invention relates to the technical field of unmanned aerial vehicle control, and provides a route-free automatic inspection system, which comprises an unmanned aerial vehicle, an airborne box, an autonomous flight module, a visual acquisition module, an environment sensing module, a control instruction module and a position evaluation module, and is characterized in that the airborne box is loaded in the unmanned aerial vehicle so as to acquire environment data in the flight process of the unmanned aerial vehicle in real time; the control instruction module receives coordinates of a target area input by a user and issues the coordinates of the target area to the unmanned aerial vehicle, the environment sensing module obtains the coordinates of the target area and environment data and dynamically calculates a pre-planned path, and the position evaluation module performs evaluation according to the real-time position of the unmanned aerial vehicle and the pre-planned path to form an evaluation result. The environment sensing module is triggered to dynamically adjust a pre-planned path according to an evaluation result and adjust to form a flight route, the autonomous flight module controls the unmanned aerial vehicle according to the flight route, and the visual acquisition module identifies the state of an inspection object in real time and executes corresponding shooting or video recording operation.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle control, and in particular to an automatic inspection system without flight paths. Background Art

[0002] Traditional drone inspection systems usually rely on preset flight routes for operation. Although this method can achieve automated inspections to a certain extent, it has obvious limitations when dealing with complex and dynamic environments.

[0003] For example, the UAV inspection route template matching method disclosed in Chinese patent CN113253290A provides an inspection route template matching method, which can automatically match and generate UAV routes for tower inspection, reduce the workload of UAV inspection route planning, and improve the efficiency of UAV inspection route planning. However, it still uses route matching or route setting, resulting in the inability of the UAV to actively respond to sudden situations during flight, resulting in a decrease in adaptive ability and difficulty in flexible response.

[0004] In addition, the prior art still has the following defects:

[0005] 1. Traditional drone inspection systems usually require a detailed flight route to be set in advance, and the drone will conduct inspections according to this route. This method works well in fixed and simple environments, but it is difficult to respond flexibly in complex or dynamic environments. For example, sudden obstacles or changes in the location of the inspection object may cause the drone to be unable to complete the task, or even cause an accident;

[0006] 2. Traditional systems rely on pre-planned flight paths and operating instructions, and lack real-time environmental perception and decision-making capabilities. Once unplanned situations occur (such as obstacles or environmental changes), drones cannot respond effectively and need to rely on human intervention, which reduces the automation level and efficiency of inspections;

[0007] 3. Since drones need to fly according to preset routes, setting and adjusting these routes often requires manual intervention, which increases the complexity and time cost of operation. This manual operation method is inefficient and prone to errors when facing large-scale or frequent inspection tasks;

[0008] 4. Due to the limited automation level of traditional systems, inspection efficiency and reliability cannot be fully guaranteed, especially in scenarios that require high-frequency and high-precision inspections. Manually setting routes and handling unexpected situations is not only time-consuming, but also prone to omissions or misjudgments.

[0009] The present invention is made to solve the common problems in this field, such as poor adaptability, lack of real-time evaluation, poor inspection efficiency, weak reliability, frequent manual intervention, low intelligence, poor flexibility, lack of real-time decision-making and environmental perception capabilities, etc. Summary of the invention

[0010] The purpose of the present invention is to propose a route-free automatic inspection system to address the current deficiencies.

[0011] In order to overcome the shortcomings of the prior art, the present invention adopts the following technical solutions:

[0012] A route-free automated inspection system, comprising a server and a drone, the automated inspection system also comprising an airborne box, an autonomous flight module, a visual acquisition module, an environmental perception module, a control instruction module, and a position evaluation module, wherein the server is respectively connected to the airborne box, the autonomous flight module, the visual acquisition module, the environmental perception module, the control instruction module, and the position evaluation module;

[0013] The onboard box is loaded in the drone to collect environmental data in real time during the flight of the drone. The control instruction module is responsible for receiving the coordinates of the target area input by the user and sending the coordinates of the target area to the drone. The environmental perception module obtains the coordinates of the target area and the environmental data collected by the onboard box, and dynamically calculates the pre-planned path. The position evaluation module evaluates the real-time position of the drone and the pre-planned path to form an evaluation result, and triggers the environmental perception module to dynamically adjust the pre-planned path according to the evaluation result, and adjusts to form a flight route. The autonomous flight module controls the drone according to the flight route. The visual acquisition module recognizes the state of the inspection object in real time, and cooperates with the autonomous flight module to automatically perform shooting or video recording operations at at least two angles;

[0014] The autonomous flight module inspects the inspection objects in the target area according to the flight route.

[0015] Optionally, the control instruction module includes an interaction unit and a transmission unit, wherein the interaction unit allows the user to input the coordinates of the target area through a human-computer interaction interface, and the transmission unit transmits the coordinates of the target area input by the user to the drone, so that the drone dynamically grasps the coordinates of the target area;

[0016] The interactive unit includes an interactive touch panel and an executable program. The executable program is executed on the interactive touch panel to form a human-computer interaction interface for the administrator to input the target area on the human-computer interaction interface.

[0017] Optionally, the environment perception module includes a data acquisition unit, an environment sampling unit and an environment perception unit, the environment sampling unit acquires environment data sampled in real time during the flight of the UAV, the data acquisition unit acquires the target area coordinates acquired by the control instruction module, and the environment perception unit dynamically calculates the pre-planned path according to the target area coordinates acquired by the data acquisition unit and the real-time environment data acquired by the environment sampling unit;

[0018] The environmental sampling unit includes a sensor array, a data fusion device and a real-time monitoring component. The sensor array collects real-time environmental data during the process of the drone executing the coordinates of the target area. The data fusion device comprehensively processes the data collected by the sensor array to perceive the environment around the drone. The real-time monitoring component refreshes the data collected by the sensor array in real time.

[0019] Optionally, the environment perception unit dynamically calculates the planned path according to the following steps:

[0020] S1, obtaining the target area coordinates and real-time environmental data from the data acquisition unit and the environmental sampling unit;

[0021] S2. Initial settings for path planning:

[0022] Initially setting the current coordinates of the drone, the coordinates of the target area, the location of obstacles and dynamic environment parameters;

[0023] S3, calculating the pre-planned path;

[0024] S4, outputting the pre-planned path, and transmitting the pre-planned path to the position assessment module of the UAV for execution;

[0025] The pre-planned path is a node sequence from the starting point to the end point.

[0026] Optionally, the position evaluation module obtains the real-time position detection of the UAV and the change of environmental data, and compares the real-time position Pcurrent with the node position Pcurrent that should be reached in the pre-planned path. target , calculate the deviation degree △d of the drone from the pre-planned path, and calculate the distance d between the current position of the drone and the obstacle obstacle :

[0027]

[0028] In the formula, (x current ,y current ,z current) is the current coordinate of the UAV in three-dimensional space, that is, the real-time position of the UAV, and its value is directly obtained through the GPS set on the UAV. (x target ,y target ,z target ) is the coordinate of the current target node in the pre-planned path, indicating the location that the UAV should reach, and its value is determined by the node sequence of the pre-planned path determined by the environment perception unit, (x obstacle ,y obstacle ,z obstacle ) represents the three-dimensional coordinates of the obstacle, and its value is the three-dimensional coordinates of the obstacle acquired in real time by the sensor array. θ is the angle between the current flight direction of the drone and the direction of the pre-planned path. R curve It is to measure the curvature radius of the UAV flight path in real time, a drone is the UAV flight acceleration, v relative is the relative speed between the drone and the obstacle, △t is the time difference from the current position to the target position;

[0029] If the deviation degree △d exceeds the set distance deviation threshold threshold and the distance d between the current position of the drone and the obstacle obstacle If the distance is less than the safe distance threshold safe_distance set by the system, the environment perception module adjusts the pre-planned path and forms a flight route.

[0030] Optionally, the autonomous flight module includes an attitude control unit, a position control unit, a speed control unit, and a communication and command receiving unit. The attitude control unit monitors the attitude of the UAV in real time and adjusts the attitude according to the requirements of the flight route. The position control unit calculates the target position of the UAV according to the route instructions received by the autonomous flight module, and adjusts the speed and direction of the UAV to gradually approach the target position. The speed control unit controls the speed of the UAV by adjusting the motor output power according to the speed requirements specified in the flight route. The communication and command receiving unit receives flight instructions from a ground station or a host control system, and feeds back the flight status to the ground station.

[0031] Optionally, the visual acquisition module includes an image acquisition unit and a data storage unit, wherein the image acquisition unit acquires image data of the inspection object, and the data storage unit stores the image data acquired by the image acquisition unit;

[0032] Wherein, the visual acquisition module is loaded on the UAV.

[0033] Optionally, the autonomous flight module controls the drone according to the following steps:

[0034] S11. Receive flight route;

[0035] S12, initializing the flight state: the attitude control unit and the position control unit initialize the current flight state, and perform initial calibration according to the current position and the starting point of the route;

[0036] S13, dynamic adjustment and feedback: If the environment changes or new flight instructions are received during the flight, the communication and instruction receiving unit will transmit the new instructions to each control unit to dynamically adjust the flight path and attitude;

[0037] S14, Mission completion and return: When the UAV reaches the final destination, the attitude control unit, position control unit, and speed control unit will decide whether to execute the next mission or return to the starting point based on the mission requirements.

[0038] Optionally, the sensor array includes a lidar, an ultrasonic sensor, an optical flow sensor, and a barometer.

[0039] Optionally, the image acquisition unit includes an image acquisition probe and a data storage device, wherein the image acquisition probe acquires image data of the inspection object, and the data storage device stores the image data acquired by the image acquisition probe;

[0040] The beneficial effects achieved by the present invention are:

[0041] 1. Through the cooperation between the autonomous flight module and the position evaluation module, the inspection route of the drone is more accurate, ensuring that the entire system has the advantages of strong adaptability, high real-time evaluation capability, reliable inspection process, high intelligence, good flexibility and strong real-time decision-making ability;

[0042] 2. Through the cooperation between the command control module and the environmental perception module, the inspection process can realize autonomous inspection of the inspection route without a route, ensuring that the entire system has the advantages of higher inspection efficiency, strong automatic operation capability, high environmental perception capability, good flexibility and high intelligence;

[0043] 3. Through the cooperation of the environmental perception module and the position assessment module, the autonomous flight route of the UAV can be accurately controlled, ensuring that the entire system has the advantages of strong real-time assessment capability, high reliability of environmental perception during flight, strong adaptive capability, high inspection efficiency, no need for manual intervention and high intelligence;

[0044] 4. Through the cooperation between the visual acquisition module and the autonomous flight module, the drone can autonomously carry out route-free inspections and collect image data during the inspection process, ensuring that the entire system has the advantages of high adaptability, good flexibility, high inspection efficiency and high intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The present invention can be further understood from the following description in conjunction with the accompanying drawings. The components in the figures are not necessarily drawn to scale, but the emphasis is placed on illustrating the principles of the embodiments. In different views, the same reference numerals designate the same parts.

[0046] Figure 1 It is an overall block diagram of the present invention.

[0047] Figure 2 It is a block diagram of the control instruction module, the environment perception module and the UAV of the present invention.

[0048] Figure 3 It is a block diagram of the data acquisition unit, the environment perception unit and the environment sampling unit of the present invention.

[0049] Figure 4 Schematic diagram of the evaluation process of the location evaluation module of the present invention.

[0050] Figure 5 The figure is a schematic diagram of the analysis process of the image acquisition unit and the image analysis unit of the present invention.

[0051] Figure 6 A schematic diagram of an application scenario in which the drone of the present invention collects image data of an inspection object from at least two angles. DETAILED DESCRIPTION

[0052] The following is an explanation of the embodiments of the present invention through specific embodiments. Those skilled in the art can understand the advantages and effects of the present invention from the contents disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and the details in this specification can also be modified and changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. In addition, the drawings of the present invention are only simple schematic illustrations and are not depicted according to actual sizes. It is stated in advance. The following embodiments will further explain the relevant technical contents of the present invention in detail, but the disclosed contents are not intended to limit the scope of protection of the present invention.

[0053] Embodiment 1: According to Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 and Figure 6As shown, this embodiment provides a route-free automated inspection system, the automated inspection system includes a server and a drone, the automated inspection system also includes an airborne box, an autonomous flight module, a visual acquisition module, an environmental perception module, a control instruction module, and a position evaluation module, the server is respectively connected to the airborne box, the autonomous flight module, the visual acquisition module, the environmental perception module, the control instruction module, and the position evaluation module, and stores intermediate data and process data connected to the airborne box, the autonomous flight module, the visual acquisition module, the environmental perception module, the control instruction module, and the position evaluation module in a database of the server for query and call;

[0054] The onboard box is loaded in the drone to collect environmental data in real time during the flight of the drone. The control instruction module is responsible for receiving the coordinates of the target area input by the user and sending the coordinates of the target area to the drone. The environmental perception module obtains the coordinates of the target area and the environmental data collected by the onboard box, and dynamically calculates the pre-planned path. The position evaluation module evaluates the real-time position of the drone and the pre-planned path to form an evaluation result, and triggers the environmental perception module to dynamically adjust the pre-planned path according to the evaluation result, and adjusts to form a flight route. The autonomous flight module controls the drone according to the flight route. The visual acquisition module recognizes the state of the inspection object in real time, and cooperates with the autonomous flight module to automatically perform shooting or video recording operations at at least two angles;

[0055] Wherein, the autonomous flight module inspects the inspection objects in the target area according to the flight route;

[0056] The automated inspection system further includes a central processor, which is respectively connected to the airborne box, the autonomous flight module, the visual acquisition module, the environmental perception module, the control command module, and the position evaluation module, and centrally controls the airborne box, the autonomous flight module, the visual acquisition module, the environmental perception module, the control command module, and the position evaluation module based on the central processor, and stores the control data of the central processor in the server, so as to ensure the accuracy and reliability of the system's automatic inspection of the drone;

[0057] Optionally, the control instruction module includes an interaction unit and a transmission unit, wherein the interaction unit allows the user to input the coordinates of the target area through a human-computer interaction interface, and the transmission unit transmits the coordinates of the target area input by the user to the drone, so that the drone dynamically grasps the coordinates of the target area;

[0058] The interactive unit includes an interactive touch panel and an executable program, wherein the executable program is executed on the interactive touch panel and forms a human-computer interaction interface for the administrator to input the target area on the human-computer interaction interface;

[0059] The transmission unit includes a wireless communication component, a data processor, and an antenna. The wireless communication component is responsible for establishing a wireless communication link between a control platform (such as a ground station) and the UAV, and transmitting the coordinate data of the target area input by the user. Before the data is transmitted, the data processor pre-processes the coordinates of the target area input by the user to ensure the compatibility of the data format and protocol. The antenna is used to send and receive wireless signals to ensure that the data can be stably transmitted between the control platform and the UAV.

[0060] The data processor is used to format and package the coordinate data of the target area to ensure that no errors occur during data transmission. At the same time, the data is encrypted according to the communication protocol (such as TCP / IP, UDP, proprietary protocol, etc.) to enhance the security of data transmission;

[0061] The wireless transmission component includes a Wi-Fi transmitter, a 4G / 5G communicator, and an RF (radio frequency) transmitter. The Wi-Fi transmitter is suitable for short-distance, high-bandwidth data transmission, and is usually used for communication between drones and ground control stations within a short range. The 4G / 5G communicator is suitable for long-distance, large-range communication, providing a wider coverage range and faster data transmission rate, and is particularly suitable for the use of drones in remote inspections. The RF (radio frequency) transmitter works in ultra-long-distance communication or in the absence of existing network coverage, providing a more stable link, but usually with a lower bandwidth;

[0062] In this embodiment, the antenna includes an omnidirectional antenna and a directional antenna. The omnidirectional antenna is used for general communication tasks, has a wide coverage range but a low gain, and is suitable for use when the position of the drone is uncertain. The directional antenna is used for directional communication, has a high gain, and is suitable for use when the direction of the drone is known, and is usually used for long-distance communication.

[0063] Optionally, the environment perception module includes a data acquisition unit, an environment sampling unit and an environment perception unit, the environment sampling unit acquires environment data sampled in real time during the flight of the UAV, the data acquisition unit acquires the target area coordinates acquired by the control instruction module, and the environment perception unit dynamically calculates the pre-planned path according to the target area coordinates acquired by the data acquisition unit and the real-time environment data acquired by the environment sampling unit;

[0064] The environmental sampling unit includes a sensor array, a data fusion device and a real-time monitoring component. The sensor array collects real-time environmental data in the process of the UAV executing the coordinates of the target area. The data fusion device performs comprehensive processing on the data collected by the sensor array to perceive the environment around the UAV. The real-time monitoring component refreshes the data collected by the sensor array in real time.

[0065] Optionally, the sensor array includes a laser radar, an ultrasonic sensor, an optical flow sensor, and a barometer;

[0066] The real-time monitoring component includes a monitor, a time synchronizer, and an analog-to-digital converter. The monitor monitors the working status of the sensor and the system in real time to ensure the stable operation of the system; detects abnormal conditions and issues an alarm; the time synchronizer performs time synchronization through GPS signals and performs a refresh operation to ensure that the time of all devices and sensors is consistent in a large-scale operation; the analog-to-digital converter is used to convert the analog signal collected by the sensor into a digital signal for subsequent data processing;

[0067] The data collection unit includes a call requester and a feedback device, wherein the call requester sends a call request of the target area coordinates collected by the interaction unit, so that the interaction unit transmits the target area coordinates input by the administrator to the environment perception module, and after the target area coordinates are called, the feedback device sends a feedback instruction to the interaction unit, so as to feed back to the interaction unit an instruction that the target area coordinates have been obtained;

[0068] Optionally, the environment perception unit dynamically calculates the planned path according to the following steps:

[0069] S1, obtaining the target area coordinates and real-time environmental data from the data acquisition unit and the environmental sampling unit;

[0070] The target area coordinates are provided by the data acquisition unit, indicating the location coordinates (such as longitude and latitude, altitude) of the target area that the drone needs to inspect;

[0071] The environmental data is provided by the environmental sampling unit and contains relevant information about the current flight environment of the UAV, and its values ​​can be collected by the sensor array;

[0072] Meanwhile, in this embodiment, the environmental data includes but is not limited to the following: distance, obstacle location, wind speed, temperature, and air pressure;

[0073] S2. Initial settings for path planning:

[0074] Initially setting the current coordinates of the drone, the coordinates of the target area, the location of obstacles and dynamic environment parameters;

[0075] The environment perception unit first needs to set some initial parameters, which play a key role in path planning:

[0076] Starting point location: the current coordinates of the drone (provided by sensor data or GPS);

[0077] Target location: coordinates of the target area;

[0078] Environmental information: location of obstacles, dynamic environmental parameters (such as wind speed and direction);

[0079] S3, calculating the pre-planned path;

[0080] The pre-planned path includes the following steps:

[0081] S311, determine the search space:

[0082] Gridded space: Divide the drone flight area into grids (grids), each grid represents a possible location;

[0083] Mark the starting point and end point: mark the current position of the drone as the starting point and the coordinates of the target area as the end point;

[0084] S312, Initialize open and closed sets:

[0085] Open Set: stores the nodes to be explored. Initially, the starting point is placed in the open set.

[0086] Closed Set: stores the nodes that have been explored, which is empty at the beginning;

[0087] Specifically, node expansion and selection are performed according to the following steps:

[0088] 1) Initial node: The environment perception unit first sets the starting point (x start ,y start ,z start ) is put into the open set as the current node;

[0089] 2) Node expansion: Starting from the current node, the environment perception unit generates new neighbor nodes (x neighbor ,y neighbor ,z neighbor );

[0090] For each neighbor node, the environment perception unit calculates the cost function f(n);

[0091] Add the neighboring nodes that meet the conditions to the open set, and record the current node as its predecessor node;

[0092] Among them, the above conditions for joining the open set are to meet the following conditions at the same time:

[0093] ① The coordinates of the neighbor node must be within the valid range of the map; if the neighbor node exceeds the boundary of the map, it does not meet the conditions;

[0094] ② The location of the neighbor node cannot be an obstacle or a restricted area, and must be accessible; ③ The node has not been explored: If the neighbor node is not in the closed set, it means that it has not been processed yet; if the neighbor node is already in the closed set, it means that it has been processed and does not need to be explored again;

[0095] ④Meet flight safety requirements: Neighboring nodes maintain a safe distance from obstacles to ensure path safety;

[0096] 3) Node selection: Cost evaluation: Select the node with the smallest cost function f(n) in the open set as the next current node, that is, (x n ,y n ,z n ) is set to the coordinates of the neighbor node with the minimum cost;

[0097] 4) Update the current node: Each time the algorithm selects a new current node (x n ,y n ,z n ), the node will move from the open set to the closed set, indicating that it has been processed;

[0098] 5) Path tracing: When the algorithm finds the target node (xgoal, ygoal, zgoal), all nodes in the current node sequence (x n ,y n ,z n ) constitutes the path from the starting point to the end point;

[0099] The coordinates of these nodes are dynamically generated and selected during the execution of the environment perception unit to calculate the planned path, and each point on the path is gradually determined based on the evaluation of the cost function f(n);

[0100] S313. Calculate the heuristic function:

[0101] The heuristic function is used to estimate the cost from the current node to the end point. Commonly used estimation methods include Manhattan distance h(n):

[0102]

[0103] In the formula, (x n ,y n ,z n ) is the coordinate of the current node, and its value is based on the coordinate of the node currently being processed or evaluated during the path search process. In the path planning algorithm, a node is a point that represents a position in space.

[0104] (xgoal, ygoal, zgoal) are the coordinates of the target area;

[0105] S314, calculate the total cost function f(n):

[0106] f(n)=g(n)+h(n);

[0107] Wherein, h(n) is the heuristic cost (estimated value) from the current node n to the end point, and g(n) is the actual cost from the starting point to the current node n (in this embodiment, the flight distance is used as the actual cost), and its value is calculated according to the following formula:

[0108] g(n)=g(precious_node)+distance(previous_node,n);

[0109] Where g(previous_node) is the cumulative cost from the starting point to the previous node previous_node, and distance(previous_node,n) is the distance from the previous node previous_node to the current node n;

[0110] Among them, distance(previous_node,n) is the distance from the previous node previous_node to the current node n, which is calculated according to the following formula:

[0111]

[0112] In the formula, (x n ,y n ,z n ) is the coordinate of the current node, (x previous_node ,y previous_node , z previous_node ) is the coordinate of the previous node;

[0113] In this embodiment, the actual cost g(n) is calculated by gradually expanding the nodes from the starting point; each time a new node is selected as the current node n, the algorithm updates g(n) using the above formula and accumulates the cost to the next node:

[0114] 1) Initial cost: starting point P startThe cost g(P start ) is usually set to 0, because the cost of going from the starting point to the starting point is zero;

[0115] 2) The cost of the first node:

[0116] The cost of the first neighbor node n1 is:

[0117] g(n1)=g(P start )+distance(P start ,n1);

[0118] Because g(P start )=0, so:

[0119] g(n1)=distance(P start ,n1);

[0120] 3) Cumulative cost of subsequent nodes:

[0121] For any current node ni and previous node previous_node=n i-1 , and its cumulative cost is:

[0122] g(n i )=g(n i-1 )+distance(n i-1 ,n i );

[0123] S315, Node expansion and selection:

[0124] Select the node with the smallest cost function f(n) in the open set for expansion;

[0125] Check the neighbor nodes of the current node. If the neighbor nodes are in the closed set, ignore them. Otherwise, calculate their cost function.

[0126] Update open and closed sets: move the current node from the open set to the closed set, and add new feasible neighbor nodes to the open set;

[0127] S316, processing dynamic environment data:

[0128] During the path planning process, the environment perception unit needs to constantly check the real-time environment data (such as obstacle movement, new obstacles), and re-plan the path when necessary;

[0129] Dynamically adjust the path: If new obstacles are found on the path, or the environment changes significantly (such as wind direction changes causing an increase in flight cost), the f(n) value of the path needs to be recalculated, or even backtracked to the previous node for replanning;

[0130] S317. Termination conditions:

[0131] When the end node is added to the closed set, it means that an optimal path from the start point to the end point has been found;

[0132] If the open set is empty, it means there is no feasible path;

[0133] S4, outputting the pre-planned path, and transmitting the pre-planned path to the position assessment module of the UAV for execution;

[0134] Wherein, the pre-planned path is a sequence of nodes from the starting point to the end point;

[0135] In this embodiment, through the cooperation between the command control module and the environment perception module, the inspection process can realize autonomous inspection of the inspection route without a flight route, ensuring that the entire system has the advantages of higher inspection efficiency, strong automatic operation capability, high environmental perception capability, good flexibility and high intelligence;

[0136] Optionally, the position evaluation module obtains the real-time position detection of the drone and the change of environmental data, and compares the real-time position P current The node position P that should be reached in the pre-planned path target , calculate the deviation degree △d of the drone from the pre-planned path, and calculate the distance d between the current position of the drone and the obstacle obstacle :

[0137]

[0138] In the formula, (x current ,y current ,z current ) is the current coordinate of the UAV in three-dimensional space, that is, the real-time position of the UAV, and its value is directly obtained through the GPS set on the UAV. (x target ,y target ,z target ) is the coordinate of the current target node in the pre-planned path, indicating the location that the UAV should reach, and its value is determined by the node sequence of the pre-planned path determined by the environment perception unit, (x obstacle ,y obstacle ,z obstacle ) represents the three-dimensional coordinates of the obstacle, and its value is the three-dimensional coordinates of the obstacle acquired in real time by the sensor array. θ is the angle between the current flight direction of the drone and the direction of the pre-planned path. R curve It is to measure the curvature radius of the UAV flight path in real time, a drone is the UAV flight acceleration, which is usually measured in real time by the UAV’s inertial measurement unit (IMU). relativeis the relative speed between the drone and the obstacle, △t is the estimated time difference from the current position to the target position. The time difference Δt is usually calculated based on the drone’s predetermined trajectory and current state. For example, the system pre-plans the time t required for the drone to reach the next target point from its current position. target , and then according to the current time t of the drone curren Calculate: Δt = t current -t target ;

[0139] The angle θ between the current flight direction of the drone and the pre-planned path is determined according to the following steps:

[0140] S100, attitude estimation: Calculate the attitude angle of the drone, including the heading angle θ, through the IMU data.

[0141] A common method is to use the IMU's gyroscope, accelerometer, and possible magnetometer data combined with an algorithm to calculate quaternions or Euler angles (pitch, roll, and heading), where the heading angle is the θ we need.

[0142] S101, calculation of heading angle change: current heading angle θ current : The current heading angle of the drone provided by the IMU in real time.

[0143] Target heading angle θ target : Determined by the target direction in the pre-planned path, that is, if the node position of the target path is (x'1, y'1) to (x'2, y'2), the target heading angle can be calculated by the following formula:

[0144]

[0145] Heading angle change θ: the angle difference between the current flight direction and the pre-planned path, which can be calculated using the following formula:

[0146] θ = θcurrent - θtarget;

[0147] Among them, the angle θ represents the deviation of the UAV on the horizontal plane relative to the predetermined path;

[0148] For real-time measurement of the curvature radius R of the UAV flight path curve Follow these steps to determine:

[0149] S200, get the acceleration data of the drone

[0150] The IMU's accelerometer measures the linear acceleration of the drone in the x, y, and z directions. x ,a y ,a zBy integrating the acceleration, we can get the linear velocity v in each axis direction. x ,v y ,v z :

[0151]

[0152] Calculate the UAV's bus speed v:

[0153]

[0154] S201. Obtain the angular velocity data of the drone

[0155] The IMU's gyro sensor measures the drone's rotational angular velocity ω in the x, y, and z axes. x ,ω y ,ω z ;

[0156] Calculate the modulus of the angular velocity |ω|:

[0157]

[0158] S202. Calculate the radius of curvature:

[0159] The radius of curvature R is calculated according to the following formula: curve : For the relative speed v between the drone and the obstacle relative Calculate according to the following formula:

[0160]

[0161] In the formula, v drone is the real-time speed of the UAV, v obstacle is the speed of the obstacle;

[0162] The velocity vector v of the obstacle obstacle Obtained through multiple measurements: The speed of the obstacle can be obtained by collecting the obstacle position multiple times and calculating its displacement difference at multiple consecutive moments. The specific steps are as follows:

[0163] S300, collecting the position of the obstacle (x obstacle1 ,y obstacle1 ,z obstacle1 ) and (x obstacle2 ,y obstacle2 ,z obstacle2 ).

[0164] S301, calculate the velocity vector through these displacement values:

[0165]

[0166] If the deviation degree △d exceeds the set distance deviation threshold threshold and the distance d between the current position of the drone and the obstacle obstacle If the distance is less than the safe distance threshold safe_distance set by the system (satisfying one of the above trigger conditions), the environment perception module adjusts the pre-planned path and adjusts the flight route;

[0167] If the deviation degree △d is less than the set distance deviation threshold threshold and the distance d between the current position of the drone and the obstacle obstacle If the distance is greater than or equal to the safe distance threshold safe_distance set by the system, the path adjustment will not be triggered and the flight will proceed according to the original flight route;

[0168] The set distance deviation threshold threshold and the system set safety distance threshold safe_distance are set by the manager or the system according to actual needs (aircraft performance, etc.), which is a technical means well known to those skilled in the art. Those skilled in the art can refer to relevant technical manuals to learn about the technology, so they will not be described one by one in this embodiment;

[0169] The environment perception module adjusts the pre-planned path and adjusts the flight route to form a flight route according to the following steps:

[0170] S41. Path Replanning

[0171] If a route adjustment is triggered, you can re-plan the route by following the steps below:

[0172] S411. Find the nearest safe path point:

[0173] Goal: Find the path node closest to the current drone position, or find a safe temporary target point based on the current environment;

[0174] Use the heuristic function h(node) to calculate the distance between the current drone position and each path node, and select the node with the closest distance as the new target point;

[0175]

[0176] In the formula, (x current ,y current ,z current ) is the current position of the drone, (x node ,y node ,z node ) is the position of the path node;

[0177] S412, generating a temporary path: generating a temporary path based on the current UAV position and the newly selected target point;

[0178] That is, the path from the current drone position to the new target point is calculated according to the total cost function f(n):

[0179] f(n′)=g(n′)+h(n′);

[0180] Wherein, h(n') is the heuristic cost from the current node n' to the new target point (estimated value, calculated in the same way as h(n)), g(n') is the actual cost from the drone position to the new target point (in this embodiment, the flight distance is used as the actual cost), and the calculation method is the same as that of g(n) above, so it will not be repeated here;

[0181] S413, updating the flight route: updating the generated temporary path to the flight control system of the UAV;

[0182] The new flight route replaces the previous deviated path, allowing the drone to gradually return to the correct route or continue flying after avoiding obstacles;

[0183] S42, Obstacle Avoidance:

[0184] If d obstacle If the distance is less than the safe distance threshold safe_distance, obstacle avoidance can be performed through the following steps:

[0185] S421, Obstacle Detection and Modeling: Use LiDAR and ultrasonic sensors to collect environmental data of obstacles in real time to accurately identify and model the location, shape and size of obstacles;

[0186] S422, calculating an obstacle avoidance path: calculating a safe path to avoid obstacles based on obstacle information;

[0187] In this embodiment, an “obstacle avoidance method” (such as an artificial potential field method) is used to generate an obstacle avoidance path;

[0188] Among them, in this embodiment, the "obstacle avoidance method" drives the drone away from obstacles by introducing a repulsive potential field:

[0189]

[0190] Where U rep (n) is the repulsive potential field of the obstacle to node n, η is the gain factor of the repulsive potential field, and its value is determined based on experience. In this embodiment, an example is provided, specifically: it is adjusted according to the speed and mission requirements of the drone;

[0191] For example: Slow flight: η can be small (e.g. between 0.1 and 1) because the drone has enough time and space to avoid obstacles;

[0192] Fast flight: η is usually large (e.g. 1 to 10 or even higher) to ensure that the drone can quickly avoid obstacles when approaching them.

[0193] S423, recovery path planning: When the obstacle avoidance is completed, the system will plan the path from the current position to the recovery point, that is, calculate the path from the current drone position to the recovery point according to the total cost function f(n″):

[0194] f(n″)=g(n″)+h(n″);

[0195] Wherein, h(n″) is the heuristic cost (estimated value) from the current node n′ to the recovery point, g(n″) is the actual cost from the drone position to the recovery point (in this embodiment, the flight distance is used as the actual cost), and the calculation method is the same as the previous one, and will not be repeated one by one;

[0196] The detailed process is:

[0197] 1) Temporary deviation and obstacle avoidance: Generation of obstacle avoidance path: When the drone detects an obstacle during flight and the obstacle is at a distance d obstacle If the distance is less than the safe distance threshold safe_distance, the drone will immediately generate an obstacle avoidance path; this path guides the drone to a safe position to avoid collision with obstacles;

[0198] 2) Deviation from the original route: During the obstacle avoidance process, the drone will usually temporarily deviate from the original flight route to bypass obstacles; this means that the drone no longer strictly follows the originally planned path, but adjusts the flight direction and altitude according to the real-time obstacle avoidance needs;

[0199] 3) Restoring the original route: Path adjustment after obstacle avoidance is completed: Once the drone successfully avoids an obstacle, the system will try to guide the drone back to the original route; at this time, the new path planning will take into account the current safe position and a certain "recovery point" on the original route;

[0200] Recovery point selection: The recovery point is usually the point on the original route that is closest to the current drone position, or the next important node on the original route (such as a key inspection point);

[0201] Path planning: replan a path from the end point of the obstacle avoidance path to the recovery point, so that the drone gradually returns to the original route;

[0202] Through the cooperation of the environmental perception module and the position assessment module, the autonomous flight route of the UAV can be accurately controlled, ensuring that the entire system has the advantages of strong real-time assessment capability, high reliability of environmental perception during flight, strong adaptive capability, high inspection efficiency, no need for manual intervention and high intelligence;

[0203] Optionally, the autonomous flight module includes an attitude control unit, a position control unit, a speed control unit, and a communication and command receiving unit. The attitude control unit monitors the attitude of the UAV in real time and adjusts the attitude according to the requirements of the flight route. The position control unit calculates the target position of the UAV according to the route instructions received by the autonomous flight module, and adjusts the speed and direction of the UAV to gradually approach the target position. The speed control unit controls the speed of the UAV by adjusting the motor output power according to the speed requirements specified in the flight route. The communication and command receiving unit receives flight instructions from a ground station or a host control system, and feeds back the flight status to the ground station.

[0204] Wherein, the attitude control unit, the position control unit, the speed control unit, and the communication and command receiving unit are arranged on the UAV, and are loaded on the UAV and move following the movement of the UAV;

[0205] The attitude control unit includes an inertial measurement component and an attitude controller, wherein the inertial measurement component is used to detect the angular velocity, acceleration and attitude angle change of the drone, and the attitude controller processes the IMU sensor data and generates an attitude adjustment command to control the attitude of the drone;

[0206] Wherein, the inertial measurement component includes three accelerometers and three gyroscopes, which respectively measure the linear acceleration and angular velocity of the drone along three orthogonal axes;

[0207] The attitude controller usually uses a control algorithm of a PID controller to adjust the attitude of the drone based on IMU data;

[0208] Specifically, the attitude error (the difference between the current attitude and the target attitude) is calculated, and the control command is generated through the three control items of proportion, integration, and differentiation to adjust the attitude of the UAV;

[0209] Meanwhile, in the present embodiment, the attitude of the UAV is controlled by the control algorithm of the PID controller, which is a technical means well known to those skilled in the art. Those skilled in the art can refer to the relevant technical manuals to learn the technology, and thus will not be described in detail in the present embodiment.

[0210] In this embodiment, the position control unit also needs to cooperate with the visual acquisition module to collect image data of at least two angles of the inspection object;

[0211] In this embodiment, the visual acquisition module is used to collect the inspection object at multiple angles using a fly-around method;

[0212] The specific steps are as follows:

[0213] 1) Set a point on the inspection object as the center coordinate of the orbit as O(x0, y0, z0), and the current position of the drone is the inspection point (xtarget, ytarget, ztarget), which is also the initial shooting position of the visual acquisition module;

[0214] 2) Calculate the orbiting radius M:

[0215]

[0216] In order to obtain image data from at least two different angles, the drone is allowed to fly around the center point O of the inspection object. The flying path is a circle with a radius of M, and the camera is always aimed at the center point O. Several points on the circle are selected as new shooting positions to ensure the collection of image data from different angles.

[0217] 3) Coordinate calculation of adjacent shooting points:

[0218] like Figure 6 As shown, if the path of the drone around the inspection object O is on the horizontal plane (the drone and the inspection object are at the same height), that is (taking the acquisition of image data at two angles as an example) ztarget = z2 = z3, where z2 and z3 are the height coordinates of the two set acquisition positions, respectively, and the same height is maintained. The azimuth angle θ of the drone will change as it moves, thereby selecting adjacent shooting points.

[0219] If the initial shooting point (xtarget, ytarget, ztarget) is at azimuth θ1, then the two adjacent shooting points A (x2, y2, z2) and B (x3, y3, z3) are located at azimuths θ2 = θ1 + Δθ and θ3 = θ1 - Δθ, respectively. Then there is a point A:

[0220]

[0221] Among them, θ1 is the azimuth of the initial shooting point (the angle can be set by yourself, usually set to 0°), h is the height difference between the drone and the inspection object, its value is set by the system and usually remains fixed, Δθ is the angle offset (for example, 30° or 45°), its value is set by the system, that is, the angle amount of each offset when collecting multiple angles of the same inspection object;

[0222] Among them, point B exists:

[0223]

[0224] In this way, the camera is aimed at the same inspection object at each shooting point, and the central coordinate of the flying object is O(x0, y0, z0), but the drone shoots from different angles, and can obtain image data from more angles; the position control unit includes a motor controller and a rangefinder, and the rangefinder provides accurate ground distance data when flying at low altitude, wherein the rangefinder is arranged on the lower end surface of the drone body to collect the distance to the ground, and the motor controller executes the attitude control command determined by the attitude controller (such as calculated by the control algorithm of the PID controller above) to adjust the motion state of the drone;

[0225] Specifically, the specific control process of the position control unit is: by calculating the target position and current position information, the target direction and speed of the UAV are determined; then, by adjusting the attitude and motor output of the UAV, the UAV is controlled to move forward in the target direction and gradually approach the target position;

[0226] Through the above process, the position control unit can ensure that the drone flies according to the predetermined route and continuously adjusts during the flight to ensure that the drone reaches the target location accurately;

[0227] The speed control unit includes an electronic speed regulator, a motor, and a speed controller. The motor is arranged on the drone and generates thrust to control the forward speed, ascending speed, and other motion states of the drone. The speed controller processes speed data from the IMU and GPS and generates a control signal to adjust the output power of the motor to ensure that the speed of the drone meets the requirements of the flight route. The electronic speed regulator receives the control signal from the speed controller and adjusts the output power (rotation speed) of the motor to control the flight speed of the drone.

[0228] Wherein, the speed controller determines the control signal according to the following steps:

[0229] S51, first obtain the target speed V from the flight route instruction target , and V target As the target speed of the speed control unit;

[0230] S52, get current speed:

[0231] IMU and GPS data: The current flight speed V of the drone can be obtained in real time through the acceleration data of the IMU and the location information of the GPS. current ;

[0232] Among them, IMU calculates instantaneous velocity by integrating acceleration;

[0233] GPS calculates horizontal flight speed through position changes;

[0234] S53, calculate speed error:

[0235] Speed ​​Error: Calculates the error between the target speed and the current speed:

[0236] ΔV=V target -V current ;

[0237] Where V current is the current flight speed of the drone, V target is the target speed of the speed control unit;

[0238] S54, adjust the motor output power:

[0239] PID control algorithm: The speed controller uses the PID control algorithm to generate a control signal u(t) based on the speed error ΔV to adjust the output power of the motor:

[0240] The control signal u(t) is determined according to the following formula:

[0241]

[0242] In the formula, K p , K i , K d is the gain coefficient of the PID controller, ΔV(τ) represents the speed error at a certain time τ in the past, and dτ represents a small time change, which is used for integral calculation;

[0243] Among them, K p (Proportional gain): usually between 0.01 and 100;

[0244] K p =0.5 is suitable for drones flying at low speed and stability, with slow response but stable;

[0245] K p =10 is suitable for high-speed flight and requires quick response;

[0246] K p =50 is suitable for scenarios where deviation correction is urgently needed, but it may cause oscillation;

[0247] K i (Integral gain): usually between 0.001 and 1;

[0248] K i =0.05 is suitable for small drones to avoid long-term deviation;

[0249] K i=0.1 is suitable for scenarios where steady-state errors need to be eliminated smoothly;

[0250] K i =0.5 is suitable for scenarios where error elimination needs to be fast, but attention should be paid to response lag;

[0251] K d (Differential gain): usually between 0.001 and 10;

[0252] K d =0.01 is suitable for low-speed flight and allows some oscillation;

[0253] K d =0.1 is suitable for most common flight scenarios, with good smoothness and anti-oscillation capabilities;

[0254] K d =1 is suitable for high-speed or precision control scenarios, which require extremely high response smoothness;

[0255] In this embodiment, a commonly used example of taking a value of a gain coefficient of the coordinates of the execution target area is provided:

[0256] K p =1: The proportional gain is set to 1, ensuring that the system responds quickly to errors but is not overly sensitive;

[0257] K i =0.1: The integral gain is set to 0.1, and the ability to eliminate steady-state errors is stable and does not cause obvious lag;

[0258] K d =0.1: The differential gain is set to 0.1 to ensure smooth system response and reduce possible oscillations;

[0259] In short, the specific gain coefficient needs to be set in combination with the specific usage scenario of the drone to optimize the control performance of the drone, so I will not go into details one by one;

[0260] After the control signal is generated, the control signal u(t) is converted into the corresponding PWM signal. The change of duty cycle directly affects the speed of the motor;

[0261] S55, adjust the motor speed

[0262] ESC controls the motor: ESC adjusts the motor speed according to the received PWM signal, changes the thrust of the drone, and thus adjusts the flight speed of the drone;

[0263] S56, real-time feedback and adjustment

[0264] Speed ​​monitoring: The speed control unit continuously monitors the current speed and provides feedback through IMU and GPS data;

[0265] Dynamic adjustment: According to the real-time speed error, the motor output power is continuously adjusted to ensure that the speed of the drone is stable at the target speed V target nearby;

[0266] The communication and command receiving unit includes a wireless communicator, a transmission antenna, a microcontroller, a data buffer and a power supply battery.

[0267] The wireless communicator is responsible for wireless communication with the ground station or the upper control system to transmit flight instructions and status information; according to different application scenarios, the wireless communication module can use different communication technologies, such as Wi-Fi, RF (radio frequency), 4G / 5G, LoRa, etc. The transmission antenna is responsible for sending and receiving wireless signals and is an important part of the wireless communication module. The microcontroller, as the core processing unit of the communication and command receiving unit, is responsible for processing the received flight instructions and distributing these instructions to the various control units of the drone. The data buffer temporarily stores the received instruction data and the status information to be sent to ensure stable data transmission and processing. The power supply battery supplies power to the drone, as well as the wireless communicator, transmission antenna, microcontroller, and data buffer;

[0268] The specific working process of the communication and instruction receiving unit includes the following steps:

[0269] S61, initialization and connection establishment:

[0270] Startup: After the drone is powered on, the wireless communication module of the communication and command receiving unit starts and tries to establish a connection with the ground station or upper control system;

[0271] Handshake protocol: The wireless communication module and the ground station confirm the establishment of the connection through the handshake protocol and synchronize the communication parameters (such as frequency, channel, etc.);

[0272] S62, command reception and processing:

[0273] Data reception: The wireless communication module receives the flight command data packet sent by the ground station or the upper control system through the antenna;

[0274] Data parsing: The microcontroller parses the data packet and extracts specific control instructions (such as route adjustment and speed setting);

[0275] Command distribution: The parsed commands are distributed to various control units of the drone (such as position control unit, attitude control unit, speed control unit) to perform corresponding flight operations;

[0276] S63, Flight status data collection:

[0277] Status data collection: The microcontroller collects the current flight status data from various sensor modules (such as GPS, IMU, battery monitoring module);

[0278] Data integration: Integrate the data from each sensor into a status information package, including the real-time position, speed, attitude, and battery status of the drone;

[0279] S64, status feedback: the microcontroller packages the integrated status data and prepares to send it back to the ground station through the wireless communication module; the wireless communication module sends the status information package back to the ground station or the upper control system through the antenna for operator monitoring;

[0280] S65, Real-time communication and monitoring:

[0281] Continuous communication: The communication and command receiving unit maintains continuous communication with the ground station, receives commands in real time and provides status feedback;

[0282] Exception handling: If a communication anomaly is detected (such as signal loss, high latency, etc.), the microcontroller may trigger safety measures such as automatic return or hovering;

[0283] Optionally, the autonomous flight module controls the drone according to the following steps:

[0284] S11. Receive flight route;

[0285] S12, initializing the flight state: the attitude control unit and the position control unit initialize the current flight state, and perform initial calibration according to the current position and the starting point of the route;

[0286] In the process of inspecting the inspection object at the inspection point, it is necessary to collect image data of at least two angles of the inspection object at the inspection point, and analyze the inspection object through the image data to determine whether the current inspection object is the inspection object required. If it is the inspection object, fly to the next inspection point; if it is not the inspection object, replace the inspection object at the inspection point, and collect image data of multiple angles for the replaced inspection object until the image data of the required inspection object is completely collected;

[0287] S13, dynamic adjustment and feedback: If the environment changes during flight (such as wind speed, obstacle changes) or new flight instructions are received, the communication and instruction receiving unit will transmit the new instructions to each control unit to dynamically adjust the flight path and attitude;

[0288] S14, Mission completion and return: When the drone reaches the final destination, the attitude control unit, position control unit, and speed control unit will decide whether to execute the next mission or return to the starting point according to the mission requirements;

[0289] Optionally, the visual acquisition module includes an image acquisition unit and a data storage unit, wherein the image acquisition unit acquires image data of the inspection object, and the data storage unit stores the image data acquired by the image acquisition unit;

[0290] Wherein, the visual acquisition module is mounted on the drone;

[0291] Through the cooperation between the autonomous flight module and the position evaluation module, the inspection route of the drone is made more accurate, ensuring that the entire system has the advantages of strong adaptability, high real-time evaluation capability, reliable inspection process, high intelligence, good flexibility and strong real-time decision-making ability;

[0292] Optionally, the image acquisition unit includes an image acquisition probe and a data storage device, wherein the image acquisition probe acquires image data of the inspection object, and the data storage device stores the image data acquired by the image acquisition probe;

[0293] In this embodiment, through the mutual cooperation of the visual acquisition module and the autonomous flight module, the drone can autonomously perform route-free inspections and collect image data during the inspection process, ensuring that the entire system has the advantages of high adaptability, good flexibility, high inspection efficiency, and high intelligence.

[0294] Embodiment 2: This embodiment should be understood to include all the features of any of the above embodiments, and further improve upon them. Figure 1 , Figure 2 , Figure 3 , Figure 4 ,, Figure 5 and Figure 6 As shown, the visual acquisition module also includes an object analysis unit, which acquires image data of at least two angles obtained by the image acquisition unit for processing, and analyzes the processed image data source to determine whether the object is an inspection object;

[0295] The object analysis unit acquires the image data of at least two angles obtained by the image acquisition unit for processing, and the processing includes graying, noise removal, and edge detection. The specific steps are as follows:

[0296] 1) Grayscale the image data:

[0297] I gray (x,y)=0.299·I R (x,y)+0.587·I G (x,y)+0.114·I B (x,y);

[0298] In the formula, I gray (x, y) is the pixel value of the grayscale image at position (x, y), I R (x, y) is the pixel value of the red channel at position (x, y) in the input color image, I G (x, y) is the pixel value of the green channel at position (x, y) in the input color image, I B (x,y): pixel value of the blue channel at position (x,y) in the input color image;

[0299] Among them, the weights of the red, green, and blue channels are 0.299, 0.587, and 0.114, respectively. These weights reflect the sensitivity of the human eye to different colors.

[0300] 2) Noise Removal:

[0301]

[0302] In the formula, I smooth (x, y) is the pixel value of the image at position (x, y) after Gaussian blur processing, G(i, j): the weight of the Gaussian filter, which is specifically defined as:

[0303]

[0304] Where σ is the standard deviation of the Gaussian filter, which determines the degree of blur; k is the kernel size of the filter, which defines the range of the Gaussian filter (such as 3×3, 5×5, etc.); Igray(x+i,y+j) is the pixel value of the grayscale image at position (x+i,y+j);

[0305] The above formula means that at each pixel point (x, y), a (2k+1)×(2k+1) window centered at (x, y) is used to perform weighted averaging of the pixel values ​​within the window, thereby achieving the purpose of smoothing the image and removing noise.

[0306] Among them, (i, j): This is the offset of the neighboring pixels relative to the current pixel (x, y); Gaussian blur is a weighted average calculation of pixels in a local area (neighborhood), so it is necessary to traverse every pixel in the area;

[0307] i: represents the lateral offset, that is, the horizontal distance relative to the x position; the value range is from -k to k, where k is the radius of the filter kernel; if a 3×3 filter is used, then k=1; if a 5×5 filter is used, then k=2;

[0308] j: represents the longitudinal offset, that is, the vertical distance relative to the y position; the value range is also from -k to k;

[0309] The working principle of Gaussian blur is to perform weighted averaging on each pixel (x, y) in the neighborhood around it (the range represented by (i, j)); the weighting coefficient is calculated by the Gaussian function, the weight of the central pixel is the largest, and the weight of the pixel farther from the center is smaller; the final result is the pixel value Ismooth(x, y) at the (x, y) position, which is the weighted average of the pixel values ​​in the neighborhood, thereby achieving image smoothing and noise removal;

[0310] 3) Edge detection:

[0311] Apply edge detection algorithms (such as the Canny algorithm) to extract edge information in the image to help locate the contours of the object:

[0312] I edge (x,y)=Canny(I smooth (x, y));

[0313] In the formula, I edge (x, y): The edge response value of the image at position (x, y) after edge detection; this value is usually binary, indicating whether the position is an edge. smooth (x,y): The pixel value at position (x,y) of the image after Gaussian blur processing. It is the input of edge detection and the operator of Canny edge detection algorithm, which detects edges by calculating the gradient magnitude and direction of the image and applying non-maximum suppression and double thresholding.

[0314] The object analysis unit obtains a contour consisting of a series of points arranged in sequence after edge detection, and the coordinates of these coordinate points are:

[0315] Contour={(x1,y1),(x2,y2),…,(x N ,y N )};

[0316] For a closed polygonal object composed of N points, the object analysis unit calculates the perimeter of the contour corresponding to the image at the jth angle according to the following formula: j :

[0317]

[0318] Where xi,yi are the horizontal and vertical coordinates of the i-th contour point. is the Euclidean distance between the i-th point and the i+1-th point;

[0319] Similarly, for a closed polygon consisting of N points, the object analysis unit calculates the area Area of ​​the contour corresponding to the image at the jth angle according to the following formula: j :

[0320]

[0321] In the formula, x i ,y i To represent the horizontal and vertical coordinates of the i-th contour point, x i+1 ,y i+1 are the horizontal and vertical coordinates of the i+1th contour point;

[0322] The object analysis unit obtains the calculated perimeter and area of ​​the object contour, and calculates the multi-angle-based shape factor index Compactness according to the following formula: multi :

[0323]

[0324] Where Perimeter is the perimeter of the object contour, Area is the area of ​​the object contour, and M is the number of different angles to be collected;

[0325] If the shape factor index Compactness multi If the coordinates of the current target area fall within the allowable range [Range1, Range2] of the template object required by the system, it is confirmed that the currently collected object is an inspection object, and feedback is triggered to the autonomous flight module to control the UAV to collect images or videos of the inspection object at the next node along the flight route;

[0326] If the shape factor index Compactness multi If the coordinates of the current target area do not fall within the allowable range [Range1, Range2] of the template object required by the system, it means that the currently captured object is not the inspection object, and the image acquisition module is triggered to rotate the acquisition angle (control the drone to turn on the spot or control the image acquisition module to rotate the acquisition probe angle) and perform image acquisition on all objects at the inspection point in turn, and analyze the captured image data one by one until the inspection object is obtained;

[0327] Among them, the allowable range of the template object required in the coordinates of the current target area set by the system

[0328] [Range1, Range2] needs to be set by the system according to the actual situation of the inspection object. This is a technical means well known to those skilled in the art. Those skilled in the art can refer to relevant technical manuals to learn about the technology, so it will not be described in detail in this embodiment.

[0329] At the same time, the method of triggering the image acquisition module to rotate the acquisition angle can be realized by a person skilled in the art by controlling the drone and the image acquisition module to rotate, so in this embodiment, it will not be described one by one;

[0330] In this embodiment, through the mutual cooperation of the object analysis unit and the image acquisition unit, the image data of the inspection object in the inspection route can be collected, ensuring that the entire system has the advantages of high inspection efficiency, high inspection accuracy, good inspection reliability, high intelligence and strong environmental perception ability.

[0331] The contents disclosed above are only preferred feasible embodiments of the present invention, and do not limit the protection scope of the present invention. Therefore, all equivalent technical changes made using the contents of the present invention specification and drawings are included in the protection scope of the present invention. In addition, the elements therein can be updated as technology develops.

Claims

1. A route-free automated inspection system, comprising a server and a drone, characterized in that: The automated inspection system further includes an airborne box, an autonomous flight module, a visual acquisition module, an environmental perception module, a control instruction module, and a position evaluation module, and the server is connected to the airborne box, the autonomous flight module, the visual acquisition module, the environmental perception module, the control instruction module, and the position evaluation module respectively; The onboard box is loaded in the drone to collect environmental data in real time during the flight of the drone. The control instruction module is responsible for receiving the coordinates of the target area input by the user and sending the coordinates of the target area to the drone. The environmental perception module obtains the coordinates of the target area and the environmental data collected by the onboard box, and dynamically calculates the pre-planned path. The position evaluation module evaluates the real-time position of the drone and the pre-planned path to form an evaluation result, and triggers the environmental perception module to dynamically adjust the pre-planned path according to the evaluation result, and adjusts to form a flight route. The autonomous flight module controls the drone according to the flight route. The visual acquisition module recognizes the state of the inspection object in real time, and cooperates with the autonomous flight module to automatically perform shooting or video recording operations at at least two angles; The autonomous flight module inspects the inspection objects in the target area according to the flight route.

2. The route-free automated inspection system according to claim 1, characterized in that: The control instruction module includes an interaction unit and a transmission unit, wherein the interaction unit allows a user to input the coordinates of a target area through a human-computer interaction interface, and the transmission unit transmits the coordinates of the target area input by the user to the drone, so that the drone dynamically grasps the coordinates of the target area; The interactive unit includes an interactive touch panel and an executable program. The executable program is executed on the interactive touch panel to form a human-computer interaction interface for the administrator to input the target area on the human-computer interaction interface.

3. The route-free automated inspection system according to claim 2, characterized in that: The environment perception module includes a data acquisition unit, an environment sampling unit and an environment perception unit. The environment sampling unit collects the environment data sampled in real time during the flight of the UAV. The data acquisition unit collects the target area coordinates collected by the control instruction module. The environment perception unit dynamically calculates the pre-planned path according to the target area coordinates collected by the data acquisition unit and the real-time environment data collected by the environment sampling unit. The environmental sampling unit includes a sensor array, a data fusion device and a real-time monitoring component. The sensor array collects real-time environmental data during the process of the drone executing the coordinates of the target area. The data fusion device comprehensively processes the data collected by the sensor array to perceive the environment around the drone. The real-time monitoring component refreshes the data collected by the sensor array in real time.

4. The route-free automated inspection system according to claim 3, characterized in that: The environment perception unit dynamically calculates the planned path according to the following steps: S1, obtaining the target area coordinates and real-time environmental data from the data acquisition unit and the environmental sampling unit; S2. Initial settings for path planning: Initially setting the current coordinates of the drone, the coordinates of the target area, the location of obstacles and dynamic environment parameters; S3, calculating the pre-planned path; S4, outputting the pre-planned path, and transmitting the pre-planned path to the position assessment module of the UAV for execution; The pre-planned path is a node sequence from the starting point to the end point.

5. The route-free automated inspection system according to claim 4, characterized in that: The position evaluation module obtains the real-time position detection of the drone and the change of environmental data, and compares the real-time position Pcurrent with the node position Ptarget that should be reached in the pre-planned path, calculates the deviation degree △d of the drone from the pre-planned path, and calculates the distance dobstacle between the current position of the drone and the obstacle: In the formula, (xcurrent, ycurrent, z current ) is the current coordinate of the UAV in three-dimensional space, that is, the real-time position of the UAV, and its value is directly obtained through the GPS set on the UAV. (xtarget, ytarget, ztarget) is the coordinate of the current target node in the pre-planned path, indicating the position that the UAV should reach, and its value is determined by the node sequence of the pre-planned path determined by the environmental perception unit. (xobstacle, yobstacle, zobstacle) represents the three-dimensional coordinate of the obstacle, and its value is the three-dimensional coordinate of the obstacle obtained by real-time acquisition of the sensor array. θ is the angle between the current flight direction of the UAV and the direction of the pre-planned path. R curve It is to measure the curvature radius of the UAV flight path in real time, a drone is the UAV flight acceleration, v relative is the relative speed between the drone and the obstacle, △t is the time difference from the current position to the target position; If the deviation degree △d exceeds the set distance deviation threshold threshold and the distance dobstacle between the current position of the drone and the obstacle is less than the safe distance threshold safe_distance set by the system, the environment perception module is triggered to adjust the pre-planned path and adjust to form a flight route.

6. The route-free automated inspection system according to claim 5, characterized in that: The autonomous flight module includes an attitude control unit, a position control unit, a speed control unit, and a communication and instruction receiving unit. The attitude control unit monitors the attitude of the UAV in real time and adjusts the attitude according to the requirements of the flight route. The position control unit calculates the target position of the UAV according to the route instructions received by the autonomous flight module, and adjusts the speed and direction of the UAV to gradually approach the target position. The speed control unit controls the speed of the UAV by adjusting the motor output power according to the speed requirements specified in the flight route. The communication and instruction receiving unit receives flight instructions from a ground station or a host control system, and feeds back the flight status to the ground station.

7. The route-free automated inspection system according to claim 6, characterized in that: The visual acquisition module includes an image acquisition unit and a data storage unit. The image acquisition unit acquires image data of the inspection object at least at two angles, and the data storage unit stores the image data acquired by the image acquisition unit. Wherein, the visual acquisition module is loaded on the UAV.

8. The route-free automated inspection system according to claim 7, characterized in that: The autonomous flight module controls the drone according to the following steps: S11. Receive flight route; S12, initializing the flight state: the attitude control unit and the position control unit initialize the current flight state, and perform initial calibration according to the current position and the starting point of the route; S13, dynamic adjustment and feedback: If the environment changes or new flight instructions are received during the flight, the communication and instruction receiving unit will transmit the new instructions to each control unit to dynamically adjust the flight path and attitude; S14, Mission completion and return: When the UAV reaches the final destination, the attitude control unit, position control unit, and speed control unit will decide whether to execute the next mission or return to the starting point based on the mission requirements.

9. The route-free automated inspection system according to claim 8, characterized in that: The sensor array includes a lidar, an ultrasonic sensor, an optical flow sensor, and a barometer.

10. The route-free automated inspection system according to claim 9, characterized in that: The image acquisition unit includes an image acquisition probe and a data storage device. The image acquisition probe acquires image data of the inspection object, and the data storage device stores the image data acquired by the image acquisition probe.

Images