A UAV remote automatic inspection method and system

Abstract

The present invention relates to the field of intelligent operation and maintenance technology, and discloses a method and system for remote automatic inspection of unmanned aerial vehicles. The method comprises: obtaining an inspection route, wherein the shortest route of the total path is a fixed inspection route; planning the inspection route, determining corresponding inspection points according to the fixed inspection route, so as to obtain a fixed inspection route work information set containing all inspection points; constructing a process parameter set of the inspection flight, and obtaining a standard control strategy parameter set based on the process parameter set fitting; drawing a boundary based on the maximum offset of the process parameter set relative to the standard control strategy parameter set, and obtaining a safe deviation boundary information parameter in combination with a conversion coefficient, and summarizing the safe deviation boundary information parameter based on the time granularity to obtain a safe deviation boundary information parameter set; combining the standard control strategy parameters and the safe deviation boundary information parameter set to form a standard control strategy for the flight process; and setting a flight auxiliary strategy. The safety of remote automatic inspection of unmanned aerial vehicles is improved, and a double insurance form is formed.

Description

Technical Field

[0001] The present invention relates to the field of intelligent operation and maintenance technology, and in particular to a method and system for remote automatic inspection of unmanned aerial vehicles. Background Art

[0002] Regular inspection is a necessary part of the operation and maintenance of hydropower stations. It is of great significance to ensure the safe and stable operation of hydropower stations and other plants. Its core work content is to move along the fixed inspection route of each high-rise corridor in the plant, arrive at each inspection point in sequence and complete the prescribed safety inspection, instrument review and information recording processes. However, at present, station corridor inspections are mostly performed manually. The staff needs to regularly patrol along the inspection route on foot. The inspection route is usually long, so there are practical problems such as repeated, heavy and tiring work for the inspectors.

[0003] A drone is a flying mobile tool that is easy to operate, has expandable functions, and is flexible in action. It can be configured with tools such as cameras to perform remote acquisition and transmission of various images and videos. Airborne camera equipment based on drones can realize various automation functions for drones and patrol objects through machine vision, which is economical and feasible. However, the existing technology for autonomous driving of drones is mostly achieved by using satellite positioning systems or three-dimensional laser scanning systems. For the automatic inspection needs of plants and stations such as hydropower stations, there is a problem of no satellite signals because the inspection corridors in the plant are indoor environments, and the installation of three-dimensional laser scanning systems has additional costs and the costs are relatively high. Currently, accurate indoor positioning and navigation of drones in complex indoor environments can be achieved through the combination of BIM data and QR code identification. However, accurate BIM data is required, and fully autonomous drone automatic flight functions have not yet been realized. Therefore, they are not applicable to the corridor environment within power plants without BIM data, and cannot meet the functional requirements of automatic inspections. There are also vision-assisted drone indoor flight three-dimensional attitude algorithms and indoor corridor flight heading angle error visual correction methods, which can use machine vision methods to assist in the combination of IMU and magnetometer to correct the cumulative error of the drone heading angle or fly in a straight line to avoid collisions with walls. However, the need for additional equipment such as IMU and magnetometer increases costs, and they are all drone flight auxiliary functions, and cannot yet achieve complete automatic flight and automatic inspection functions.

[0004] Therefore, there is an urgent need for a drone flight method that can achieve fully automated and active planning of inspection routes. Summary of the invention

[0005] In view of the above existing problems, the present invention is proposed.

[0006] Therefore, the present invention provides a method and system for remote automatic inspection by unmanned aerial vehicles to solve the problems that indoor inspections are currently mostly carried out using satellite positioning systems or three-dimensional laser scanning systems, which are not applicable to sites without satellite signals, require the configuration of various auxiliary hardware, are too costly, and cannot fully realize autonomous flight functions.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] In a first aspect, the present invention provides a method for remote automatic inspection of a drone, comprising:

[0009] Configure drone inspection conditions;

[0010] Obtaining an inspection path, including obtaining at least one inspection route, wherein the shortest route is a fixed inspection route;

[0011] Planning an inspection route, wherein the planning determines corresponding inspection points according to the fixed inspection route to obtain a fixed inspection route work information set including all inspection points;

[0012] Based on the fixed inspection route work information set, a process parameter set of the inspection flight is constructed, and a standard control strategy parameter set is obtained based on the process parameter set fitting;

[0013] Construct a safety deviation boundary information parameter set, draw a boundary based on the maximum offset of the process parameter set of the inspection flight relative to the standard control strategy parameter set, and obtain the safety deviation boundary information parameter in combination with the conversion coefficient, and summarize the safety deviation boundary information parameter based on the time granularity to obtain the safety deviation boundary information parameter set;

[0014] Combining standard control strategy parameters and safety deviation boundary information parameter set to form a standard control strategy for the flight process;

[0015] Set up flight assistance strategies, including the drone obtaining flight information based on safe flight signs, combining standard control strategies for the flight process, and predicting and executing operations; as well as flight altitude assistance and emergency response strategies.

[0016] As a preferred solution of the remote automatic inspection method of the unmanned aerial vehicle described in the present invention, wherein: a process parameter set of the inspection flight is constructed, and a standard control strategy parameter set is obtained based on the process parameter set fitting, including: the process parameter set of the inspection flight is a set of multiple manually operated inspection flight process parameters;

[0017] The parameters of multiple manual inspection flight processes are fitted into a standard flight process standard control strategy parameter set, which is expressed as:

[0018]

[0019] in, It is a set of standard control strategy parameters for the flight process; is a process parameter set of an inspection flight, that is, a set of multiple manually operated inspection flight process parameters; For the The process parameters of the manually operated UAV inspection flight are in the form of operation information instruction parameters And the time information corresponding to each parameter A collection of; The established parameter set for the manually operated UAV inspection flight process The total number of all process parameters in .

[0020] As a preferred solution of the UAV remote automatic inspection method of the present invention, wherein: constructing a safe deviation boundary information parameter set, drawing a boundary based on the maximum offset of the inspection flight process parameter set relative to the standard control strategy parameter set, and combining the conversion coefficient to obtain the safe deviation boundary information parameter, including: performing time decomposition and offset data visualization, specifically including:

[0021] Set the standard control strategy parameters And each process parameter in the inspection flight process parameter set According to the time granularity, it is decomposed into moment information and corresponding operation information parameters;

[0022] At any time, the standard control strategy parameter set is an operation parameter, the theoretical position processed by the control drone is controlled as the origin, and a first plane perpendicular to the corridor is established, and the origin is on the first plane;

[0023] Each process parameter For the operation parameters, the projection points of the theoretical position of the UAV on the first plane are plotted;

[0024] The origin and projection points on the first plane are used to reflect the various process parameters at that moment. Compared with standard control strategy parameter set The degree of deviation;

[0025] It also includes determining the orientation domain and the maximum safe deviation boundary, including:

[0026] Divide a first plane having an origin and a projection point and being perpendicular to the corridor into a plurality of orientation domains based on the origin and with perpendicular or parallel side walls as boundaries;

[0027] Based on the divided azimuth domains, the maximum offset point in each azimuth domain is determined. The maximum offset point is the process parameter with the largest distance from the origin to each azimuth domain. The corresponding projection point;

[0028] Connect the maximum deviation points in each domain with straight lines to obtain the maximum safe deviation boundary;

[0029] A safety deviation factor is introduced to adjust the maximum safety deviation boundary range to the safety deviation boundary range of the final application.

[0030] As a preferred solution of the UAV remote automatic inspection method of the present invention, the safety deviation boundary information parameters are summarized based on the time granularity to obtain a safety deviation boundary information parameter set, which is expressed as:

[0031]

[0032] in, For safe deviation from boundary information parameter set; For the moment Maximum safe deviation boundary parameter when ; is the safety offset factor, which is less than 1; is the number of azimuth domains; For the A function to find the maximum deviation point in an azimuth domain.

[0033] As a preferred solution of the remote automatic inspection method of the unmanned aerial vehicle of the present invention, wherein: combining the standard control strategy parameter set and the safety deviation boundary information parameter set to form a standard control strategy for the flight process, including: automatically flying and performing inspection work according to the standard control parameter set for the standard flight process until returning;

[0034] During the flight, the deviation of the drone from the standard control parameter set during the flight process is controlled within the safe deviation boundary information parameter set;

[0035] Specifically expressed as: ;

[0036] in, Provide standard control strategies for the flight process of drone inspections; It is a processing function for the standard control strategy of the flight process to assist with the safety deviation boundary information parameters.

[0037] As a preferred solution of the remote automatic inspection method of the unmanned aerial vehicle of the present invention, wherein: the acquisition of flight information based on the safe flight mark, combined with the standard control strategy of the flight process, and the pre-judgment execution operation include: recognizing the information image of the interval arrangement of the safe flight mark through the visual module of the unmanned aerial vehicle, and acquiring the flight information transmitted by the safe flight mark, wherein the flight information includes the position and flight operation information;

[0038] The location information is the distance or stake number from the starting point of the safe flight mark; the flight operation information is based on the standard control strategy of the UAV flight process. The flight control parameters obtained by analysis are used to predict and execute operations.

[0039] As a preferred solution of the remote automatic inspection method of the unmanned aerial vehicle of the present invention, wherein: the flight altitude assists in obtaining the flight altitude based on the safe flight mark and maintaining the flight attitude within the preset safe altitude range, including: obtaining images at various scale distances when flying at the corresponding altitudes based on the upper and lower parallel bars in the safe flight mark through the visual module of the unmanned aerial vehicle, and constructing two sets of image databases;

[0040] Through convolutional neural network training, the image recognition capability of upper and lower limit flight states is acquired;

[0041] Based on the image recognition capability of the upper and lower limit flight states, the flight altitude is controlled to keep the flight attitude within a preset safe altitude range;

[0042] The emergency response strategy constructs an obstacle-free and obstacle-involved image database and inputs it into a convolutional neural network for visual training to acquire obstacle recognition capabilities;

[0043] Based on the obstacle recognition capability, an obstacle warning is issued.

[0044] In a second aspect, the present invention provides a UAV remote automatic inspection system, comprising:

[0045] Configuration module, used to configure drone inspection conditions;

[0046] An acquisition module is used to acquire an inspection path, including acquiring at least one inspection route, wherein the shortest route of the total path is a fixed inspection route;

[0047] A planning module is used to plan the inspection route, wherein the planning determines corresponding inspection points according to the fixed inspection route to obtain a fixed inspection route work information set including all inspection points;

[0048] The first construction module is used to construct a process parameter set for the inspection flight, and obtain a standard control strategy parameter set based on the process parameter set fitting;

[0049] The second construction module is used to construct a safety deviation boundary information parameter set, draw boundaries based on the maximum offset of each process parameter relative to the standard control strategy parameter, and obtain safety deviation boundary information parameters in combination with a conversion coefficient, and summarize the safety deviation boundary information parameters based on a time granularity to obtain a safety deviation boundary information parameter set;

[0050] A strategy generation module, which is used to combine standard control strategy parameters and safety deviation boundary information parameter set to form a standard control strategy for the flight process;

[0051] The auxiliary strategy setting module is used to set the flight assistance strategy, including the drone obtaining flight information based on the safe flight mark, combining the standard control strategy of the flight process, predicting and executing operations; as well as flight altitude assistance and emergency response strategies.

[0052] In a third aspect, the present invention provides a computing device, comprising:

[0053] Memory and processor;

[0054] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the UAV remote automatic inspection method are implemented.

[0055] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the UAV remote automatic inspection method.

[0056] Compared with the prior art, the present invention has the following beneficial effects: the present invention aims at the inspection work of power plants and stations. By setting the standard control strategy of the UAV inspection flight process, the present invention can realize the automatic realization of UAV flight, checkpoint inspection, return and other functions, which is beneficial to reduce the heavy and repetitive physical labor of the hydropower station staff, improve the accuracy of inspection data acquisition, and reduce human errors; more accurate flight trajectory control is performed through the safe flight mark combined with machine vision, and its flight parameters are continuously corrected according to the standard flight process parameters to realize a safer and more reliable remote automatic inspection function; through auxiliary flight strategies such as image recognition, it can automatically decide to execute logical operations such as hovering waiting, alarming, remote takeover control, and suspending return, so that the UAV has the ability to deal with sudden obstacles during the inspection process, which is beneficial to improve the overall safety of the UAV remote automatic patrol system and reduce the risk rate of UAV source accidents during operation; while using the safety deviation boundary information parameters to assist the standard control strategy of the flight process to achieve safe flight, the safe flight assistance of the UAV is realized based on the safe flight mark by machine vision means, that is, a double insurance form is formed, which greatly improves the reliability of the safe flight of the UAV. Intelligent inspection methods are conducive to the formation of an inspection information database, which facilitates the traceability of inspection data and the overall information and digital construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0058] Figure 1 This is a schematic diagram of the overall process of the UAV remote automatic inspection method according to the first embodiment of the present invention;

[0059] Figure 2 A schematic diagram of a safe flight mark in the remote automatic inspection method for a drone according to the first embodiment of the present invention;

[0060] Figure 3 This is a schematic diagram of inspection operation identification in the UAV remote automatic inspection method according to the first embodiment of the present invention;

[0061] Figure 4 A schematic diagram of a plane construction required for safely deviating from the boundary in the method for remote automatic inspection of a drone according to the first embodiment of the present invention;

[0062] Figure 5 It is a schematic diagram of the theoretical position and projection of each process parameter controlling the drone when drawing the safe deviation boundary in the method for remote automatic inspection of drones according to the first embodiment of the present invention;

[0063] Figure 6 A schematic diagram of a method for remote automatic inspection by a drone according to a first embodiment of the present invention, wherein the method is divided into four azimuth domains with vertical or parallel side walls as boundaries;

[0064] Figure 7 A schematic diagram of preliminary results is drawn for the maximum safe deviation boundaries of four azimuth domains in the UAV remote automatic inspection method according to the first embodiment of the present invention;

[0065] Figure 8 A schematic diagram of the final result is drawn by taking the four azimuth domains as an example of the maximum safe deviation boundary in the UAV remote automatic inspection method described in the first embodiment of the present invention. DETAILED DESCRIPTION

[0066] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0067] Example 1, reference Figure 1-Figure 8 , as an embodiment of the present invention, provides a method for remote automatic inspection by a drone, comprising:

[0068] S100: Configure drone inspection conditions;

[0069] Specifically, the drone can be equipped with a visual module and an automatic flight control module. The visual module includes cameras (two sideways) placed forward and sideways facing the main flight direction of the drone and related video storage, image analysis, and information transmission equipment; the automatic flight control module includes a logic processor that can be directly connected to the drone to control its flight status and related information remote transmission equipment.

[0070] The selection criteria for drones include: (1) The maneuverability of the drone after full load should meet the following requirements: it should be able to complete the flight of the inspection route and return home, it should be able to maintain a hovering state in the air and turn while hovering, the route should remain basically straight during flight, and there should be no obvious shaking when hovering. (2) The drone should have the load capacity and space for installing two or more cameras. (3) The flight control system of the drone should have access and expansion capabilities, and it can achieve unmanned automatic flight by configuring a flight control processor.

[0071] In addition, according to actual needs, the drone can also be equipped with an electronic compass, alarm device, lighting device, etc.

[0072] Specifically, a flight identification system needs to be configured. The flight identification system includes two types: safe flight identification and checkpoint operation identification. Both are static identification and are posted or placed on the wall when in use.

[0073] like Figure 2 As shown, the safe flight sign is configured on the side wall of the corridor to maintain the safe flight posture of the drone, including flight altitude control, distance control from the side wall, and flight angle control. The safe flight sign must be used together with the side camera in the visual module configured by the drone to achieve the safe flight function.

[0074] like Figure 3 As shown, the inspection point operation mark D is configured on the wall of the target inspection area, such as on the wall next to the instrument to be inspected, to guide the drone to carry out specific inspection work. Figure 3 In the figure, D1 is the outer ring of the guide mark of the checkpoint operation mark; D2 is the QR code image in the checkpoint operation mark containing the operation information of the checkpoint. The checkpoint operation mark is used together with the forward camera in the visual module configured by the drone, and the operation information of the checkpoint is transmitted to the drone by machine vision to guide the drone to perform inspection operations.

[0075] The operation information of the inspection point includes information about the object to be inspected, information about the inspection operation steps that need to be performed by the drone, location coordinate information of the object to be inspected, orientation information, etc.

[0076] Place safe flight signs on the side walls of the corridor along the inspection route; place checkpoint operation signs on the walls of the target inspection area.

[0077] S200: Obtaining an inspection path, including obtaining at least one inspection route, wherein the shortest route of the total path is a fixed inspection route;

[0078] Specifically, the inspection route is determined based on the inspection content and sequence of the inspection within the plant and station, as well as the route conditions within the plant, and at least one route is determined. Routes that are not the shortest are all alternative routes.

[0079] It should be noted that in order to realize automatic inspection of drones, it is necessary to first determine the inspection route and relevant specific inspection point operation requirements based on the inspection objectives.

[0080] S300: Planning an inspection route, determining corresponding inspection points according to a fixed inspection route, so as to obtain a fixed inspection route work information set including all inspection points;

[0081] Specifically, the corresponding inspection points are all areas and inspection objects that need to be inspected, and the specific inspection content that needs to be carried out at each inspection point is clearly defined, such as: meter reading or status inspection and other information.

[0082] S400: constructing a process parameter set of the inspection flight based on the fixed inspection route work information set, and obtaining a standard control strategy parameter set based on the process parameter set fitting;

[0083] It should be noted that for the fixed inspection route work information set determined in the previous step, it is necessary to establish a process parameter set for manually operated UAV inspection flights, so as to construct a standard control strategy for the UAV inspection flight process. The fixed inspection route work information set is a specific guiding standard for the establishment of a process parameter set for manually operated UAV inspection flights by UAVs, that is, the inspection flight of manually operated UAVs should follow the fixed inspection route described in the fixed inspection route work information set, and the inspection of manually operated UAVs should be carried out in sequence according to the inspection objects described in the fixed inspection route work information set, so as to complete the work content of executing the inspection within the plant.

[0084] Specifically, according to the requirements of the fixed inspection route work information set, the drone is manually operated to complete the flight and inspection of all inspections along the inspection route until it returns, and the manual operation parameter information of the inspection is fully recorded and saved. Repeat the above manual operation to complete the inspection process and record and save the manual operation parameter information until a sufficient number of inspection flight process parameters are obtained and a process parameter set is formed.

[0085] Exemplarily, the process parameter set includes 10 or more qualified manual inspection flight process parameters. Qualified manual inspection flight process parameters refer to: manual operation information instruction parameters and time information parameters corresponding to the entire process of the drone flying without interruption and without being affected by external events, performing inspection work, and returning according to the fixed inspection route work information set.

[0086] In the embodiment of the present application, a process parameter set of the inspection flight is constructed, and a standard control strategy parameter set is obtained based on the process parameter set fitting, including:

[0087] The process parameter set of the inspection flight is a collection of multiple manually operated inspection flight process parameters;

[0088] The parameters of multiple manual inspection flight processes are fitted into a standard flight process standard control strategy parameter set, which is expressed as:

[0089] ,

[0090] in, It is a set of standard control strategy parameters for the flight process; is a process parameter set of an inspection flight, that is, a set of multiple manually operated inspection flight process parameters; For the The process parameters of the manually operated UAV inspection flight are in the form of operation information instruction parameters And the time information corresponding to each parameter A collection of; The established parameter set for the manually operated UAV inspection flight process The total number of all process parameters in .

[0091] S500: construct a safety deviation boundary information parameter set, draw a boundary based on the maximum offset of the inspection flight process parameter set relative to the standard control strategy parameter set, and obtain the safety deviation boundary information parameter in combination with the conversion coefficient, and summarize the safety deviation boundary information parameter based on the time granularity to obtain the safety deviation boundary information parameter set;

[0092] In the embodiment of the present application, based on the process parameter set of the inspection flight Relative to the standard control strategy parameter set The maximum offset is used to draw the boundary, and the safe deviation boundary information parameters are obtained by combining the conversion coefficient, including: time decomposition and offset data visualization, specifically including:

[0093] Set the standard control strategy parameters And each process parameter in the inspection flight process parameter set According to the time granularity, it is decomposed into moment information and corresponding operation information parameters;

[0094] It should be noted that for a moment, there are corresponding standard control strategy parameters Operation information parameters and various process parameters The time granularity is determined according to the actual data processing capability, and can be 1 minute, 30 seconds, 10 seconds, etc.

[0095] At any time, the standard control strategy parameter set is the operation parameter, such as Figure 4 As shown, the theoretical position processed by the drone is controlled as the origin O, and a first plane a perpendicular to the corridor b is established, and the origin is on the first plane;

[0096] Each process parameter As the operating parameter, draw the projection point e' of the theoretical position e of the UAV on the first plane;

[0097] The origin and projection points on the first plane are used to reflect the various process parameters at that moment. Compared with standard control strategy parameter set The degree of deviation;

[0098] It should be noted that after the above steps, data visualization can be achieved, such as Figure 5 shown.

[0099] It also includes determining the orientation domain and the maximum safe deviation boundary, including:

[0100] Divide a first plane having an origin and a projection point and being perpendicular to the corridor into a plurality of orientation domains based on the origin and with perpendicular or parallel side walls as boundaries;

[0101] For example, it can be considered that the origin is divided into four orientation domains with vertical or parallel side walls as boundaries, such as Figure 6 As shown, it includes four areas a1, a2, a3 and a4, or is expanded to be divided into eight azimuth domains.

[0102] Based on the divided orientation domain, determine the maximum offset point in each orientation domain. The maximum offset point is the process parameter with the largest distance from the origin to each orientation domain. The corresponding projection point;

[0103] Connect the maximum deviation points in each domain with straight lines to obtain the maximum safe deviation boundary S;

[0104] For example, taking the case of four orientation domains as an example, the drawing result can be as follows Figure 7 As shown, the four points EFGH are the maximum offset points from the origin in each azimuth domain.

[0105] A safety deviation factor is introduced to adjust the maximum safety deviation boundary range to the safety deviation boundary range of the final application.

[0106] It should be noted that the value of the safety deviation coefficient is less than 1. The coefficient is multiplied by the distance from the maximum offset point of each azimuth domain to the origin to obtain the new point position. Then, the safe deviation boundary of the actual application can be determined by connecting the lines. For example, taking the case of four azimuth domains as an example, the drawing result can be as follows: Figure 8 As shown, the four points E', F', G' and H' are the updated points in each azimuth domain calculated based on the safety offset coefficient, and S' is the safety deviation boundary of the actual application.

[0107] In the embodiment of the present application, based on the time granularity, the safety deviation boundary information parameters are aggregated to obtain a safety deviation boundary information parameter set, which is expressed as:

[0108] ,

[0109] in, For safe deviation from boundary information parameter set; For the moment Maximum safe deviation boundary parameter when ; is the safety offset factor, which is less than 1; is the number of azimuth domains; For the A function to find the maximum deviation point in an azimuth domain.

[0110] S600: Combining the standard control strategy parameters and the safety deviation boundary information parameter set to form a standard control strategy for the flight process;

[0111] In the embodiment of the present application, a standard control strategy parameter set and a safety deviation boundary information parameter set are combined to form a standard control strategy for the flight process, including: automatically flying and performing inspection work until returning according to the standard control parameter set for the standard flight process;

[0112] During the flight, the deviation of the drone from the standard control parameter set during the flight process is controlled within the safe deviation boundary information parameter set;

[0113] Specifically expressed as: ;

[0114] in, Provide standard control strategies for the flight process of drone inspections; It is a processing function for the standard control strategy of the flight process to assist with the safety deviation boundary information parameters.

[0115] Furthermore, the standard flight control strategy guides the flight control of the drone through the automatic flight control module configured by the drone. Actual tests can be carried out in barrier-free hydropower plant buildings, that is, the drone is equipped with the standard flight control strategy, automatically performs flight and conducts inspections. If the drone can complete the automatic inspection safely and completely according to the requirements of the fixed inspection route work information set and return home, the determined standard flight control strategy is considered to be effective and meets the inspection requirements.

[0116] S700: Setting flight assistance strategies, including obtaining flight information based on safe flight identification, combining standard flight process control strategies, predicting and executing operations; and flight altitude assistance and emergency response strategies.

[0117] It should be noted that in order to ensure the safe flight and operation execution of the drone during the automatic inspection, it is considered to achieve flight and operation assistance through machine vision. That is, through the aforementioned flight identification system, using the visual module configured by the drone, a vision-based inspection assistance strategy is established. The purpose is to control the flight posture of the drone in the corridor, keep it within a suitable and safe range, and confirm the location information of the drone.

[0118] For example, the safe flight logo may be in the form of Figure 1 As shown, double parallel strip signs are arranged on the side walls of the corridor, and two-dimensional code images C containing location and operation information are arranged at intervals. The safe flight signs are arranged continuously along the inspection route, and the direction can be changed according to the inspection route and the stairs can be taken to the next floor or the previous floor.

[0119] Determine the location of the safe flight sign. The upper parallel strip A in the safe flight sign is placed at the upper limit of the safe flight altitude of the corresponding drone. The upper limit is set based on the height of the roof and the equipment already placed on the roof. The determination method can also be expressed by the formula:

[0120]

[0121] Where: The location of the upper parallel stripes in the safe flight logo;

[0122] is the location of the gallery roof;

[0123] The vertical length of the equipment arranged on the roof;

[0124] is the margin factor, which takes a value greater than 0 based on safety requirements.

[0125] The lower parallel strip B in the safe flight sign is arranged at the position corresponding to the lower limit of the safe flight altitude of the UAV. The lower limit position is determined according to the corridor height and the average height of the human body, and can usually be set to 2.0m.

[0126] In the embodiment of the present application, the flight information is obtained based on the safe flight identification, and the operation is pre-judged and executed in combination with the standard control strategy of the flight process, including:

[0127] S701: Identify the information image of the safe flight sign arranged at intervals through the visual module of the UAV, and obtain the flight information transmitted by the safe flight sign, the flight information including the position and flight operation information;

[0128] The location information is the distance or stake number from the starting point of the safe flight mark; the flight operation information is based on the standard control strategy of the UAV flight process. The flight control parameters obtained by analysis are used to predict and execute operations.

[0129] Specifically, flight operation information refers to the current flight plan of the drone, which is based on the standard control strategy of the drone flight process. Flight control parameters obtained through database analysis. For example: the estimated time node to the location, the specific operations to be performed at the subsequent time nodes, etc. For example, when the drone is flying in a horizontal corridor, the specific operations to be performed at the subsequent time nodes are to continue moving forward according to the strategy; when the drone is flying in an area where it needs to turn or go up and down stairs, the specific operations to be performed at the subsequent time nodes include at which subsequent time node to start turning, in what direction and how many degrees to turn, or to start turning and move up or down, how much distance to move, etc. In this way, the drone can combine its standard control strategy during the flight process , predict the operation to be performed.

[0130] In the embodiments of the present application, Figure 1 As shown, the flight altitude assistance obtains the flight altitude based on the safe flight mark and keeps the flight attitude within the preset safe altitude range, including:

[0131] S702-1: Obtain the upper and lower parallel bars in the safe flight mark through the UAV's visual module. Figure 1 The images at each scale distance when flying at the corresponding altitudes of A and B are used to construct two sets of image databases;

[0132] For example, when the drone is at various scale distances from the corridor side wall, where the safe flight sign is located (based on the corridor width, divided equally according to the accuracy requirements, such as a 5m wide corridor divided into 10 equal parts, i.e., 0.5m, 1.0m, ..., 4.5m scale distances). When the drone flies at the height corresponding to the lower parallel bar in the safe flight sign, the images taken by the side camera at various scale distances are collected to establish a lower limit image database; when the drone flies at the height corresponding to the upper parallel bar in the safe flight sign, the images taken by the side camera at various scale distances are collected to establish an upper limit image database.

[0133] S702-2: Train the convolutional neural network to obtain the image recognition capability of upper and lower flight states;

[0134] Furthermore, based on step S702-2:

[0135] S702-3: When the drone is flying at the altitude corresponding to the lower parallel bar in the safe flight sign, test whether it can identify the current situation as the lower limit flight state; when the drone is flying at the altitude corresponding to the upper parallel bar in the safe flight sign, test whether it can identify the current situation as the upper limit flight state. If the test is successful for both the upper and lower limit flight states, it can be considered that the drone has the recognition capability based on the safe flight sign.

[0136] S702-4: Based on the image recognition capability of the upper and lower flight status, control the flight altitude and keep the flight attitude within the preset safe altitude range;

[0137] For example, when the drone recognizes the lower limit flight state through machine vision, it returns a signal to the flight control module to control the drone to raise its flight altitude until it leaves the lower limit flight state upward; when the drone recognizes the upper limit flight state through machine vision, it returns a signal to the flight control module to control the drone to lower its flight altitude until it leaves the upper limit flight state downward. That is, a drone safe flight assistance strategy based on safe flight identification is established.

[0138] According to this strategy, the UAV can automatically maintain the flight altitude within the interval corresponding to the upper and lower parallel bars of the safe flight mark, that is, maintain the flight altitude within the safe flight mark.

[0139] It should be noted that based on the safe flight mark, a drone operation assistance strategy based on the checkpoint operation mark can also be set up. Specifically, according to the aforementioned checkpoint operation mark configured on the wall of the target inspection area, when the drone approaches the checkpoint and recognizes the checkpoint operation mark information, the inspection operation is performed according to the mark information.

[0140] Combining the above steps, automatic safe flight assistance and checkpoint inspection operation assistance for drones can be achieved based on the safe flight signs and checkpoint operation signs that have been arranged on the inspection route.

[0141] The emergency response strategy builds a barrier-free and barrier-free image database and inputs it into the convolutional neural network for visual training to acquire obstacle recognition capabilities;

[0142] Obstacle warning is issued based on obstacle recognition capability.

[0143] Exemplarily, the specific steps for building an emergency response strategy are:

[0144] (1) The steps to build barrier-free and barrier-free image databases are as follows:

[0145] When there are no obstacles in the corridor and the UAV flies according to the standard control strategy of its inspection flight process, the pictures of the corridor ahead taken by the camera arranged in the forward direction of the visual module are collected to establish a safe corridor image database.

[0146] Arrange personnel (one or more) in the corridor corresponding to the flight direction of the UAV. Then, collect pictures of the corridor ahead taken by the UAV at a safe flight altitude when the personnel are in various positions, and establish an image database of corridors with personnel obstacles.

[0147] Typical obstacles (typical objects that may appear in the corridor, such as a cart) are arranged in the corridor corresponding to the flight direction of the UAV. Then, pictures of the corridor ahead taken by the UAV at a safe flight altitude with obstacles in various positions are collected to establish an object obstacle corridor image database.

[0148] (2) Machine vision training

[0149] Using the convolutional neural network (CNN) model, we conduct machine vision training, targeting the safety corridor image database, the personnel obstacle corridor image database, and the object obstacle corridor image database, respectively, to obtain the ability to identify the corridor safety status, personnel obstacles, and object obstacles. And input the data into the drone vision module

[0150] (3) Drone obstacle recognition capability test

[0151] Based on the previous step, test whether the drone can identify the current situation of people and objects obstacles based on the visual module when there are people and objects obstacles in the corridor. If the test for the state of obstacles is successful (recognition rate exceeds 98%), it can be considered that the drone has a certain obstacle recognition ability test

[0152] (4) Drone obstacle response strategy

[0153] When the drone encounters a human obstacle, it will respond according to the following steps: First, the drone will remain in a hovering state and stop moving forward; then it will use its configured alarm device to sound an alarm to remind the personnel to evacuate; if the personnel have evacuated and the corridor image recognition is safe, the inspection will continue. If the human obstacle is still recognized after a certain period of time (such as 5 minutes), the duty officer will be notified remotely to take over and perform manual operation until the human obstacle is removed; if the duty officer cannot be contacted to take over and the human obstacle has not been evacuated, the drone will be controlled to stop the inspection and return home, and continue to notify the corresponding responsible personnel. If the obstacle is removed, the automatic inspection will continue along the route.

[0154] When the drone encounters an obstacle, it will respond according to the following steps: First, the drone will remain in a hovering state and stop moving forward; then the on-duty personnel will be notified remotely to take over and perform manual operations until the obstacle is cleared; if the on-duty personnel cannot be contacted to take over, the drone will be controlled to stop the inspection and return home, and the corresponding responsible personnel will be notified. If the obstacle is removed, the automatic inspection will continue along the route.

[0155] It should also be noted that the auxiliary strategy can also establish inspection data collection, transmission and analysis rules, including inspection data recording rules and inspection data transmission rules.

[0156] The rules for recording drone inspection data are as follows:

[0157] The UAV obtains inspection information during the automatic inspection process, and according to its form, it should be first recorded in the memory configured inside the UAV in combination with the inspection time information.

[0158] Among them, when the inspection form is to obtain data information of the inspection object, you should first take a photo of the corresponding object, and then use the existing image recognition technology to identify and extract the specific data information (such as instrument readings), and record it in the format of picture, data information, and time information.

[0159] Among them, when the inspection form is to obtain appearance information of the inspection object, a photo of the corresponding object should be taken first, and then the existing image recognition technology should be used to identify and give preliminary status judgment suggestions, and record them in the format of pictures, status judgment suggestions, and time information.

[0160] The purpose of retaining the image information here is to preserve the original inspection data archive to facilitate manual verification of automatically extracted or judged information.

[0161] The data transmission rules for drone inspections are as follows:

[0162] For example, the drone and the site have specific real-time data interaction functions. If the drone and the factory are equipped with a wireless network system, the inspection data can be transmitted back to the user end in real time.

[0163] If the drone does not have real-time data interaction capabilities, the inspection data will first be stored in the drone, and then the inspection information will be transmitted back to the user end after returning.

[0164] Finally, an inspection information database is established, and each inspection information is sorted according to its time information to form a traceable database, that is, an inspection information database is established.

[0165] It should be noted that the present invention sets the form, usage, data rules, etc. of the safe flight identification and checkpoint operation identification; based on this, a method for establishing a standard control strategy for the UAV inspection flight process and a method for obtaining a parameter set of UAV safety deviation boundary information are constructed, as well as a method for combining the method with the standard control strategy for application, which can realize fully automatic and precise UAV flight.

[0166] The above is a schematic scheme of a method for remote automatic inspection of unmanned aerial vehicles in this embodiment. It should be noted that the technical scheme of the system for remote automatic inspection of unmanned aerial vehicles and the technical scheme of the method for remote automatic inspection of unmanned aerial vehicles described above are of the same concept. For details not described in detail in the technical scheme of the system for remote automatic inspection of unmanned aerial vehicles in this embodiment, please refer to the description of the technical scheme of the method for remote automatic inspection of unmanned aerial vehicles described above.

[0167] The UAV remote automatic inspection system in this embodiment includes:

[0168] Configuration module, used to configure drone inspection conditions;

[0169] An acquisition module is used to acquire an inspection path, including acquiring at least one inspection route, wherein the shortest route of the total path is a fixed inspection route;

[0170] The planning module is used to plan the inspection route and determine the corresponding inspection points according to the fixed inspection route to obtain a fixed inspection route work information set containing all the inspection points;

[0171] The first construction module is used to construct a process parameter set for the inspection flight, and obtain a standard control strategy parameter set based on the process parameter set fitting;

[0172] The second construction module is used to construct a safety deviation boundary information parameter set, draw boundaries based on the maximum offset of each process parameter relative to the standard control strategy parameter, and obtain safety deviation boundary information parameters in combination with the conversion coefficient. Based on the time granularity, the safety deviation boundary information parameters are summarized to obtain the safety deviation boundary information parameter set;

[0173] A strategy generation module, which is used to combine standard control strategy parameters and safety deviation boundary information parameter set to form a standard control strategy for the flight process;

[0174] The auxiliary strategy setting module is used to set the flight assistance strategy, including the drone obtaining flight information based on the safe flight mark, combining the standard control strategy of the flight process, predicting and executing operations; as well as flight altitude assistance and emergency response strategies.

[0175] This embodiment also provides a computing device suitable for the case of remote automatic inspection by a drone, including:

[0176] Memory and processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the method for realizing remote automatic inspection of unmanned aerial vehicles as proposed in the above embodiment.

[0177] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for realizing remote automatic inspection of a drone as proposed in the above embodiment is implemented.

[0178] The storage medium proposed in this embodiment and the method for realizing remote automatic inspection of unmanned aerial vehicles proposed in the above embodiment belong to the same inventive concept. The technical details not described in detail in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0179] Through the above description of the implementation mode, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course can also be implemented by hardware, but in many cases the former is a better implementation mode. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer's floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention.

[0180] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for remote automatic inspection by unmanned aerial vehicle, characterized in that: include: Configure drone inspection conditions; Obtaining an inspection path, including obtaining at least one inspection route, wherein the shortest route is a fixed inspection route; Planning an inspection route, wherein the planning determines corresponding inspection points according to the fixed inspection route to obtain a fixed inspection route work information set including all inspection points; Based on the fixed inspection route work information set, a process parameter set of the inspection flight is constructed, and a standard control strategy parameter set is obtained based on the process parameter set fitting; Construct the process parameter set of the inspection flight, and obtain the standard control strategy parameter set based on the process parameter set fitting, including: The inspection flight process parameter set is a collection of multiple manually operated inspection flight process parameters; The parameters of multiple manual inspection flight processes are fitted into a standard flight process standard control strategy parameter set, which is expressed as: ， in, It is a set of standard control strategy parameters for the flight process; is a process parameter set of an inspection flight, that is, a set of multiple manually operated inspection flight process parameters; For the The process parameters of the manually operated UAV inspection flight are in the form of operation information instruction parameters And the time information corresponding to each parameter A collection of; The established parameter set for the manually operated UAV inspection flight process The total number of all process parameters in; Construct a safety deviation boundary information parameter set, draw a boundary based on the maximum offset of the process parameter set of the inspection flight relative to the standard control strategy parameter set, and obtain the safety deviation boundary information parameter in combination with the conversion coefficient, and summarize the safety deviation boundary information parameter based on the time granularity to obtain the safety deviation boundary information parameter set; Among them, construct a safe deviation boundary information parameter set, draw a boundary based on the maximum offset of the process parameter set of the inspection flight relative to the standard control strategy parameter set, and obtain the safe deviation boundary information parameter in combination with the conversion coefficient, including: time decomposition and offset data visualization, specifically including: Set the standard control strategy parameters And each process parameter in the inspection flight process parameter set According to the time granularity, it is decomposed into moment information and corresponding operation information parameters; At any time, the standard control strategy parameter set is an operation parameter, the theoretical position processed by the control drone is controlled as the origin, and a first plane perpendicular to the corridor is established, and the origin is on the first plane; Each process parameter For the operation parameters, the projection points of the theoretical position of the UAV on the first plane are plotted; The origin and projection points on the first plane are used to reflect the various process parameters at that moment. Compared with standard control strategy parameter set The degree of deviation; It also includes determining the orientation domain and the maximum safe deviation boundary, including: Divide a first plane having an origin and a projection point and being perpendicular to the corridor into a plurality of orientation domains based on the origin and with perpendicular or parallel side walls as boundaries; Based on the divided azimuth domains, the maximum offset point in each azimuth domain is determined. The maximum offset point is the process parameter with the largest distance from the origin to each azimuth domain. The corresponding projection point; Connect the maximum deviation points in each domain with straight lines to obtain the maximum safe deviation boundary; Introduce a safety deviation factor to adjust the maximum safety deviation boundary range to the safety deviation boundary range of the final application; Among them, based on the time granularity, the safety deviation boundary information parameters are summarized to obtain the safety deviation boundary information parameter set, which is expressed as: ， in, For safe deviation from boundary information parameter set; For the moment Maximum safe deviation boundary parameter when ; is the safety offset factor, which is less than 1; is the number of azimuth domains; For the A function to find the maximum offset point in the azimuth domain; Combining standard control strategy parameters and safety deviation boundary information parameter set to form a standard control strategy for the flight process; Combining the standard control strategy parameter set and the safety deviation boundary information parameter set to form a standard control strategy for the flight process, including: automatically flying and performing inspection work until returning according to the standard control parameter set for the standard flight process; During the flight, the deviation of the drone from the standard control parameter set during the flight process is controlled within the safe deviation boundary information parameter set; Specifically expressed as: ; in, Provide standard control strategies for the flight process of drone inspections; A processing function to assist the standard control strategy of the flight process for safety deviation boundary information parameters; Set up flight assistance strategies, including the drone obtaining flight information based on safe flight signs, combining standard control strategies for the flight process, and predicting and executing operations; as well as flight altitude assistance and emergency response strategies.

2. The method for remote automatic inspection of a drone as claimed in claim 1, characterized in that: The method of obtaining flight information based on the safe flight sign and combining the standard control strategy of the flight process to predict and execute the operation includes: recognizing the information image of the safe flight sign arranged at intervals through the visual module of the unmanned aerial vehicle, and obtaining the flight information transmitted by the safe flight sign, wherein the flight information includes the position and flight operation information; The location information is the distance or stake number from the starting point of the safe flight mark; the flight operation information is based on the standard control strategy of the UAV flight process. The flight control parameters obtained by analysis are used to predict and execute operations.

3. The method for remote automatic inspection of a drone as claimed in claim 2, characterized in that: The flight altitude assistance obtains the flight altitude based on the safe flight mark and keeps the flight attitude within the preset safe altitude range, including: using the visual module of the drone to respectively obtain images at various scale distances when flying at the corresponding altitudes based on the upper and lower parallel bars in the safe flight mark, and constructing two sets of image databases; Through convolutional neural network training, the image recognition capability of upper and lower limit flight states is acquired; Based on the image recognition capability of the upper and lower limit flight states, the flight altitude is controlled to keep the flight attitude within a preset safe altitude range; The emergency response strategy constructs an obstacle-free and obstacle-involved image database and inputs it into a convolutional neural network for visual training to acquire obstacle recognition capabilities; Based on the obstacle recognition capability, an obstacle warning is issued.

4. A system using the UAV remote automatic inspection method according to any one of claims 1 to 3, characterized in that: include: Configuration module, used to configure drone inspection conditions; An acquisition module is used to acquire an inspection path, including acquiring at least one inspection route, wherein the shortest route of the total path is a fixed inspection route; A planning module is used to plan the inspection route, wherein the planning determines corresponding inspection points according to the fixed inspection route to obtain a fixed inspection route work information set including all inspection points; The first construction module is used to construct a process parameter set for the inspection flight, and obtain a standard control strategy parameter set based on the process parameter set fitting; The second construction module is used to construct a safety deviation boundary information parameter set, draw boundaries based on the maximum offset of each process parameter relative to the standard control strategy parameter, and obtain safety deviation boundary information parameters in combination with a conversion coefficient, and summarize the safety deviation boundary information parameters based on a time granularity to obtain a safety deviation boundary information parameter set; A strategy generation module, which is used to combine standard control strategy parameters and safety deviation boundary information parameter set to form a standard control strategy for the flight process; The auxiliary strategy setting module is used to set the flight assistance strategy, including the drone obtaining flight information based on the safe flight mark, combining the standard control strategy of the flight process, predicting and executing operations; as well as flight altitude assistance and emergency response strategies.

5. An electronic device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the remote automatic inspection method of the unmanned aerial vehicle described in any one of claims 1 to 3 are implemented.

6. A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the remote automatic inspection method of a drone as described in any one of claims 1 to 3.

Images