Control method and device of unmanned aerial vehicle laser obstacle removing machine

Abstract

The invention discloses a control method and device of an unmanned aerial vehicle laser obstacle removing machine. The method comprises the following steps: acquiring obstacle clearing direction data and first position data of a laser obstacle clearing machine and second position data of a to-be-cleared obstacle under the condition that the laser obstacle clearing machine carried by the unmanned aerial vehicle is started; determining a yaw angle of the unmanned aerial vehicle based on the obstacle clearing direction data, the first position data and the second position data; and under the condition that the absolute value of the yaw angle of the unmanned aerial vehicle is larger than a preset yaw threshold value, the laser obstacle removing machine is controlled to be closed. The problem that the control accuracy of the laser obstacle removing machine is low is solved, and the beneficial effect of improving the control accuracy of the laser obstacle removing machine is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of equipment control, and in particular to a control method and device for an unmanned aerial vehicle laser obstacle remover. Background Art

[0002] Transmission lines are restricted areas where foreign objects may exist. In order to remove foreign objects from transmission lines, laser obstacle removal is carried out using drone laser obstacle removal devices.

[0003] The target of the drone laser obstacle remover is light floating objects such as kite streamers and plastic films on high-voltage transmission lines. Its "laser cannon" uses a high-power density laser generator that does not require water cooling, which can "destroy" floating objects in a very short time. However, this seemingly simple process still faces some difficulties. For example, if the obstacle removal target is lost or the direction is offset during the automatic inspection of the drone, the laser obstacle remover will not be able to be shut down in time and hit the wires in the transmission line and other important power facilities. Summary of the invention

[0004] The present invention provides a control method and device for a laser obstacle remover of an unmanned aerial vehicle, so as to solve the problem of low control accuracy of the laser obstacle remover.

[0005] According to one aspect of the present invention, a control method for a UAV laser obstacle removal machine is provided, the method comprising:

[0006] When the laser obstacle remover carried by the UAV is turned on, obtaining obstacle removal direction data, first position data and second position data of the obstacle to be removed by the laser obstacle remover;

[0007] Determine a yaw angle of the drone based on the obstacle clearance direction data, the first position data, and the second position data;

[0008] When the absolute value of the yaw angle of the UAV is greater than a preset yaw threshold, the laser obstacle remover is controlled to be turned off.

[0009] According to another aspect of the present invention, a control device for a UAV laser obstacle removal machine is provided, the device comprising:

[0010] A data acquisition module, used for acquiring obstacle removal direction data, first position data and second position data of the obstacle to be removed by the laser obstacle removal device when the laser obstacle removal device carried by the UAV is turned on;

[0011] A yaw angle determination module, used to determine the yaw angle of the UAV based on the obstacle clearance direction data, the first position data and the second position data;

[0012] The obstacle removal machine shutdown module is used to control the laser obstacle removal machine to shut down when the absolute value of the yaw angle of the UAV is greater than a preset yaw threshold.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the control method of the drone laser obstacle remover described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the control method of the unmanned aerial vehicle laser obstacle removal machine described in any embodiment of the present invention when executed.

[0018] The technical solution of the embodiment of the present invention is to obtain the obstacle clearing direction data, first position data and second position data of the obstacle to be cleared of the laser obstacle clearer when the laser obstacle clearer carried by the drone is turned on; accurately determine the obstacle clearing direction and locate the position of the laser obstacle clearer and the position of the obstacle to be cleared; then, determine the yaw angle of the drone based on the obstacle clearing direction data, the first position data and the second position data; accurately judge the angle of deviation of the drone from the route; finally, when the absolute value of the yaw angle of the drone is greater than the preset yaw threshold, control the laser obstacle clearer to turn off. When the direction of the drone deviates to the point where it cannot accurately clear obstacles, the laser obstacle clearer is turned off in time, which solves the problem of low safety factor that if the obstacle clearing target is lost or the direction is deviated during the automatic inspection of the drone, the laser obstacle clearer cannot be turned off in time and hits the wires and other important power facilities in the transmission line, and achieves the beneficial effects of improving the safety and stability of the laser obstacle clearer of the drone and improving the automation of the drone inspection.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 is a flow chart of a control method of a UAV laser obstacle removal machine provided according to the first embodiment of the present invention;

[0022] Figure 2a This is a flow chart of a control method of a UAV laser obstacle removal machine provided according to Embodiment 2 of the present invention;

[0023] Figure 2b is a flow chart of an optional example of a control method for a UAV laser obstacle removal machine provided according to Embodiment 2 of the present invention;

[0024] Figure 3 2 is a schematic diagram of the structure of a control device for a UAV laser obstacle removal machine provided according to Embodiment 3 of the present invention;

[0025] Figure 4 It is a structural schematic diagram of an electronic device for implementing the control method of the unmanned aerial vehicle laser obstacle removal machine according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] Embodiment 1

[0029] Figure 1A flowchart of a control method for a UAV laser obstacle remover is provided for the first embodiment of the present invention. This embodiment is applicable to controlling a UAV laser obstacle remover. The method can be executed by a control device for a UAV laser obstacle remover. The control device for the UAV laser obstacle remover can be implemented in the form of hardware and / or software. The control device for the UAV laser obstacle remover can be configured in an electronic device. Figure 1 As shown, the method includes:

[0030] S110: When the laser obstacle remover carried by the UAV is turned on, obtain obstacle removal direction data, first position data, and second position data of the obstacle to be removed by the laser obstacle remover.

[0031] Among them, the laser obstacle remover can be understood as a device that uses laser technology to remotely remove foreign objects from overhead lines. The obstacle removal direction data can be understood as the emission direction of the laser beam of the laser obstacle remover. The obstacle removal direction data can be expressed in the form of angles or coordinates, such as horizontal angles, vertical angles, or three-dimensional spatial coordinates. The first position data can be understood as the spatial position data of the laser obstacle remover carried by the human machine. The second position data can be understood as the spatial position data of the obstacle to be removed.

[0032] Specifically, the image of the target obstacle is acquired and displayed by the sighting system equipped with the laser obstacle remover carried by the drone. Obstacle removal direction data of the laser obstacle remover based on the acquired image of the target obstacle. Or by determining the direction data of the drone head, the alignment direction of the laser obstacle remover is always in the same direction as the direction of the drone head, and the obstacle removal direction data of the laser obstacle remover is determined based on the direction data of the drone head. The first position data of the laser obstacle remover can be obtained in real time through the global positioning system on the drone. Among them, the first position data can be represented by longitude and latitude, altitude and attitude angle (such as pitch angle, yaw angle, etc.). The laser radar carried by the drone can be used to emit a laser beam and receive the reflected signal to construct a three-dimensional point cloud model of the surrounding environment. By analyzing the point cloud data, the second position data, shape data and size data of the obstacle can be identified. The second position data can be represented in the form of three-dimensional space coordinates, including the center point coordinates, boundary coordinates or contour coordinates of the obstacle.

[0033] S120: Determine the yaw angle of the UAV based on the obstacle clearance direction data, the first position data, and the second position data.

[0034] Specifically, the horizontal axis in the world coordinate system is defined as a zero degree yaw angle, and the angle between the drone head direction (obstacle clearance direction data) and the x-axis of the world coordinate system is determined and calculated based on the obstacle clearance direction data, the first position data, and the second position data by the following formula:

[0035] l0=(0,1)

[0036] l1=(x f -x,y f -y)

[0037]

[0038] Among them, l0 is the unit vector when the yaw angle of the drone is zero degrees, l1 is the unit vector when the yaw angle of the drone is The vector at the time (x f ,y f ) is the second position data of the obstacle, (x, y) is the first position data of the laser obstacle remover, and θ is the angle between the yaw direction and the preset route direction;

[0039]

[0040] in, is the yaw angle of the drone.

[0041] S130: When the absolute value of the yaw angle of the UAV is greater than a preset yaw threshold, control the laser obstacle remover to be turned off.

[0042] The preset yaw threshold may be preset based on experience, such as 15 degrees, and this embodiment does not limit it.

[0043] Specifically, when the calculated absolute value of the drone's yaw angle is greater than the preset yaw threshold, it indicates that the drone has deviated from the route. At this time, the laser obstacle remover cannot aim at the center of the obstacle to be cleared. At this time, if the obstacle removal continues, it is easy to hit the wires in the transmission line and other important power facilities. Therefore, when the calculated absolute value of the drone's yaw angle is greater than the preset yaw threshold, the first control instruction is sent to control the laser obstacle remover to shut down in time.

[0044] In an embodiment of the present invention, by detecting the yaw angle of the UAV, when the yaw angle is greater than a preset yaw threshold, the laser obstacle remover is promptly controlled to shut down, which helps to save the energy of the laser obstacle remover and improve the safety and intelligence of the obstacle removal of the obstacle remover.

[0045] Optionally, the yaw angle of the UAV is detected in real time. If the absolute value of the yaw angle is not greater than a preset yaw threshold, it indicates that the UAV route has been corrected, and a second control instruction is sent to control the laser obstacle remover to start.

[0046] The technical solution of the embodiment of the present invention is to obtain the obstacle clearing direction data, first position data and second position data of the obstacle to be cleared of the laser obstacle clearer when the laser obstacle clearer carried by the drone is turned on; accurately determine the obstacle clearing direction and locate the position of the laser obstacle clearer and the position of the obstacle to be cleared; then, determine the yaw angle of the drone based on the obstacle clearing direction data, the first position data and the second position data; accurately judge the angle of deviation of the drone from the route; finally, when the absolute value of the yaw angle of the drone is greater than the preset yaw threshold, control the laser obstacle clearer to turn off. When the direction of the drone deviates to the point where it cannot accurately clear obstacles, the laser obstacle clearer is turned off in time, which solves the problem of low safety factor that if the obstacle clearing target is lost or the direction is deviated during the automatic inspection of the drone, the laser obstacle clearer cannot be turned off in time and hits the wires and other important power facilities in the transmission line, and achieves the beneficial effects of improving the safety and stability of the laser obstacle clearer of the drone and improving the automation of the drone inspection.

[0047] Embodiment 2

[0048] Figure 2a The flowchart of a control method of a UAV laser obstacle remover provided in the second embodiment of the present invention is a further refinement of how to control the UAV laser obstacle remover in the above embodiment. Optionally, when the laser obstacle remover carried by the UAV is turned on, it also includes: tracking the obstacle, and when the obstacle is not tracked, controlling the laser obstacle remover to turn off.

[0049] like Figure 2a As shown, the method includes:

[0050] S210: When the laser obstacle remover carried by the UAV is turned on, obtain obstacle removal direction data, first position data, and second position data of the obstacle to be removed by the laser obstacle remover.

[0051] S220: Determine the yaw angle of the UAV based on the obstacle clearance direction data, the first position data, and the second position data.

[0052] S230: When the absolute value of the yaw angle of the UAV is greater than a preset yaw threshold, control the laser obstacle remover to be turned off.

[0053] S240: When the laser obstacle remover carried by the UAV is turned on, the obstacle is tracked; when the obstacle is not tracked, the laser obstacle remover is controlled to be turned off.

[0054] Specifically, after the laser obstacle remover is turned on, the obstacle is tracked based on the target detection algorithm. When it is detected that the obstacle is lost, the laser obstacle remover is turned off in time.

[0055] Optionally, tracking the obstacle includes: acquiring multiple frames of images during the flight of the UAV, detecting obstacles in each frame of the image based on a target detection algorithm to identify obstacle areas in the images; using the image in which the obstacle is first detected as a reference frame, and using multiple images acquired after the reference frame and adjacent to the reference frame as tracking frames; determining the image similarity between the reference frame and the tracking frame, and determining a tracking result of the obstacle based on the image similarity and a preset image similarity threshold, wherein the tracking result includes tracking the obstacle and not tracking the obstacle.

[0056] The reference frame may be understood as an image frame used as a reference, and the tracking frame may be understood as an image frame used for obstacle tracking processing.

[0057] Specifically, during the flight of the drone, multiple frames of images are continuously collected. Each frame of the image is processed using a target detection algorithm to identify and mark the obstacle area in the image, providing an accurate initial position for subsequent tracking. The image in which the obstacle is first detected is used as a reference frame. Multiple images whose acquisition time is after the reference frame and adjacent to the reference frame are used as tracking frames. The image similarity between the reference frame and each tracking frame is calculated, for example, by comparing the feature points, color histograms, texture features, and other methods in the image to calculate the image similarity between the reference frame and each tracking frame. The tracking result is judged based on the calculated image similarity and a preset image similarity threshold.

[0058] For example, a target is tracked based on the target detection algorithm to generate a collection of video frames, defined as T,

[0059]

[0060] in, is the tracking frame, S is the obstacle area, and S0 is the reference frame.

[0061] The obstacle area of ​​the reference frame is expressed as S0 = (x0, y0, w0, h0), the obstacle area of ​​the tracking frame is expressed as S1 = (x1, y1, w1, h1), and the obstacle area of ​​the next tracking frame located after the tracking frame and adjacent to the tracking frame is expressed as S2 = (x2, y2, w2, h2).

[0062] In an embodiment of the present invention, a target detection algorithm is used to identify and mark obstacle areas in an image, separate obstacles from complex backgrounds, and provide an accurate initial position for subsequent tracking. The image in which the obstacle is first detected is used as a reference frame. The initial position and appearance features of the obstacle are provided for subsequent tracking. Multiple images whose acquisition time is after the reference frame and adjacent to the reference frame are used as tracking frames. These images are used to observe the changes of obstacles in continuous time and verify the accuracy of the tracking algorithm. By analyzing the similarity between images and determining the tracking result based on the image similarity and the preset image similarity threshold, it is possible to accurately determine whether the laser obstacle remover can align and remove the obstacle.

[0063] Optionally, determining the image similarity between the reference frame and the tracking frame includes: respectively determining a first hash value of the obstacle area in the reference frame and a second hash value of the obstacle area in the tracking frame; and determining the image similarity between the reference frame and the tracking frame based on the first hash value and the second hash value.

[0064] The hash value can be understood as a value that converts an input of any length into an output value of fixed length through a hash algorithm. The preset image similarity threshold can be preset based on experience, and this embodiment does not limit it.

[0065] Specifically, a first hash value of the obstacle area in the reference frame and a second hash value of the obstacle area in the tracking frame are determined, and the first hash value and the second hash value are compared to determine the image similarity between the reference frame and the tracking frame based on the comparison result.

[0066] Optionally, determining the first hash value of the obstacle area in the reference frame includes: obtaining a local image corresponding to the obstacle area in the reference frame, scaling the local image, and converting the scaled local image into a grayscale image; respectively determining an initial grayscale value of each pixel in the grayscale image, and determining a grayscale average value based on the initial grayscale values ​​of multiple pixels; for each pixel, determining a target grayscale value of the pixel based on the grayscale value of the pixel and the grayscale average value, wherein the target grayscale value is a first preset grayscale value or a second preset grayscale value; combining the target grayscale values ​​of the pixels into a character string of preset digits according to the layout order of the pixels, and determining the combined character string of preset digits as the first hash value of the obstacle area in the reference frame.

[0067] Among them, the first preset grayscale value is 0, and the second preset grayscale value is 1.

[0068] Exemplarily, the steps for calculating the hash value are as follows:

[0069] The first step is to reduce the size. Each obstacle area is reduced to 8×8 size, a total of 64 pixels. The details of the image are removed, only the basic information such as structure and brightness is retained, and the image differences caused by different sizes and proportions are discarded.

[0070] The second step is to simplify the colors and convert the reduced obstacle area into a 64-level grayscale image.

[0071] The third step is to calculate the grayscale average value of 64 pixels in the obstacle area converted into a grayscale image.

[0072] The fourth step is to compare the grayscale of each pixel with its average grayscale value.

[0073] For each pixel, compare the grayscale of each pixel with the grayscale average. If it is greater than or equal to the average, it is recorded as 1; if it is less than the average, it is recorded as 0;

[0074] Step 5: Calculate the hash value. The 64 values ​​obtained in step 4 are combined in the order of pixel layout to form a 64-bit string, which is the hash value of the obstacle area.

[0075] Optionally, determining the second hash value of the obstacle area in the tracking frame includes: obtaining a local image corresponding to the obstacle area in the tracking frame, scaling the local image, and converting the scaled local image into a grayscale image; respectively determining an initial grayscale value of each pixel in the grayscale image, and determining a grayscale average value based on the initial grayscale values ​​of multiple pixels; for each pixel, determining a target grayscale value of the pixel based on the grayscale value of the pixel and the grayscale average value, wherein the target grayscale value is a third preset grayscale value or a fourth preset grayscale value; combining the target grayscale values ​​of the pixels into a character string of preset digits according to the layout order of the pixels, and determining the combined character string of preset digits as the second hash value of the obstacle area in the tracking frame.

[0076] Among them, the third preset grayscale value is 0, and the fourth preset grayscale value is 1.

[0077] Optionally, determining the image similarity between the reference frame and the tracking frame based on the first hash value and the second hash value includes: performing an XOR operation on the first hash value and the second hash value to obtain an XOR result string, wherein the XOR result string includes a first preset value and / or a second preset value; and determining the image similarity between the reference frame and the tracking frame based on the number of the first preset value and / or the second preset value in the XOR result string.

[0078] The first preset value includes 0, and the second preset value includes 1. The XOR result character string can be understood as a binary character string obtained by the XOR operation.

[0079] Specifically, an XOR operation is performed on the first hash value and the second hash value to obtain an XOR result string including 1 and 0. If the number of 0 data bits in the XOR result string is not greater than a preset number threshold, it means that the two pictures are very similar, and the obstacle in the tracking frame is not lost; if the number of 0 data bits in R is greater than the preset number threshold, it means that these are two different pictures, and the obstacle in the tracking frame has been lost. The preset number threshold can be pre-set based on experience, for example: 10, and this embodiment does not limit it.

[0080] Optionally, determining the tracking result of the obstacle based on the image similarity and a preset image similarity threshold includes: determining that the obstacle is not tracked when there are a preset number of consecutive tracking frames whose corresponding image similarities are less than a preset image similarity threshold.

[0081] Specifically, if the obstacle in the tracking frame is lost, the next tracking frame that is located after the tracking frame and adjacent to the tracking frame is updated as a new tracking frame. The new tracking frame is matched with the reference frame for target area image similarity. If the matching result is still that the obstacle in the tracking frame is lost, the tracking frame is updated again. A counter is set to record the number of frames in which obstacles are lost continuously. If the number of frames in which obstacles are lost continuously reaches a preset loss threshold, a corresponding control signal is output to immediately shut down the laser obstacle remover. The preset loss threshold can be pre-set based on experience, for example: 3 times, and this embodiment does not limit it.

[0082] For example, if the obstacle in the tracking frame is lost, the next tracking frame is updated as a new tracking frame, and the image similarity matching is performed again with the reference frame, and this process is repeated three times. A counter is set to record the number of frames where obstacles are lost continuously. If obstacles are lost in three consecutive tracking frames, the main control device outputs a corresponding control signal to immediately shut down the laser obstacle remover.

[0083] In an embodiment of the present invention, the laser obstacle remover is controlled by the number of frames of consecutively lost obstacles to avoid accidental shutdown of the laser obstacle remover, which would affect the normal performance of the obstacle removal task of the laser obstacle remover. When the number of frames of consecutively lost obstacles exceeds a preset loss threshold, the laser obstacle remover is shut down to achieve precise control of the laser obstacle remover.

[0084] Optionally, after controlling the laser obstacle remover to shut down, the method further includes:

[0085] When the obstacle is included in the image acquired after the tracking frame, the laser obstacle remover is controlled to start.

[0086] Specifically, the obstacle is continuously tracked, and the tracking frame is matched with the reference frame to determine whether the obstacle reappears in the tracking frame. If the obstacle is detected to reappear in the tracking frame, the laser obstacle remover is turned on again.

[0087] The technical solution of the embodiment of the present invention tracks the obstacle when the laser obstacle remover carried by the drone is turned on, and controls the laser obstacle remover to be turned off when the obstacle is not tracked. It can further determine whether the obstacle to be cleared is lost, thereby determining whether to turn off the laser obstacle remover in time. The control efficiency and accuracy of the laser obstacle remover are improved.

[0088] Figure 2b A flowchart of an optional example of a control method for a UAV laser obstacle removal machine is provided. Figure 2b As shown, the control method of the UAV laser obstacle remover includes:

[0089] In actual inspections, the target tracking algorithm is used to guide the drone to approach the location of the obstacle clearance target, the indicator laser and laser ranging controls are turned on, the indicator laser and laser ranging are used to determine the distance between the drone and the obstacle clearance target, and the indicator laser is used to accurately aim the laser at the obstacle clearance target.

[0090] When the laser obstacle remover on the drone is turned on, the aiming direction of the laser obstacle remover should always be in the same direction as the direction of the drone's nose. To ensure that the aiming direction of the laser obstacle remover always points to the center of the obstacle, the yaw angle The design process of the set value solver is as follows: define the x-axis in the world coordinate system as a yaw angle of 0°, and calculate the angle between the drone's nose direction (the laser obstacle removal direction) and the x-axis of the world coordinate system.

[0091] The yaw angle of the drone is determined based on the obstacle clearance direction data, the first position data, and the second position data by the following formula:

[0092] l0=(0,1)

[0093] l1=(x f -x,y f -y)

[0094]

[0095] Among them, l0 is the unit vector when the yaw angle of the drone is zero degrees, l1 is the unit vector when the yaw angle of the drone is The vector at the time (x f ,y f ) is the second position data of the obstacle, (x, y) is the first position data of the laser obstacle remover, and θ is the angle between the yaw direction and the preset route direction;

[0096]

[0097] in, is the yaw angle of the drone.

[0098] For example, when the calculated yaw angle of the drone If the laser obstacle remover is judged to be off the route, the main control device will output a corresponding control signal to shut down the laser obstacle remover in time. When the route is corrected, the laser obstacle remover will be turned on again.

[0099] To further determine whether the obstacle is lost, the target detection algorithm is combined to predict the changes in the content of the obstacle area of ​​the tracking frame, and the image similarity between the reference frame and the tracking frame is calculated. The image similarity is compared with the preset image similarity threshold to determine whether the tracked obstacle is lost. If the obstacle clearance target of the tracking frame is lost, the next frame is updated as the tracking frame, and the target area image similarity is performed again with the reference frame. This process is repeated three times. If there is an obstacle loss phenomenon in three consecutive frames, the laser obstacle removal machine is turned off in time.

[0100] For example, a target is tracked based on the target detection algorithm to generate a collection of video frames, defined as T,

[0101]

[0102] in, is the tracking frame, S is the obstacle area, and S0 is the reference frame.

[0103] The obstacle area of ​​the reference frame is expressed as S0 = (x0, y0, w0, h0), the obstacle area of ​​the tracking frame is expressed as S1 = (x1, y1, w1, h1), and the obstacle area of ​​the next tracking frame located after the tracking frame and adjacent to the tracking frame is expressed as S2 = (x2, y2, w2, h2).

[0104] Exemplarily, the steps for calculating the hash value are as follows:

[0105] The first step is to reduce the size. Each obstacle area is reduced to 8×8 size, a total of 64 pixels. The details of the image are removed, only the basic information such as structure and brightness is retained, and the image differences caused by different sizes and proportions are discarded.

[0106] The second step is to simplify the colors and convert the reduced obstacle area into a 64-level grayscale image.

[0107] The third step is to calculate the grayscale average value of 64 pixels in the obstacle area converted into a grayscale image.

[0108] The fourth step is to compare the grayscale of each pixel with its average grayscale value.

[0109] For each pixel, compare the grayscale of each pixel with the grayscale average. If it is greater than or equal to the average, it is recorded as 1; if it is less than the average, it is recorded as 0;

[0110] Step 5: Calculate the hash value. The 64 values ​​obtained in step 4 are combined in the order of pixel layout to form a 64-bit string, which is the hash value of the obstacle area.

[0111] Finally, if the obstacle in the tracking frame is lost, the next tracking frame that is located after the tracking frame and adjacent to the tracking frame is updated as a new tracking frame. The new tracking frame is matched with the reference frame for target area image similarity. If the matching result is still that the obstacle in the tracking frame is lost, the tracking frame is updated again. A counter is set to record the number of frames that continuously lose obstacles. If the number of frames that continuously lose obstacles reaches a preset loss threshold, a corresponding control signal is output to immediately shut down the laser obstacle remover. The preset loss threshold can be pre-set based on experience, for example: 3 times, and this embodiment does not limit it.

[0112] For example, if the obstacle in the tracking frame is lost, the next tracking frame is updated as a new tracking frame, and the image similarity matching is performed again with the reference frame, and this process is repeated three times. A counter is set to record the number of frames where obstacles are lost continuously. If obstacles are lost in three consecutive tracking frames, the main control device outputs a corresponding control signal to immediately shut down the laser obstacle remover.

[0113] Optionally, after controlling the laser obstacle remover to shut down, the method further includes:

[0114] When the obstacle is included in the image acquired after the tracking frame, the laser obstacle remover is controlled to start.

[0115] According to the technical solution of the embodiment of the present invention, the alignment direction of the laser obstacle remover should always point to the center of the obstacle removal target. By calculating the yaw angle, it is determined whether it has deviated from the route and the laser obstacle remover is turned off in time. When the route is corrected, the laser obstacle remover is turned on again. In combination with the target detection algorithm, the changes of obstacles in the tracking frame are tracked, and the image similarity of the reference frame and the tracking frame is calculated. The comparison result between the image similarity and the preset image similarity threshold is used to determine whether the tracked obstacle is lost. If the obstacle is lost, the next frame is updated to the tracking frame, and the similarity is matched with the reference frame again. If there is an obstacle loss phenomenon for three consecutive frames, the laser obstacle remover is turned off in time. It can improve the safety and stability of the laser obstacle remover of the unmanned aerial vehicle, which is conducive to improving the automation level of the unmanned aerial vehicle inspection.

[0116] Embodiment 3

[0117] Figure 3This is a schematic diagram of the structure of a control device for a UAV laser obstacle removal machine provided in the third embodiment of the present invention. Figure 3 As shown, the device includes: a data acquisition module 310, a yaw angle determination module 320 and a wrecker closing module 330.

[0118] Among them, the data acquisition module 310 is used to obtain the obstacle clearance direction data, first position data and second position data of the obstacle to be cleared of the laser obstacle clearer carried by the UAV when the laser obstacle clearer is turned on; the yaw angle determination module 320 is used to determine the yaw angle of the UAV based on the obstacle clearance direction data, the first position data and the second position data; the obstacle clearer closing module 330 is used to control the laser obstacle clearer to close when the absolute value of the yaw angle of the UAV is greater than the preset yaw threshold.

[0119] The technical solution of the embodiment of the present invention is to obtain the obstacle clearing direction data, the first position data and the second position data of the obstacle to be cleared of the laser obstacle clearer through the data acquisition module when the laser obstacle clearer carried by the drone is turned on; accurately determine the obstacle clearing direction and locate the position of the laser obstacle clearer and the position of the obstacle to be cleared; then, determine the yaw angle of the drone based on the obstacle clearing direction data, the first position data and the second position data through the yaw angle determination module; accurately judge the angle of the drone's deviation route; finally, through the obstacle clearer closing module, when the absolute value of the yaw angle of the drone is greater than the preset yaw threshold, control the laser obstacle clearer to close. When the direction of the drone deviates to the point where it cannot accurately clear obstacles, the laser obstacle clearer is closed in time, which solves the problem of low safety factor that if the obstacle clearing target is lost or the direction is offset during the automatic inspection of the drone, the laser obstacle clearer cannot be closed in time and hits the wires and other important power facilities in the transmission line, and the safety factor is low, and the beneficial effect of improving the safety and stability of the laser obstacle clearer of the drone and improving the automation of the drone inspection is achieved.

[0120] Optionally, the yaw angle determination module is specifically used for:

[0121] The yaw angle of the drone is determined based on the obstacle clearance direction data, the first position data and the second position data by the following formula:

[0122] l0=(0,1)

[0123] l1=(x f -x,y f -y)

[0124]

[0125] Among them, l0 is the unit vector when the yaw angle of the drone is zero degrees, l1 is the unit vector when the yaw angle of the drone is The vector at the time (xf ,y f ) is the second position data of the obstacle, (x, y) is the first position data of the laser obstacle remover, and θ is the angle between the yaw direction and the preset route direction;

[0126]

[0127] in, is the yaw angle of the drone.

[0128] Optionally, the device also includes a wrecker control module.

[0129] The obstacle clearer control module is used to track the obstacle when the laser obstacle clearer carried by the UAV is turned on, and to control the laser obstacle clearer to be turned off when the obstacle is not tracked.

[0130] Optionally, the wrecker control module includes:

[0131] The obstacle detection unit is used to collect multiple frames of images during the flight of the UAV, and detect obstacles in each frame of the image based on the target detection algorithm to identify the obstacle area in the image;

[0132] A tracking frame determination unit, configured to use the image in which the obstacle is detected for the first time as a reference frame, and use a plurality of images whose acquisition time is after the reference frame and adjacent to the reference frame as tracking frames;

[0133] The similarity determination unit is used to determine the image similarity between the reference frame and the tracking frame, and determine the tracking result of the obstacle based on the image similarity and a preset image similarity threshold, wherein the tracking result includes tracking the obstacle and not tracking the obstacle.

[0134] Optionally, the similarity determination unit includes:

[0135] A hash value determination subunit, used to respectively determine a first hash value of the obstacle region in the reference frame and a second hash value of the obstacle region in the tracking frame;

[0136] The similarity determination subunit is configured to determine the image similarity between the reference frame and the tracking frame based on the first hash value and the second hash value.

[0137] Optionally, the hash value determination subunit is specifically used for:

[0138] Acquire a local image corresponding to the obstacle area in the reference frame, scale the local image, and convert the scaled local image into a grayscale image;

[0139] Determine the initial grayscale value of each pixel in the grayscale image respectively, and determine the grayscale average value according to the initial grayscale values ​​of multiple pixels;

[0140] For each pixel, determine a target grayscale value of the pixel according to the grayscale value of the pixel and the grayscale average value, wherein the target grayscale value is a first preset grayscale value or a second preset grayscale value;

[0141] The target grayscale values ​​of the pixels are combined into a character string of preset digits according to the arrangement order of the pixels, and the combined character string of preset digits is determined as a first hash value of the obstacle area in the reference frame.

[0142] The similarity determination subunit is specifically used for:

[0143] Performing an XOR operation on the first Hash value and the second Hash value to obtain an XOR result string, wherein the XOR result string includes a first preset value and / or a second preset value;

[0144] The image similarity between the reference frame and the tracking frame is determined based on the number of the first preset value and / or the second preset value in the XOR result character string.

[0145] Optionally, the similarity determination unit is specifically used to:

[0146] When there are a preset number of consecutive tracking frames whose corresponding image similarities are less than a preset image similarity threshold, it is determined that the obstacle is not tracked.

[0147] The device also includes a wrecker opening module.

[0148] The obstacle remover activation module is used to control the laser obstacle remover to be activated after the laser obstacle remover is controlled to be deactivated and, if the image whose acquisition time is after the tracking frame includes the obstacle, activate the laser obstacle remover.

[0149] The control device of the UAV laser obstacle remover provided in the embodiment of the present invention can execute the control method of the UAV laser obstacle remover provided in any embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method.

[0150] Embodiment 4

[0151] Figure 4A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0152] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0153] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0154] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as the method for controlling a drone laser obstacle remover.

[0155] In some embodiments, the control of the method drone laser obstacle remover can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the control of the method drone laser obstacle remover described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the control of the method drone laser obstacle remover in any other suitable manner (for example, by means of firmware).

[0156] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0157] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0158] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0159] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0160] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0161] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0162] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0163] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A control method for a UAV laser obstacle removal machine, characterized in that: include: When the laser obstacle remover carried by the UAV is turned on, obtaining obstacle removal direction data, first position data and second position data of the obstacle to be removed by the laser obstacle remover; Determine a yaw angle of the drone based on the obstacle clearance direction data, the first position data, and the second position data; When the absolute value of the yaw angle of the UAV is greater than a preset yaw threshold, the laser obstacle remover is controlled to be turned off.

2. The method according to claim 1, characterized in that The yaw angle of the drone is determined based on the obstacle clearance direction data, the first position data and the second position data by the following formula: l0＝(0,1)； l1=(x f -x,y f -and); Among them, l0 is the unit vector when the yaw angle of the drone is zero degrees, and l1 is the unit vector when the yaw angle of the drone is The vector at the time (x f ,y f ) is the second position data of the obstacle, (x, y) is the first position data of the laser obstacle remover, and θ is the angle between the yaw direction and the preset route direction; in, is the yaw angle of the drone.

3. The method according to claim 1, characterized in that When the laser obstacle remover on the drone is turned on, it also includes: The obstacle is tracked, and when the obstacle is not tracked, the laser obstacle remover is controlled to be turned off.

4. The method according to claim 3, characterized in that The step of tracking the obstacle comprises: Collect multiple frames of images during the flight of the drone, and detect obstacles in each frame of the image based on a target detection algorithm to identify obstacle areas in the image; The image in which the obstacle is detected for the first time is used as a reference frame, and a plurality of images whose acquisition time is after the reference frame and adjacent to the reference frame are used as tracking frames; The image similarity between the reference frame and the tracking frame is determined, and the tracking result of the obstacle is determined based on the image similarity and a preset image similarity threshold, wherein the tracking result includes tracking the obstacle and not tracking the obstacle.

5. The method according to claim 4, characterized in that The determining of the image similarity between the reference frame and the tracking frame comprises: respectively determining a first hash value of the obstacle region in the reference frame and a second hash value of the obstacle region in the tracking frame; Image similarity between the reference frame and the tracking frame is determined based on the first hash value and the second hash value.

6. The method according to claim 5, characterized in that Determining a first hash value of the obstacle area in the reference frame includes: Acquire a local image corresponding to the obstacle area in the reference frame, scale the local image, and convert the scaled local image into a grayscale image; Determine the initial grayscale value of each pixel in the grayscale image respectively, and determine the grayscale average value according to the initial grayscale values ​​of multiple pixels; For each pixel, determine a target grayscale value of the pixel according to the grayscale value of the pixel and the grayscale average value, wherein the target grayscale value is a first preset grayscale value or a second preset grayscale value; The target grayscale values ​​of the pixels are combined into a character string of preset digits according to the arrangement order of the pixels, and the combined character string of preset digits is determined as a first hash value of the obstacle area in the reference frame.

7. The method according to claim 5, characterized in that Determining the image similarity between the reference frame and the tracking frame based on the first hash value and the second hash value includes: Performing an XOR operation on the first Hash value and the second Hash value to obtain an XOR result string, wherein the XOR result string includes a first preset value and / or a second preset value; The image similarity between the reference frame and the tracking frame is determined based on the number of the first preset value and / or the second preset value in the XOR result character string.

8. The method according to claim 4, characterized in that The determining the tracking result of the obstacle based on the image similarity and a preset image similarity threshold comprises: When there are a preset number of consecutive tracking frames whose corresponding image similarities are less than a preset image similarity threshold, it is determined that the obstacle is not tracked.

9. The method according to claim 6, characterized in that After controlling the laser obstacle remover to shut down, the method further includes: When the obstacle is included in the image acquired after the tracking frame, the laser obstacle remover is controlled to start.

10. A control device for a UAV laser obstacle removal machine, characterized in that: include: A data acquisition module, used for acquiring obstacle removal direction data, first position data and second position data of the obstacle to be removed by the laser obstacle removal device when the laser obstacle removal device carried by the UAV is turned on; A yaw angle determination module, used to determine the yaw angle of the UAV based on the obstacle clearance direction data, the first position data and the second position data; The obstacle removal machine shutdown module is used to control the laser obstacle removal machine to shut down when the absolute value of the yaw angle of the UAV is greater than a preset yaw threshold.

Images