Control method and device of unmanned aerial vehicle laser obstacle remover

Abstract

The application discloses a control method and device of a laser obstacle clearing machine of a UAV. The method comprises the following steps: acquiring obstacle clearing direction data, first position data and second position data of an obstacle to be removed of the laser obstacle clearing machine in the case that the laser obstacle clearing machine carried by the UAV is turned on; determining a yaw angle of the UAV based on the obstacle clearing direction data, the first position data and the second position data; and controlling the laser obstacle clearing machine to be turned off in the case that the absolute value of the yaw angle of the UAV is greater than a preset yaw threshold. The method solves the problem of low control accuracy of the laser obstacle clearing machine and improves the control accuracy of the laser obstacle clearing machine.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of device control, and in particular to a control method and device of a laser obstacle clearing machine of a UAV. BACKGROUND

[0002] A power transmission line is a forbidden area for foreign objects, and in order to remove foreign objects in the power transmission line, a laser obstacle clearing instrument of a UAV is used to carry out laser obstacle clearing work.

[0003] The object attacked by the laser obstacle clearing instrument of the UAV is a kite ribbon, plastic film and other light external objects on the high-voltage power transmission line. The "laser gun" of the laser obstacle clearing instrument uses a high-power density laser generator without water cooling, which can "destroy" the external objects in a very short time. However, this seemingly simple process still faces some difficulties. For example, if the obstacle clearing target is lost or deviates in direction during the automatic inspection process of the UAV, the laser obstacle clearing machine cannot be closed in time and will hit the conductor and other important power facilities in the power transmission line. SUMMARY

[0004] The present application provides a control method and device of a laser obstacle clearing machine of a UAV to solve the problem of low control accuracy of the laser obstacle clearing machine.

[0005] According to an aspect of the present application, a control method of a laser obstacle clearing machine of a UAV is provided, which comprises:

[0006] In the case that the laser obstacle clearing machine carried by the UAV is turned on, the obstacle clearing direction data, the first position data of the laser obstacle clearing machine and the second position data of the obstacle to be removed are acquired;

[0007] The yaw angle of the UAV is determined based on the obstacle clearing direction data, the first position data and the second position data.

[0008] In the case that the absolute value of the yaw angle of the UAV is greater than a preset yaw threshold, the laser obstacle clearing machine is controlled to be turned off.

[0009] According to another aspect of the present application, a control device of a laser obstacle clearing machine of a UAV is provided, which comprises:

[0010] A data acquisition module is configured to acquire, in the case that the laser obstacle clearing machine carried by the UAV is turned on, the obstacle clearing direction data, the first position data of the laser obstacle clearing machine and the second position data of the obstacle to be removed;

[0011] A yaw angle determination module is configured to determine the yaw angle of the UAV based on the obstacle clearing direction data, the first position data and the second position data.

[0012] A laser obstacle clearing machine turning-off module is configured to control the laser obstacle clearing machine to be turned off in the case that the absolute value of the yaw angle of the UAV is greater than a preset yaw threshold.

[0013] According to another aspect of the present application, there is provided an electronic device, comprising:

[0014] at least one processor; and

[0015] a memory connected with the at least one processor in communication; wherein,

[0016] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the control method of the unmanned aerial vehicle laser obstacle clearing machine according to any one of the embodiments of the present application.

[0017] According to another aspect of the present application, there is provided a computer readable storage medium storing computer instructions for enabling a processor to implement the control method of the unmanned aerial vehicle laser obstacle clearing machine according to any one of the embodiments of the present application when executed by the processor.

[0018] The technical solution of the embodiments of the present application, by acquiring the obstacle clearing direction data, the first position data of the laser obstacle clearing machine and the second position data of the obstacle to be cleared when the laser obstacle clearing machine carried by the unmanned aerial vehicle is turned on, accurately determines the obstacle clearing direction and the positions of the laser obstacle clearing machine and the obstacle to be cleared, then determines the yaw angle of the unmanned aerial vehicle based on the obstacle clearing direction data, the first position data and the second position data, accurately judges the angle of the unmanned aerial vehicle deviating from the course, and finally controls the laser obstacle clearing machine to be turned off when the absolute value of the yaw angle of the unmanned aerial vehicle is greater than a preset yaw threshold. In the case that the direction of the unmanned aerial vehicle deviates and cannot accurately clear the obstacle, the laser obstacle clearing machine is turned off in time, which solves the problem that the laser obstacle clearing machine cannot be turned off in time and hits the conductor and other important power facilities in the power transmission line when the obstacle clearing target is lost or the direction deviates during the automatic inspection of the unmanned aerial vehicle, and has the beneficial effects of improving the safety and stability of the unmanned aerial vehicle laser obstacle clearing machine and improving the automation degree of the unmanned aerial vehicle inspection.

[0019] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0021] Figure 1 is a flow chart of a control method of a UAV laser obstacle clearing machine according to an embodiment of the present application;

[0022] Figure 2a is a flow chart of a control method of a UAV laser obstacle clearing machine according to an embodiment of the present application;

[0023] Figure 2b is a flow chart of an optional example of a control method of a UAV laser obstacle clearing machine according to an embodiment of the present application;

[0024] Figure 3 is a structural schematic diagram of a control device of a UAV laser obstacle clearing machine according to an embodiment of the present application;

[0025] Figure 4 is a structural schematic diagram of an electronic device for implementing a control method of a UAV laser obstacle clearing machine according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the personnel in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by the person of ordinary skill in the art without making creative efforts should belong to the scope of protection of the present application.

[0027] It should be noted that the terms “first”, “second” and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms “include” and “have” and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0028] Embodiment one

[0029] Figure 1A flowchart of a control method of a UAV laser obstacle clearing machine is provided for the first embodiment of the present application. The present embodiment can be applied to the control of a UAV laser obstacle clearing machine. The method can be executed by a control device of the UAV laser obstacle clearing machine. The control device of the UAV laser obstacle clearing machine can be realized in the form of hardware and / or software and can be configured in an electronic device. As shown in FIG. 8, the method comprises the following steps. Figure 1

[0030] S110, in the case that the laser obstacle clearing machine carried by the UAV is turned on, obtaining the obstacle clearing direction data of the laser obstacle clearing machine, the first position data and the second position data of the obstacle to be removed.

[0031] The laser obstacle clearing machine can be understood as a device for remotely removing foreign matter on overhead lines using laser technology. The obstacle clearing direction data can be understood as the emission direction of the laser beam of the laser obstacle clearing machine. The obstacle clearing direction data can be expressed in the form of angle or coordinate, such as horizontal angle, vertical angle or three-dimensional space coordinate, etc. The first position data can be understood as the spatial position data of the laser obstacle clearing machine carried by the UAV. The second position data can be understood as the spatial position data of the obstacle to be removed.

[0032] Specifically, the image of the target obstacle is obtained and displayed by the observation system provided on the laser obstacle clearing machine carried by the UAV. The obstacle clearing direction data of the laser obstacle clearing machine is determined based on the obtained image of the target obstacle. Or the direction of the laser obstacle clearing machine is aligned with the direction of the UAV nose, and the obstacle clearing direction data of the laser obstacle clearing machine is determined based on the direction data of the UAV nose. The first position data of the laser obstacle clearing machine can be obtained in real time by the global positioning system on the UAV. The first position data can be expressed by latitude, longitude, height and attitude angle (such as pitch angle, yaw angle, etc.). The three-dimensional point cloud model of the surrounding environment can be constructed by emitting laser beams and receiving reflected signals through the laser radar carried by the UAV. By analyzing the point cloud data, the second position data, shape data and size data of the obstacle can be recognized. The second position data can be expressed in the form of three-dimensional space coordinate, including the center point coordinate, boundary coordinate or contour coordinate of the obstacle, etc.

[0033] S120, determining the yaw angle of the UAV based on the obstacle clearing direction data, the first position data and the second position data.

[0034] Specifically, the horizontal axis in the world coordinate system is defined as the zero yaw angle. The yaw angle of the UAV is determined based on the obstacle clearing direction data, the first position data and the second position data by the following formula:

[0035] l0=(0,1) ​

[0036] l1 = (x f -y, y f )

[0037]

[0038] wherein, l0 is a unit vector when the yaw angle of the unmanned aerial vehicle is zero degree, l1 is a vector when the yaw angle of the unmanned aerial vehicle is (x f ,y f ) is the second position data of the obstacle, (x, y) is the first position data of the laser obstacle clearing machine, and θ is the included angle between the yaw direction and the preset course direction.

[0039]

[0040] wherein, is the yaw angle of the unmanned aerial vehicle.

[0041] S130, in the case that the absolute value of the yaw angle of the unmanned aerial vehicle is greater than a preset yaw threshold, controlling the laser obstacle clearing machine to be closed.

[0042] wherein, the preset yaw threshold can be pre-set according to experience, for example, 15 degrees, and the embodiment does not limit it.

[0043] Specifically, in the case that the absolute value of the calculated yaw angle of the unmanned aerial vehicle is greater than the preset yaw threshold, it indicates that the unmanned aerial vehicle deviates from the course, and at this time, the laser obstacle clearing machine cannot be aligned with the center of the obstacle to be removed, and at this time, if the obstacle is continuously removed, it is easy to hit the conductor in the power transmission line and other important power facilities. Therefore, in the case that the absolute value of the calculated yaw angle of the unmanned aerial vehicle is greater than the preset yaw threshold, the first control instruction is sent to control the laser obstacle clearing machine to be closed in time.

[0044] In the embodiment of the application, by detecting the yaw angle of the unmanned aerial vehicle, in the case that the yaw angle is greater than the preset yaw threshold, the laser obstacle clearing machine is controlled to be closed in time, which helps to save the energy of the laser obstacle clearing machine and improve the safety and intelligence of the obstacle clearing machine.

[0045] Optionally, the yaw angle of the unmanned aerial vehicle is detected in real time, and in the case that the absolute value of the yaw angle is not greater than the preset yaw threshold, it indicates that the course of the unmanned aerial vehicle has been corrected, and the second control instruction is sent to control the laser obstacle clearing machine to be opened.

[0046] The technical scheme of the embodiment of the present application comprises the following steps: obtaining the obstacle-removing direction data, the first position data of the laser obstacle remover and the second position data of the obstacle to be removed when the laser obstacle remover carried by the unmanned aerial vehicle is turned on; accurately determining the obstacle-removing direction and the positions of the laser obstacle remover and the obstacle to be removed; determining the yaw angle of the unmanned aerial vehicle based on the obstacle-removing direction data, the first position data and the second position data; accurately judging the angle of the unmanned aerial vehicle deviating from the flight path; and turning off the laser obstacle remover when the absolute value of the yaw angle of the unmanned aerial vehicle is greater than a preset yaw threshold. In the case that the direction of the unmanned aerial vehicle deviates and the obstacle cannot be accurately removed, the laser obstacle remover is turned off in time, thereby solving the problem that, in the automatic inspection process of the unmanned aerial vehicle, if the obstacle-removing target is lost or the direction deviates, the laser obstacle remover cannot be turned off in time and hits the conductor and other important power facilities in the power transmission line, and the safety coefficient is low, and the beneficial effects of improving the safety and stability of the laser obstacle remover of the unmanned aerial vehicle and improving the automation degree of the unmanned aerial vehicle inspection are achieved.

[0047] Embodiment two

[0048] Figure 2a A flowchart of a control method of a laser obstacle remover of an unmanned aerial vehicle is provided for the second embodiment of the present application. The present embodiment is a further refinement of how to control the laser obstacle remover of the unmanned aerial vehicle in the above-mentioned embodiment. Optionally, when the laser obstacle remover carried by the unmanned aerial vehicle is turned on, the method further comprises: tracking the obstacle, and turning off the laser obstacle remover when the obstacle is not tracked.

[0049] As shown in Figure 2a , the method comprises:

[0050] S210, when the laser obstacle remover carried by the unmanned aerial vehicle is turned on, obtaining the obstacle-removing direction data, the first position data of the laser obstacle remover and the second position data of the obstacle to be removed.

[0051] S220, determining the yaw angle of the unmanned aerial vehicle based on the obstacle-removing direction data, the first position data and the second position data.

[0052] S230, when the absolute value of the yaw angle of the unmanned aerial vehicle is greater than a preset yaw threshold, turning off the laser obstacle remover.

[0053] S240, when the laser obstacle remover carried by the unmanned aerial vehicle is turned on, tracking the obstacle, and turning off the laser obstacle remover when the obstacle is not tracked.

[0054] Specifically, after the laser obstacle remover is turned on, the obstacle is tracked based on a target detection algorithm. In the case that the obstacle is lost, the laser obstacle remover is turned off in time.

[0055] Optionally, the tracking of the obstacle comprises: collecting multiple frames of images during flight of the unmanned aerial vehicle, detecting the obstacle in each of the frames of images based on a target detection algorithm to identify an obstacle region in the images, taking an image in which the obstacle is first detected as a reference frame, taking multiple images collected after the reference frame and adjacent to the reference frame as tracking frames, determining image similarity between the reference frame and the tracking frames, and determining a tracking result of the obstacle based on the image similarity and a preset image similarity threshold, wherein the tracking result comprises tracking of the obstacle and non-tracking of the obstacle.

[0056] The reference frame can be understood as an image frame used as a reference, and the tracking frame can be understood as an image frame used for obstacle tracking processing.

[0057] Specifically, during flight of the unmanned aerial vehicle, multiple frames of images are continuously collected. Each frame of image is processed using a target detection algorithm to identify and identify an obstacle region in the image, to provide an accurate initial position for subsequent tracking. An image in which the obstacle is first detected is taken as a reference frame. Multiple images collected after the reference frame and adjacent to the reference frame are taken as tracking frames. Image similarity between the reference frame and each tracking frame is calculated, for example, feature points, color histograms, texture features, and the like in the image are compared to calculate the image similarity between the reference frame and each tracking frame. The tracking result is determined according to the calculated image similarity and a preset image similarity threshold.

[0058] Illustratively, a target tracking video frame set is generated based on a target detection algorithm, defined as T,

[0059]

[0060] wherein, is the tracking frame, S is the obstacle region, and S0 is the reference frame.

[0061] The obstacle region of the reference frame is represented as S0=(x0, y0, w0, h0), the obstacle region of the tracking frame is represented as S1=(x1, y1, w1, h1), and the obstacle region of the next tracking frame located after the tracking frame and adjacent to the tracking frame is represented as S2=(x2, y2, w2, h2).

[0062] In the embodiment of the present application, the target detection algorithm is used to identify and label the obstacle region in the image, separate the obstacle from the complex background, and provide an accurate initial position for subsequent tracking. The image in which the obstacle is first detected is taken as a reference frame. The initial position and appearance characteristics of the obstacle are provided for subsequent tracking. A plurality of images collected after the reference frame and adjacent to the reference frame are taken as tracking frames. These images are used to observe the changes of the obstacle in continuous time and verify the accuracy of the tracking algorithm. The similarity between the images is analyzed, and the tracking result is determined based on the image similarity and a preset image similarity threshold, so that whether the laser obstacle removing machine can aim at and remove the obstacle can be accurately judged.

[0063] Optionally, the image similarity between the reference frame and the tracking frame is determined by: determining a first hash value of the obstacle region in the reference frame and a second hash value of the obstacle region in the tracking frame, respectively; and determining the image similarity between the reference frame and the tracking frame based on the first hash value and the second hash value.

[0064] The hash value can be understood as a kind of input of arbitrary length being transformed into a fixed length output value through a hash algorithm. The preset image similarity threshold can be pre-set according to experience, and the embodiment does not limit it.

[0065] Specifically, the first hash value of the obstacle region in the reference frame and the second hash value of the obstacle region in the tracking frame are determined, the first hash value and the second hash value are compared, and the image similarity between the reference frame and the tracking frame is determined based on the comparison result.

[0066] Optionally, the first hash value of the obstacle region in the reference frame is determined by: obtaining a local image corresponding to the obstacle region in the reference frame, performing scaling processing on the local image, and converting the scaled local image into a gray image; determining an initial gray value of each pixel point in the gray image, and determining a gray average value according to the initial gray values of a plurality of pixel points; for each pixel point, determining a target gray value of the pixel point according to the gray value of the pixel point and the gray average value, wherein the target gray value is a first preset gray value or a second preset gray value; combining the target gray value of the pixel point into a string of a preset number of bits according to the layout order of the pixel points, and determining the string of the preset number of bits obtained by combination as the first hash value of the obstacle region in the reference frame.

[0067] The first preset gray value is 0, and the second preset gray value is 1.

[0068] Exemplarily, the steps of calculating the hash value are as follows:

[0069] First, reduce the size. Reduce the obstacle area to 8x8 size, a total of 64 pixels. Remove the details of the picture, only keep the basic information such as structure, light and shade, and discard the picture differences caused by different sizes and proportions.

[0070] Second, simplify the color. Convert the reduced obstacle area to a 64-level grayscale image.

[0071] Third, calculate the average gray value of the 64 pixels in the obstacle area converted to a grayscale image.

[0072] Fourth, compare the gray value of each pixel with the average gray value.

[0073] For each pixel, compare the gray value of each pixel with the average gray value. If it is greater than or equal to the average value, it is recorded as 1; if it is less than the average value, it is recorded as 0;

[0074] Fifth, calculate the hash value. Combine the 64 numerical values obtained in the fourth step into a 64-bit string according to the layout order of the pixel points, which is the hash value of the obstacle area.

[0075] Optionally, determining the second hash value of the obstacle area in the tracking frame comprises: obtaining a local image corresponding to the obstacle area in the tracking frame, performing scaling processing on the local image, and converting the scaled local image into a grayscale image; determining an initial gray value of each pixel point in the grayscale image, and determining a gray average value according to the initial gray values of a plurality of pixel points; for each pixel point, determining a target gray value of the pixel according to the gray value of the pixel point and the gray average value, wherein the target gray value is a third preset gray value or a fourth preset gray value; combining the target gray value of the pixel point into a string of a preset number of bits according to the layout order of the pixel points, and determining the string of the preset number of bits obtained by combination as the second hash value of the obstacle area in the tracking frame.

[0076] Wherein, the third preset gray value is 0, and the fourth preset gray value is 1.

[0077] Optionally, determining the image similarity between the reference frame and the tracking frame based on the first hash value and the second hash value comprises: performing XOR operation on the first hash value and the second hash value to obtain an XOR result string, wherein the XOR result string includes a first preset value and / or a second preset value; determining the image similarity between the reference frame and the tracking frame based on the number of the first preset value and / or the second preset value in the XOR result string.

[0078] Wherein, the first preset value includes 0, and the second preset value includes 1. The XOR result string can be understood as a binary string obtained by XOR operation.

[0079] Specifically, the first hash value and the second hash value are subjected to an exclusive OR operation to obtain an exclusive OR result string including 1 and 0. If the number of 0 data bits in the exclusive OR result string is not greater than a preset number threshold, it is indicated that the two pictures are very similar, and it is indicated that the obstacle of the tracking frame is not lost. If the number of 0 data bits in the exclusive OR result string is greater than the preset number threshold, it is indicated that the two pictures are different, and it is indicated that the obstacle of the tracking frame has been lost. The preset number threshold can be pre-set according to experience, for example, 10, which is not limited in the embodiment.

[0080] Optionally, the tracking result of the obstacle is determined based on the image similarity and a preset image similarity threshold, including: in the case that the image similarity corresponding to a preset number of continuous tracking frames is less than the preset image similarity threshold, determining that the obstacle is not tracked.

[0081] Specifically, if the obstacle of the tracking frame is lost, the next frame tracking frame located after the tracking frame and adjacent to the tracking frame is updated to a new tracking frame. The new tracking frame is subjected to target region image similarity matching with the reference frame. If the matching result is still that the obstacle of the tracking frame is lost, the tracking frame is updated again. A counter is set to record the number of frames in which the obstacle is continuously lost. If the number of frames in which the obstacle is continuously lost reaches a preset loss threshold, a corresponding control signal is output, and the laser obstacle clearing machine is immediately turned off. The preset loss threshold can be pre-set according to experience, for example, 3 times, which is not limited in the embodiment.

[0082] Specifically, if the obstacle of the tracking frame is lost, the next frame tracking frame is updated to a new tracking frame, and image similarity matching with the reference frame is performed again. This process is continuously performed for 3 times. A counter is set to record the number of frames in which the obstacle is continuously lost. If the obstacle is lost in the three continuous tracking frames, a corresponding control signal is output by the master control device, and the laser obstacle clearing machine is immediately turned off.

[0083] In the embodiment of the application, the laser obstacle clearing machine is controlled by the number of frames in which the obstacle is continuously lost, so as to avoid false closing of the laser obstacle clearing machine and affect the normal performance of the obstacle clearing task of the laser obstacle clearing machine. When the number of frames in which the obstacle is continuously lost exceeds a preset loss threshold, the laser obstacle clearing machine is turned off, so as to realize precise control of the laser obstacle clearing machine.

[0084] Optionally, after the laser obstacle clearing machine is controlled to be turned off, the method further includes:

[0085] In the case that the obstacle is included in the image collected after the tracking frame, the laser obstacle clearing machine is controlled to be turned on.

[0086] Specifically, the obstacle is tracked continuously, and the tracking frame is matched with the reference frame to determine whether the obstacle reappears in the tracking frame. If the obstacle is detected to reappear in the tracking frame, the laser obstacle clearing machine is started again.

[0087] The technical scheme of the embodiment of the application determines whether the obstacle is tracked, and controls the laser obstacle clearing machine to be turned off if the obstacle is not tracked. Whether the obstacle to be removed is lost can be further determined, so as to determine whether the laser obstacle clearing machine is timely turned off. The control efficiency and accuracy of the laser obstacle clearing machine are improved.

[0088] Figure 2b An optional flowchart of a control method of a laser obstacle clearing machine of a UAV is provided. As shown in Figure 2b , the control method of the laser obstacle clearing machine of the UAV comprises:

[0089] In actual inspection, the target tracking algorithm is used to guide the UAV to approach the position of the obstacle clearing target, the indicating laser and the laser ranging control are turned on, the distance between the UAV and the obstacle clearing target is determined by using the indicating laser and the laser ranging, and the laser is accurately aligned to the obstacle clearing target by using the indicating laser.

[0090] In the case that the laser obstacle clearing machine carried by the UAV is turned on, the alignment direction of the laser obstacle clearing machine should always be the same as the direction of the UAV head, so as to ensure that the alignment direction of the laser obstacle clearing machine always points to the center of the obstacle. The yaw angle The design process of the set value solver is as follows: the x axis in the world coordinate system is defined as the 0° yaw angle, and the included angle between the UAV head direction (the obstacle clearing direction of the laser obstacle clearing machine) and the x axis in the world coordinate system is calculated by the following formula.

[0091] The yaw angle of the UAV is determined based on the obstacle clearing direction data, the first position data and the second position data by the following formula:

[0092] l0=(0,1)

[0093] l1=(x f -x,y f -y)

[0094]

[0095] Wherein, l0 is a unit vector when the yaw angle of the UAV is zero degrees, l1 is a vector when the yaw angle of the UAV is , (x f ,y f ) is the second position data of the obstacle, (x,y) is the first position data of the laser obstacle clearing machine, and θ is the included angle between the yaw direction and the preset flight line direction.

[0096]

[0097] wherein, is the yaw angle of the UAV.

[0098] For example, when the calculated yaw angle of the UAV is greater than 5 degrees, it is determined that the UAV deviates from the flight path, and the corresponding control signal is output by the master control device to timely turn off the laser obstacle remover. When the flight path is corrected, the laser obstacle remover is turned on again.

[0099] To further determine whether the obstacle is lost, the target detection algorithm is combined to predict the change of the content of the obstacle region in the tracking frame, the image similarity between the reference frame and the tracking frame is calculated, and the comparison result of the image similarity and the preset image similarity threshold is determined to determine whether the tracked obstacle is lost. If the tracking obstacle target is lost, the next frame is updated as the tracking frame, and the target region image similarity is calculated again with the reference frame. This process is continuously performed three times, and if there is an obstacle loss phenomenon in three consecutive frames, the laser obstacle remover is turned off in time.

[0100] For example, based on the target detection algorithm, a video frame set is generated by tracking a target, which is defined as T,

[0101]

[0102] wherein, is the tracking frame, S is the obstacle region, and S0 is the reference frame.

[0103] The obstacle region of the reference frame is represented as S0=(x0, y0, w0, h0), the obstacle region of the tracking frame is represented as S1=(x1, y1, w1, h1), and the obstacle region of the next frame tracking frame located after the tracking frame and adjacent to the tracking frame is represented as S2=(x2, y2, w2, h2).

[0104] For example, the steps of calculating the hash value are as follows:

[0105] First, reduce the size. Reduce the obstacle region to 8x8 size, a total of 64 pixels. Remove the details of the picture, only keep the basic information such as structure, brightness, etc., and discard the picture differences caused by different sizes and proportions.

[0106] Second, simplify the color. The reduced obstacle region is converted to a 64-level grayscale image.

[0107] Third, calculate the average gray value of the 64 pixels in the converted grayscale obstacle region.

[0108] Fourth, compare the gray value of each pixel with the average gray value.

[0109] ​For each pixel, the gray scale of each pixel is compared with the average gray scale. If greater than or equal to the average, it is recorded as 1; if less than the average, it is recorded as 0;

[0110] In the fifth step, the hash value is calculated. The 64 numerical values obtained in the fourth step are combined according to the layout order of the pixel points to form a 64-bit string, which is the hash value corresponding to the obstacle region.

[0111] Finally, if the tracked frame obstacle is lost, the next frame tracking frame located after the tracking frame and adjacent to the tracking frame is updated as a new tracking frame. The new tracking frame is subjected to target region image similarity matching with the reference frame. If the matching result is still that the tracked frame obstacle is lost, the tracking frame is updated again. A counter is set to record the number of consecutive obstacle loss frames. If the number of consecutive obstacle loss frames reaches a preset loss threshold, a corresponding control signal is output, and the laser obstacle clearing machine is immediately turned off. The preset loss threshold can be pre-set according to experience, for example, 3 times, which is not limited in the embodiment.

[0112] For example, if the tracked frame obstacle is lost, the next frame tracking frame is updated as a new tracking frame, and image similarity matching with the reference frame is performed again. This process is continuously performed for 3 times. A counter is set to record the number of consecutive obstacle loss frames. If the obstacle loss phenomenon exists in the three consecutive tracking frames, a corresponding control signal is output through the main control device, and the laser obstacle clearing machine is immediately turned off.

[0113] Optionally, after the laser obstacle clearing machine is controlled to be turned off, the method further comprises:

[0114] In the case that the obstacle is included in the image collected after the tracking frame, the laser obstacle clearing machine is controlled to be turned on.

[0115] The technical scheme of the embodiment of the application is that the alignment direction of the laser obstacle clearing machine should always point to the center of the obstacle clearing target. Whether the laser obstacle clearing machine deviates from the flight path is judged by calculating the drift angle, and the laser obstacle clearing machine is turned off in time. After the flight path is corrected, the laser obstacle clearing machine is turned on again. The change of the obstacle in the tracking frame is tracked in combination with the target detection algorithm. The image similarity of the reference frame and the tracking frame is calculated. Whether the tracked obstacle is lost is determined according to the comparison result of the image similarity and the preset image similarity threshold. If the obstacle is lost, the next frame is updated as a tracking frame, and similarity matching with the reference frame is performed again. If the obstacle loss phenomenon exists in the three consecutive frames, the laser obstacle clearing machine is turned off in time. The safety and stability of the unmanned aerial vehicle laser obstacle clearing machine can be improved, and the automation degree of the unmanned aerial vehicle inspection can be improved.

[0116] Embodiment three

[0117] Figure 3This is a schematic diagram of the structure of a control device for a UAV laser obstacle removal machine provided by the third embodiment of the present invention. Figure 3 As shown, the device includes: a data acquisition module 310 , a yaw angle determination module 320 and a wrecker closing module 330 .

[0118] Among them, the data acquisition module 310 is used to obtain the obstacle clearance direction data, first position data and second position data of the obstacle to be cleared of the laser obstacle clearer carried by the UAV when the laser obstacle clearer is turned on; the yaw angle determination module 320 is used to determine the yaw angle of the UAV based on the obstacle clearance direction data, the first position data and the second position data; the obstacle clearer shutdown module 330 is used to control the laser obstacle clearer to shut down when the absolute value of the yaw angle of the UAV is greater than the preset yaw threshold.

[0119] The technical solution of the embodiment of the present invention is to obtain the obstacle clearance direction data, first position data and second position data of the obstacle to be cleared of the laser obstacle clearer carried by the drone through the data acquisition module when the laser obstacle clearer is turned on; accurately determine the obstacle clearance direction and locate the position of the laser obstacle clearer and the position of the obstacle to be cleared; then, determine the yaw angle of the drone based on the obstacle clearance direction data, the first position data and the second position data through the yaw angle determination module; accurately judge the angle of the drone's deviation from the route; finally, control the laser obstacle clearer to be shut down through the obstacle clearer shutdown module when the absolute value of the drone's yaw angle is greater than a preset yaw threshold. When the drone's direction deviates to the point where it cannot accurately clear the obstacle, the laser obstacle clearer is shut down in time, solving the problem of low safety factor caused by the laser obstacle clearer not being able to shut down in time if the obstacle clearance target is lost or the direction deviates during the drone's automatic inspection process, thereby improving the safety and stability of the drone's laser obstacle clearer and increasing the degree of automation of drone inspections.

[0120] Optionally, the yaw angle determination module is specifically used to:

[0121] The yaw angle of the drone is determined based on the obstacle clearance direction data, the first position data, and the second position data using the following formula:

[0122] l0=(0,1)

[0123] l1=(x f -x,y f -y)

[0124]

[0125] Among them, l0 is the unit vector when the yaw angle of the drone is zero degrees, l1 is the unit vector when the yaw angle of the drone is The vector when (xf (x,y) f (x,y) is first position data of the laser obstacle clearer, and θ is an included angle between a yaw direction and a preset course direction;

[0126]

[0127] wherein, is a yaw angle of the UAV.

[0128] Optionally, the device further comprises an obstacle clearer control module.

[0129] The obstacle clearer control module is configured to, in a case where the laser obstacle clearer carried by the UAV is turned on, track the obstacle, and in a case where the obstacle is not tracked, control the laser obstacle clearer to be turned off.

[0130] Optionally, the obstacle clearer control module comprises:

[0131] An obstacle detection unit is configured to, in a flight process of the UAV, collect multiple images, and detect an obstacle in each of the images based on a target detection algorithm to identify an obstacle region in the images.

[0132] A tracking frame determination unit is configured to take an image in which the obstacle is first detected as a reference frame, and take multiple images collected after the reference frame and adjacent to the reference frame as tracking frames.

[0133] A similarity determination unit is configured to determine an image similarity between the reference frame and the tracking frames, and determine a tracking result of the obstacle based on the image similarity and a preset image similarity threshold, wherein the tracking result comprises tracking the obstacle and not tracking the obstacle.

[0134] Optionally, the similarity determination unit comprises:

[0135] A hash value determination subunit is configured to determine a first hash value of the obstacle region in the reference frame and a second hash value of the obstacle region in the tracking frames.

[0136] A similarity determination subunit is configured to determine the image similarity between the reference frame and the tracking frames based on the first hash value and the second hash value.

[0137] Optionally, the hash value determination subunit is specifically configured to:

[0138] Obtain a local image corresponding to the obstacle region in the reference frame, scale the local image, and convert the scaled local image into a grayscale image.

[0139] Determine an initial gray value of each pixel point in the gray image respectively, and determine a gray average value according to the initial gray values of the plurality of pixel points;

[0140] For each pixel point, determine a target gray value of the pixel according to the gray value of the pixel point and the gray average value, wherein the target gray value is a first preset gray value or a second preset gray value;

[0141] Combine the target gray value of the pixel point into a preset bit string according to the layout order of the pixel point, and determine the preset bit string obtained by combination as a first hash value of the obstacle region in the reference frame.

[0142] The similarity determination subunit is specifically used for:

[0143] XOR operation is performed on the first hash value and the second hash value to obtain an XOR result string, wherein the XOR result string includes a first preset value and / or a second preset value;

[0144] The image similarity between the reference frame and the tracking frame is determined based on the number of the first preset value and / or the second preset value in the XOR result string.

[0145] Optionally, the similarity determination unit is specifically used for:

[0146] In the case that a preset number of continuous tracking frames correspond to the image similarity less than a preset image similarity threshold, it is determined that the obstacle is not tracked.

[0147] The device further includes a barrier removing machine starting module.

[0148] The barrier removing machine starting module is configured to, after the laser barrier removing machine is controlled to be closed, in the case that the image collected at a time after the tracking frame includes the obstacle, control the laser barrier removing machine to be started.

[0149] The control device of the unmanned aerial vehicle laser barrier removing machine provided in the embodiments of the present application can execute the control method of the unmanned aerial vehicle laser barrier removing machine provided in any embodiment of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0150] Embodiment four

[0151] Figure 4A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0152] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0153] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0154] The processor 11 can be various general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the method for controlling a drone laser obstacle removal machine.

[0155] In some embodiments, the method of controlling the unmanned aerial laser obstacle remover can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., storage unit 18. In some embodiments, parts or all of the computer program can be loaded and / or installed onto electronic device 10 via, e.g., ROM 12 and / or communication unit 19. When the computer program is loaded onto RAM 13 and executed by processor 11, one or more steps of the method of controlling the unmanned aerial laser obstacle remover described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the method of controlling the unmanned aerial laser obstacle remover by any other suitable means, e.g., with the aid of firmware.

[0156] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0157] Computer programs used to implement the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program running on the processor implements the functions / operations specified in the flowcharts and / or the block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as part of a standalone software package, or entirely on a remote machine or server.

[0158] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0159] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0160] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0161] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0162] It should be understood that the various forms of flow shown above can be used to reorder, add or delete steps. For example, each step described in the present application can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which is not limited herein.

[0163] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A control method of a UAV laser obstacle remover, characterized in that, Comprise: In the case that the laser obstacle remover carried by the unmanned aerial vehicle is turned on, obtain the obstacle removing direction data, the first position data of the laser obstacle remover and the second position data of the obstacle to be removed; Determine the yaw angle of the unmanned aerial vehicle based on the obstacle removing direction data, the first position data and the second position data; Determine the yaw angle of the unmanned aerial vehicle based on the obstacle removing direction data, the first position data and the second position data by the following formula: ； ； ； wherein, is a unit vector when the yaw angle of the UAV is zero degrees, is a vector when the yaw angle of the UAV is degrees, is second position data of the obstacle, is first position data of the laser obstacle remover, is an included angle between the yaw direction and the preset flight line direction; ； wherein, is the yaw angle of the drone; In the case that the absolute value of the yaw angle of the unmanned aerial vehicle is greater than the preset yaw threshold, control the laser obstacle remover to be turned off.

2. The method of claim 1, wherein, In the case that the laser obstacle remover carried by the unmanned aerial vehicle is turned on, further comprising: Track the obstacle, and control the laser obstacle remover to be turned off in the case that the obstacle is not tracked.

3. The method of claim 2, wherein, The tracking of the obstacle comprises: Collect multiple images during the flight of the unmanned aerial vehicle, and detect the obstacle in each of the images based on a target detection algorithm to identify the obstacle region in the image; Take the image in which the obstacle is first detected as a reference frame, and take multiple images collected after the reference frame and adjacent to the reference frame as tracking frames; Determine the image similarity between the reference frame and the tracking frame, and determine the tracking result of the obstacle based on the image similarity and a preset image similarity threshold, wherein the tracking result comprises tracking the obstacle and not tracking the obstacle.

4. The method of claim 3, wherein, The determination of the image similarity between the reference frame and the tracking frame comprises: Determine the first hash value of the obstacle region in the reference frame and the second hash value of the obstacle region in the tracking frame respectively; Determine the image similarity between the reference frame and the tracking frame based on the first hash value and the second hash value.

5. The method of claim 4, wherein, The determination of the first hash value of the obstacle region in the reference frame comprises: Obtain the local image corresponding to the obstacle region in the reference frame, scale the local image, and convert the scaled local image into a grayscale image; Determine the initial grayscale value of each pixel point in the grayscale image, and determine the grayscale average value according to the initial grayscale values of multiple pixel points; For each pixel point, determine the target grayscale value of the pixel point according to the grayscale value of the pixel point and the grayscale average value, wherein the target grayscale value is a first preset grayscale value or a second preset grayscale value; Combine the target grayscale values of the pixel points into a string of a preset number of bits according to the layout order of the pixel points, and determine the string of the preset number of bits obtained by the combination as the first hash value of the obstacle region in the reference frame.

6. The method of claim 4, wherein, Determination of the image similarity between the reference frame and the tracking frame based on the first hash value and the second hash value comprises: Perform XOR operation on the first hash value and the second hash value to obtain an XOR result string, wherein the XOR result string comprises a first preset numerical value and / or a second preset numerical value; Determine the image similarity between the reference frame and the tracking frame based on the number of the first preset numerical value and / or the second preset numerical value in the XOR result string.

7. The method of claim 3, wherein, The tracking result of the obstacle is determined based on the image similarity and a preset image similarity threshold, and the determination includes: In a case where the image similarity corresponding to a preset number of continuous tracking frames is less than the preset image similarity threshold, it is determined that the obstacle is not tracked.

8. The method of claim 5, wherein, After the control of the laser obstacle removing machine is closed, the method further includes: In a case where the image collected at the time after the tracking frame includes the obstacle, the laser obstacle removing machine is controlled to be opened.

9. A control device of a UAV laser obstacle clearing machine, characterized in that, The method includes: A data acquisition module is configured to acquire obstacle removing direction data, first position data, and second position data of an obstacle to be removed in a case where the laser obstacle removing machine carried by the unmanned aerial vehicle is opened. A yaw angle determination module is configured to determine a yaw angle of the unmanned aerial vehicle based on the obstacle removing direction data, the first position data, and the second position data. The yaw angle determination module is specifically configured to: The yaw angle of the unmanned aerial vehicle is determined based on the obstacle removing direction data, the first position data, and the second position data by using the following formula: ； ； ； in, is the unit vector when the yaw angle of the drone is zero degrees, The yaw angle of the UAV is The vector of time, is the second position data of the obstacle, is the first position data of the laser obstacle remover, is the angle between the yaw direction and the preset course direction; ； wherein, is the yaw angle of the drone; An obstacle removing machine closing module is configured to control the laser obstacle removing machine to be closed in a case where the absolute value of the yaw angle of the unmanned aerial vehicle is greater than a preset yaw threshold.

Images