Worksite block deviation correction method and apparatus, remote control device, and storage medium

Abstract

The application discloses a work plot deviation rectification method and device, a remote control device and a storage medium, and relates to the technical field of unmanned equipment. The method comprises the following steps: acquiring first position information of a deviation rectification reference point in a work plot, and controlling an unmanned equipment to fly to above the deviation rectification reference point according to the first position information; acquiring a first image and a first height associated with the deviation rectification reference point, the first image being an image captured by an earth camera when the unmanned equipment collects the first position information, and the first height being a flight height when the unmanned equipment collects the first image; controlling the unmanned equipment to fly to the first height, and displaying the first image and a real-time preview picture of the earth camera on a remote control interface; adjusting the position of the unmanned equipment to match the real-time preview picture with the first image; acquiring second position information of the unmanned equipment under the condition that the real-time preview picture is matched with the first image, and adjusting the position information of the work plot according to the offset between the first position information and the second position information.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned equipment, and particularly relates to a work plot deviation rectification method and device, a remote control equipment and a storage medium. BACKGROUND

[0002] With the development of unmanned equipment technology, more and more unmanned equipment is applied to various plots for high-altitude operation. The staff can remotely control the unmanned equipment to fly over the work plot to mark points to collect the position information of the work plot, and then plan a flight route for the unmanned equipment according to the position information of the work plot, so that the unmanned equipment can work on the work plot according to the flight route. When the positioning system of the unmanned equipment adopts a self-difference technology for positioning, the positioning information at the same position when marking points and working after the positioning system is restarted or continuously operated for a long time will have errors, and therefore the position of the work plot needs to be rectified to improve the work accuracy of the unmanned equipment.

[0003] In the prior art, the staff needs to control the unmanned equipment to fly to the actual position of the deviation rectification reference point for repositioning when rectifying, but when there are many plot files, or when the person who creates the plot and the person who works is not the same person, the staff cannot clearly know the actual position of the deviation rectification reference point of each plot, and therefore cannot smoothly complete the plot rectification. SUMMARY

[0004] The present application provides a work plot deviation rectification method and device, a remote control equipment and a storage medium, which use a first image taken by the unmanned equipment when marking a deviation rectification reference point to assist the unmanned equipment to accurately fly to the actual position of the deviation rectification reference point, solve the problem that the staff cannot smoothly complete the plot rectification due to the inability to know the actual position of the deviation rectification reference point in the prior art, reduce the rectification difficulty and improve the rectification efficiency, ensure the position accuracy of the work plot after rectification, and further ensure the work accuracy of the unmanned equipment.

[0005] In a first aspect, the present application provides a work plot deviation rectification method, which comprises:

[0006] obtaining first position information of a deviation rectification reference point in a work plot, and controlling an unmanned equipment to fly above the deviation rectification reference point according to the first position information;

[0007] obtaining a first image and a first height associated with the deviation rectification reference point, the first image being an image taken by a downward camera of the unmanned equipment when collecting the first position information, and the first height being a flight height of the unmanned equipment when collecting the first image;

[0008] control the unmanned device to fly to the first height, and display the first image and a real-time preview picture of the ground camera on a remote control interface;

[0009] adjust the position of the unmanned device to match the real-time preview picture with the first image;

[0010] In a case where it is determined that the real-time preview picture matches the first image, obtain second position information of the unmanned device, and adjust position information of the work plot according to an offset between the first position information and the second position information.

[0011] In a second aspect, the present application provides a work plot deviation rectification device, comprising:

[0012] A first flight control module is configured to obtain first position information of a deviation rectification reference point in a work plot, and control an unmanned device to fly to above the deviation rectification reference point according to the first position information.

[0013] A reference data acquisition module is configured to obtain a first image and a first height associated with the deviation rectification reference point, the first image being an image taken by a ground camera when the unmanned device collects the first position information, and the first height being a flight height when the unmanned device collects the first image.

[0014] An interface display module is configured to control the unmanned device to fly to the first height, and display the first image and a real-time preview picture of the ground camera on a remote control interface.

[0015] A second flight control module is configured to adjust the position of the unmanned device to match the real-time preview picture with the first image.

[0016] A position deviation rectification module is configured to, in a case where it is determined that the real-time preview picture matches the first image, obtain second position information of the unmanned device, and adjust position information of the work plot according to an offset between the first position information and the second position information.

[0017] In a third aspect, the present application provides a remote control device, comprising:

[0018] one or more processors; a memory storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the work plot deviation rectification method according to the first aspect.

[0019] In a fourth aspect, the present application provides a storage medium containing computer executable instructions, when the computer executable instructions are executed by a computer processor, the computer executable instructions are used to execute the work plot deviation rectification method according to the first aspect.

[0020] In the present application, the first position information obtained when the unmanned device dots the deviation correction reference point is used to control the unmanned device to fly to the vicinity of the deviation correction reference point, the unmanned device is adjusted to the first height when the unmanned device dots the deviation correction reference point, and the first image captured by the downward camera of the unmanned device when the unmanned device dots the deviation correction reference point is displayed on the remote control interface. At the same time, the downward camera of the unmanned device is turned on, and the real-time preview picture of the downward camera is displayed on the remote control interface together with the first image, and the user can control the unmanned device to move by referring to the first image and the real-time preview picture, or the remote control device controls the unmanned device to move by itself according to the matching result. When the unmanned device accurately moves above the deviation correction reference point, the first image is the same as the real-time preview picture, and the second position information of the unmanned device obtained at this time is the working position that can be collected when the unmanned device locates the deviation correction reference point during work. According to the offset between the dotting position and the working position of the deviation correction reference point, the position information of the working plot generated by dotting is corrected, and the position information of the working plot that can be located by the unmanned device during work is obtained. When the unmanned device works according to the position information of the working plot after correction, the unmanned device can accurately fly to the working plot and work on the working plot, realizing the precise work of the unmanned device. Through the above technical means, the user can quickly control the unmanned device to move above the deviation correction reference point by referring to the first image and the real-time preview picture, greatly reducing the difficulty of deviation correction and improving the efficiency of deviation correction. Moreover, under the assistance of the first image, the unmanned device can accurately move to the position where the first image is captured, ensuring that the unmanned device is located at the same position during dotting and deviation correction, and correcting the working plot based on the offset between the dotting position and the actual position collected at the same position before and after, ensuring the position accuracy of the working plot after correction, and further ensuring the work accuracy of the unmanned device. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 is a flowchart of a working plot deviation correction method provided by an embodiment of the present application;

[0022] Figure 2 is a schematic diagram of a working plot provided by an embodiment of the present application;

[0023] Figure 3 is a flowchart of saving the first image and the first height of the deviation correction reference point provided by an embodiment of the present application;

[0024] Figure 4 is a flowchart of controlling the unmanned device to move above the boundary point provided by an embodiment of the present application;

[0025] Figure 5 is a first schematic diagram of a remote control interface provided by an embodiment of the present application;

[0026] Figure 6 is a second schematic diagram of a remote control interface provided by an embodiment of the present application;

[0027] Figure 7 is a flowchart of controlling the unmanned device to move to the deviation correction reference point provided by an embodiment of the present application;

[0028] Figure 8 is a flowchart of determining whether the first image is the same as the real-time preview picture provided by an embodiment of the present application;

[0029] Figure 9 is a structural schematic diagram of a work plot deviation correction device provided by an embodiment of the present application;

[0030] Figure 10 is a structural schematic diagram of a remote control device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the objects, technical solutions and advantages of the present application clearer, the following further describes the specific embodiments of the present application in conjunction with the drawings. It can be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, only the parts related to the present application are shown in the drawings, but not all the contents. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted by flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when the operations are completed, but can also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0032] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of a kind, and do not limit the number of objects, for example, the first object can be one or more. In addition, the specification and claims "and / or" indicate at least one of the connected objects, and the character " / ", generally indicates that the objects before and after are in a "or" relationship.

[0033] The job plot rectification method provided in this embodiment can be executed by a remote control device, which can be realized by software and / or hardware, and can be composed of two or more physical entities or one physical entity. For example, the remote control device can be a screen-equipped remote controller for controlling a unmanned device or a mobile phone or tablet computer loaded with a remote control application.

[0034] The remote control device is installed with at least one type of operating system, and the remote control device can install at least one application program based on the operating system. The application program can be an application program provided by the operating system or an application program downloaded from a third-party device or server. In this embodiment, the remote control device has at least an application program that can execute the job plot rectification method.

[0035] For ease of understanding, the remote control device is taken as an example to describe the main body for executing the job plot rectification method in this embodiment.

[0036] In one embodiment, the worker selects a boundary point in the job plot as a rectification reference point through the remote control device, and flies the unmanned device above the rectification reference point according to the geographical features of the rectification reference point in memory to collect the job position of the rectification reference point. The job position of the rectification reference point can be understood as the position of the rectification reference point that can be collected by the positioning system when the unmanned device works on the job plot. The remote control device rectifies the position information of the job plot generated by the dotting according to the deviation between the dotting position and the job position of the rectification reference point, so as to ensure that the unmanned device can accurately fly above the job plot based on the position information of the rectified job plot when working on the job plot. If the remote control device stores more job plots or the dotting and rectification of the job plot are not operated by the same worker, the worker cannot guarantee that he can clearly know the geographical features of the rectification reference point, that is, the worker cannot clearly know the actual position of the rectification reference point of each plot. When the worker remotely controls the unmanned device, it is difficult to remotely control the unmanned device to the actual position of the rectification reference point, and the rectification difficulty is high and the position accuracy of the rectified job plot cannot be guaranteed.

[0037] To solve the problem that it is difficult to control the unmanned device to accurately and correctly fly above the boundary point in the prior art, the present embodiment provides a job plot rectification method.

[0038] Figure 1 A flowchart of the job plot rectification method provided by the present embodiment is given. Referring to Figure 1 , the job plot rectification method specifically includes:

[0039] S110, acquiring first position information of a rectification reference point in a job plot, and controlling a unmanned device to fly above the rectification reference point according to the first position information.

[0040] The deviation correction reference point is a point of the work plot set by the embodiment for correcting the deviation of the work plot caused by restarting the self-differential positioning or not restarting but using the self-differential positioning for a long time. The deviation correction reference point can be a boundary point of the work plot or a marker in the work plot.

[0041] The embodiment takes a certain boundary point of the work plot as the deviation correction reference point for description. Figure 2 is a schematic diagram of the work plot provided by the embodiment of the application. As shown in Figure 2 The worker controls the unmanned device to fly over the boundary points A, B, C and D of the work plot 11 to mark points to collect the position information of the boundary points, and generates the position information of the work plot 11 according to the position information of the boundary points. When the worker controls the unmanned device to fly over the boundary point A by the remote control device, the worker selects the boundary point A as the deviation correction reference point, and determines the position information of the boundary point A collected when the unmanned device marks points as the first position information of the deviation correction reference point.

[0042] When the unmanned device marks points over the boundary points, the positioning system of the unmanned device performs positioning by the self-differential technology to obtain the horizontal coordinates of the unmanned device, so as to take the horizontal coordinates as the position information of the corresponding boundary points. The self-differential technology can be understood as a technology that the unmanned device performs convergence calculation based on the position reference point to enter the real-time kinematic (RTK) positioning of the position of the unmanned device. Since the position reference point changes after each power-off restart of the positioning system, and the self-differential technology accumulates a large error after continuous operation for a long time, causing calculation divergence, the positioning system has a large deviation after each power-off restart or continuous operation for a long time. For reference Figure 2 If the positioning system has a large deviation, the unmanned device may fly to the point A' when flying according to the position information of the boundary point A, that is, the unmanned device cannot accurately work on the work plot according to the position information of the work plot generated when marking points, which affects the work effect of the unmanned device. Therefore, after each power-off restart or continuous operation for a long time, if the unmanned device is to work on the work plot, the work position information of the deviation correction reference point is collected first, the work position information is offset from the first position information of the deviation correction reference point collected when marking points, the position information of the work plot generated when marking points is corrected according to the offset, the position information of the work plot generated when marking points is corrected to the work position information of the work plot that can be collected when the unmanned device works, so that the unmanned device can accurately fly over the work plot when working according to the work position information of the work plot after correction, and the work precision of the unmanned device is ensured. For reference Figure 2When the work plot is rectified based on the offset of the rectification reference point, it is equivalent to moving the work plot 12 to the work plot 11, and accordingly, when the positioning system of the unmanned device navigates according to the position information of the rectified work plot, it can accurately fly above the work plot 11.

[0043] It should be noted that the unmanned device can immediately perform work on the work plot after dotting the work plot, at which time the positioning system may not have been restarted or continuously operated for a long time, i.e., the unmanned device can perform work on the work plot based on the position information of the work plot generated by dotting. In this regard, before performing work on the work plot, it can be determined whether a preset first rectification condition is met, and if the preset rectification condition is met, the work plot is rectified, and if the preset first rectification condition is not met, the position information of the work plot generated by dotting is used. The first rectification condition is that the positioning system is restarted after dotting is completed or the running time of the positioning system after dotting is completed exceeds a preset time threshold, and the preset time threshold is the longest time for which the positioning system continuously operates without error. In addition, the unmanned device can also perform two or more consecutive works on the same work plot, and the positioning system may not have been restarted or continuously operated for a long time during the two works, and thus the unmanned device can perform the work based on the position information of the work plot used in the previous work. For example, in the case where the preset first rectification condition is met, it is determined whether a preset second rectification condition is met, and the second rectification condition is whether the positioning system is restarted or the running time exceeds the preset time threshold after the previous rectification is completed. If the second rectification condition is met, it indicates that the position information of the work plot generated by the previous rectification has deviated, and thus the work position information of the rectification reference point for the current work needs to be re-acquired, and the position information of the work plot generated during dotting is rectified based on the work position information and the first position information. If the second rectification condition is not met, it indicates that the position information of the work plot generated by the previous rectification has not deviated, and thus the position information of the work plot generated by the previous rectification is used for the current work.

[0044] In this embodiment, in order for the unmanned device to currently acquire the work position information of the rectification reference point, the remote control device first controls the unmanned device to fly above the rectification reference point based on the first position information of the rectification reference point. It should be noted that since the first position information deviates from the work position information of the rectification reference point, the unmanned device can only fly above the rectification reference point based on the first position information, and subsequently the unmanned device can be manually controlled by the staff to fly above the rectification reference point or automatically controlled by the remote control device to move to the rectification reference point.

[0045] S120, acquire the first image and the first height associated with the deviation correction reference point, the first image is an image captured by the nadir camera when the unmanned device collects the first position information, and the first height is the flight height when the unmanned device collects the first image.

[0046] For example, when the unmanned device flies over the deviation correction reference point for dotting, the nadir camera of the unmanned device can capture the first image, and the GPS (Global Positioning System), the nadir radar or the barometer of the unmanned device can be used to determine the first height when the first image is captured. Although the positioning system can also calculate the height information when the unmanned device captures the first image through the self-difference technology, the error of the height information is large, and therefore, the embodiment preferably uses other ways to determine the height information when the unmanned device captures the first image.

[0047] It is easy to understand that when the unmanned device is at the same height above the deviation correction reference point, the images captured by the nadir camera are the same, and the geographical features near the deviation correction reference point are clearly recorded in the images captured by the nadir camera. Therefore, the embodiment proposes that the first image can be used to assist the unmanned device to fly to the same position when the deviation correction reference point is dotted, and then the unmanned device is controlled to be positioned through the self-difference technology to obtain the working position information of the deviation correction reference point.

[0048] In an embodiment, Figure 3 is a flowchart for saving the first image and the first height of the deviation correction reference point provided by the embodiment. As shown in Figure 3 , the step of saving the first image and the first height of the deviation correction reference point specifically includes S1601-S1603:

[0049] S1601, control the unmanned device to fly over the boundary point of the working land, and control the unmanned device to dot to collect the position information of the boundary point.

[0050] Referring to Figure 2 , the staff controls the unmanned device to fly over the boundary point A of the working land through the remote control device, sends a dotting instruction to the unmanned device through the remote control device, and the unmanned device determines the horizontal position information of the unmanned device itself through the self-difference technology after receiving the dotting instruction, and sends the horizontal position information of the unmanned device itself to the remote control device. The remote control device receives the horizontal position information as the position information of the boundary point A.

[0051] In an embodiment, the staff member controls the unmanned device to fly above the boundary point according to the satellite map displayed on the remote control interface. However, due to the low accuracy of the satellite map, the staff member may need to observe the site by naked eyes to compare the position of the unmanned device with the position of the boundary point, so as to move the unmanned device above the boundary point. In another embodiment, the ground camera of the unmanned device can be turned on, and the real-time preview picture of the ground camera is displayed on the remote control interface, so that the staff member can move the unmanned device above the boundary point by referring to the real-time preview picture. Exemplarily, Figure 4 is a flowchart provided by an embodiment of the present application for controlling the unmanned device to move above the boundary point. As shown in Figure 4 , the step of controlling the unmanned device to move above the boundary point specifically includes S16011-S16012:

[0052] S16011, display the real-time preview picture of the ground camera on the remote control interface.

[0053] S16012, generate a first moving instruction in response to a first input operation, and send the first moving instruction to the unmanned device, so that the unmanned device moves according to the first moving instruction, and the boundary point is located at the center of the real-time preview picture.

[0054] , the first input operation refers to the operation input by the staff member when controlling the joystick of the remote control device to move the unmanned device to the dotting position, and the first moving instruction refers to the instruction generated by the remote control device to control the unmanned device to move in response to the first input operation. When the unmanned device is located above the boundary point, the boundary point is located at the center of the real-time preview picture. Exemplarily, the remote control device sends an instruction to the unmanned device to turn on the ground camera, and the unmanned device obtains the real-time preview picture collected by the ground camera after turning on the ground camera, and transmits the real-time preview picture to the remote control device, so that the remote control device displays the real-time preview picture on the remote control interface. Figure 5 is a first schematic diagram of the remote control interface provided by an embodiment of the present application. As shown in Figure 5As shown, the remote control interface 13 displays a satellite map 14, a real-time preview screen 16, a dotting control 18 and a first joystick control 17, the satellite map 14 displays a positioning icon 15 of the unmanned device, and the cross center point of the real-time preview screen 16 corresponds to the position of the unmanned device. When the first joystick control 17 is touched, the remote control device inputs a first input operation, and the remote control device generates a first movement instruction for controlling the unmanned device to move a corresponding distance in response to the first input operation, and sends the first movement instruction to the unmanned device, so that the unmanned device moves a corresponding distance according to the first movement instruction. For example, when the staff remotely controls the unmanned device to dot the boundary point A of the work plot, the staff can refer to the satellite map 14 to move the unmanned device to the upper air of the boundary point A through the first joystick control 17, and then refer to the real-time preview screen 16 to fine-tune the position of the unmanned device through the first joystick control 17, so that the cross center point of the real-time preview screen 16 is at the same position as the boundary point A. At this time, the unmanned device is located above the boundary point. When the cross center point of the real-time preview screen 16 is at the same position as the boundary point A, the staff clicks the dotting control to make the remote control device send a dotting instruction to the unmanned device. After receiving the dotting instruction, the unmanned device determines its position information through the self-difference technology and sends the position information to the remote control device. The remote control device takes the received position information as the position information of the boundary point A.

[0055] It should be noted that, compared with the staff comparing the positions of the unmanned device and the boundary point by naked eyes, comparing the positions of the unmanned device and the boundary point through the real-time preview screen improves the convenience of the staff in controlling the unmanned device to move to the boundary point, and is conducive to improving the dotting efficiency of the work plot. Moreover, when the unmanned device is located above the boundary point as the correction reference point to shoot the first image, the correction reference point is located at the center of the first image, and when the work plot is corrected, the unmanned device can be quickly moved to above the correction reference point according to the correction reference point located at the center of the first image, which is conducive to improving the correction efficiency of the work plot.

[0056] S1602, select a boundary point as a correction reference point, and control the unmanned device to shoot a first image through the ground camera and determine a first height of the unmanned device when shooting the first image.

[0057] For example, after the remote control device obtains the position information of the boundary point A collected by the unmanned device, a window is popped up on the remote control interface of the remote control device. The window displays the text "whether to take the boundary point A as the deviation correction reference point" and the selection controls "yes" and "no". When the worker clicks the selection control "yes", the remote control device selects the boundary point A as the deviation correction reference point in response to the current selection operation input by the worker, and sends an auxiliary image shooting instruction to the unmanned device. After the unmanned device receives the auxiliary image shooting instruction, the unmanned device shoots a first image above the boundary point by using the ground camera, determines a first height of the unmanned device when the first image is shot by using the barometer, the ground radar or the GPS, and sends the first image and the first height to the remote control device. When the worker clicks the selection control "no", the remote control device does not take the boundary point A as the deviation correction reference point in response to the current selection operation input by the worker, and moves the unmanned device above the boundary point B according to the current remote control operation input by the worker.

[0058] In an embodiment, the remote control device pops up a window to determine whether to select a boundary point as a deviation correction reference point each time the unmanned device collects the position information of the boundary point, that is, one or more boundary points can be selected as the deviation correction reference points for each work plot, so that the next time the work plot is worked, the deviation correction reference point closer to the takeoff point or the deviation correction reference point with more obvious geographical features is selected to correct the work plot, which is beneficial to improve the deviation correction efficiency of the work plot. In another embodiment, when the unmanned device selects a boundary point as a deviation correction reference point, the window is no longer popped up to avoid the window from being frequently popped up to affect the dotting efficiency of the work plot. When the window is no longer popped up, the window is displayed in the form of a control on the remote control interface. When the user clicks the control corresponding to the window, the remote control interface pops up the window again.

[0059] It should be noted that the "yes" and "no" controls displayed by the window are only one embodiment of selecting the deviation correction reference point. In addition to the window display mode, the deviation correction reference point can also be selected by other modes, and the present embodiment is not limited in this regard.

[0060] S1603, determining first position information of the deviation correction reference point according to the position information of the boundary point, and saving the first image, the first height and the first position information in association with the deviation correction reference point.

[0061] For example, after the user selects boundary point A as the correction reference point through the remote control device and controls the unmanned equipment to capture the first image and determine the first height, the position information of boundary point A is determined as the first position information of the correction reference point, and the first position information, the first image, and the first height are associated and saved with the correction reference point. After the remote control device generates the position information of the work area based on the position information of each boundary point collected by the unmanned equipment, the first position information, the first image, and the first height of the work area are associated and saved with the correction reference point.

[0062] The above embodiment describes the process of unmanned equipment marking the work site. After marking, the unmanned equipment may start working on the work site immediately, or it may start working on the work site after several days. Therefore, the remote control equipment does not necessarily have to execute steps S110-S150 immediately after executing steps S1601-S1603, and the execution time of these steps is not necessarily continuous.

[0063] In this embodiment, when correcting a work site, the location information of the work site and the correction reference points associated with and saved with the work site are obtained. If multiple correction reference points are associated with the work site, the remote control device can determine the correction reference point closest to the takeoff position based on the current takeoff position of the unmanned equipment and the first location information of the correction reference points. A first image and a first altitude of the correction reference point closest to the takeoff position are obtained, so that the operator can control the unmanned equipment to move above the correction reference point based on the geographical features recorded in the first image. Since this embodiment selects the correction reference point closest to the takeoff position of the unmanned equipment, the time required for the unmanned equipment to move above the correction reference point can be shortened. Moreover, the first location information records the approximate location of the correction reference point, and the first image records the geographical features of the correction reference point. Even if the operator is unfamiliar with the geographical features of the correction reference point, they can still successfully control the unmanned equipment to move above the correction reference point, greatly reducing the difficulty of correction and improving the correction efficiency of the work site.

[0064] S130: Adjust the unmanned equipment to the first altitude and display the first image and the real-time preview of the ground camera on the remote control interface.

[0065] For example, after the unmanned device flies to the first location information, the remote control device generates an altitude adjustment command based on the first altitude and sends the altitude adjustment command to the unmanned device. The unmanned device adjusts its altitude to the first altitude according to the altitude adjustment command. After adjusting its altitude to the first altitude, the unmanned device activates its ground-facing camera and captures a real-time preview image, which is then sent to the remote control device. The remote control device simultaneously displays the received real-time preview image and the first image on the remote control interface. Figure 6 This is a second schematic diagram of the remote control interface provided in an embodiment of this application. For example...Figure 6 As shown, the real-time preview picture 16, the first image 19, the confirmation control 20 and the second joystick control 21 are displayed in the remote control interface 13, the cross center point is displayed in the real-time preview picture 16 and the first image 19, the cross center point of the real-time preview picture 16 is the position of the unmanned device, and the cross center point of the first image 19 is the deviation correction reference point. The staff can compare the real-time preview picture 16 and the first image 19, and then move the unmanned device above the deviation correction reference point through the remote control device.

[0066] S140, adjusting the position of the unmanned device to match the real-time preview picture with the first image.

[0067] In this embodiment, before confirming that the real-time preview picture is the same as the first image, the staff can manually control the unmanned device to move above the deviation correction reference point, or the remote control device controls the unmanned device to move above the deviation correction reference point based on the feature deviation between the real-time preview picture and the first image.

[0068] In an embodiment, when the staff manually controls the unmanned device to move above the deviation correction reference point, the staff controls the joystick of the remote control device to input a second input operation to the remote control device, and the remote control device generates a second movement instruction in response to the second input operation, sends the second movement instruction to the unmanned device, so that the unmanned device moves according to the second movement instruction. Wherein, the second input operation refers to the operation input by the staff when controlling the joystick of the remote control device to move the unmanned device to the deviation correction reference point, and the second movement instruction refers to the instruction generated by the remote control device in response to the second input operation to control the unmanned device to move. Reference Figure 6 When the second joystick control 21 is touched, the remote control device inputs the second input operation, and the remote control device generates a second movement instruction for controlling the unmanned device to move a corresponding distance in response to the second input operation, and sends the second movement instruction to the unmanned device, so that the unmanned device moves according to the second movement instruction. For example, when the staff checks the real-time preview picture 16 and finds that the boundary point A is still a certain distance away from the unmanned device, the unmanned device can be moved in the direction of the boundary point A through the second joystick control 21. When the staff adjusts the position of the unmanned device through the second joystick control 21, the position of the ground camera changes, and the real-time preview picture 16 collected by the ground camera also changes, and the position of the boundary point A in the real-time preview picture 16 also changes. The staff can refer to the control direction of the second joystick control 21 and the moving direction of the boundary point A in the real-time preview picture 16 to analyze how to control the second joystick control 21 to move the boundary point A in the real-time preview picture 16 to the cross center point, which reduces the operation difficulty of controlling the unmanned device to move above the deviation correction reference point, and is beneficial to improve the deviation correction efficiency and ensure the deviation correction accuracy.

[0069] It needs to be explained that when the remote control device determines that the real-time preview picture is the same as the first image, the remote control device prohibits sending the second movement instruction to the unmanned device, so as to avoid the unmanned device deviating from the rectification reference point due to the mis-touching of the worker when the unmanned device moves above the rectification reference point, and ensure the accuracy of rectification.

[0070] In another embodiment, Figure 7 is a flowchart provided by the embodiment of the application for controlling the unmanned device to move to the rectification reference point. As shown in the figure, Figure 7 the step of controlling the unmanned device to move to the rectification reference point specifically includes S1401-S1402:

[0071] S1401, determine the feature matching pair between the first image and the real-time preview picture, and determine the translation parameter between the first image and the preview picture according to the feature matching pair.

[0072] S1402, determine the movement distance and movement direction of the unmanned device according to the translation parameter, and send the movement distance and movement direction to the unmanned device, so that the unmanned device moves according to the movement distance and movement direction.

[0073] Exemplarily, the first feature points are extracted from the first image and the second feature points are extracted from the real-time preview picture by the SIFT feature operator, the first feature points and the second feature points are matched to take the matched first feature points and second feature points as the feature matching pair. The translation parameter between the first image and the preview picture is determined according to a plurality of feature matching pairs. Since the ground camera always faces the opposite side and the height of the ground camera for shooting the first image and the real-time preview picture is the same, the translation parameter can be understood as the relative translation transformation matrix between the camera pose when the ground camera shoots the first image and the camera pose when the ground camera shoots the real-time preview picture. Based on the translation parameter and the coordinate system transformation matrix of the ground camera and the unmanned device, the movement distance and movement direction of the unmanned device from the position of shooting the real-time preview picture to the position of shooting the first image can be determined. The movement distance and movement direction are sent to the unmanned device, so that the unmanned device moves to the position of shooting the first image, i.e. to the rectification reference point, in the corresponding movement direction by the corresponding movement distance.

[0074] S150, in the case of determining that the real-time preview picture matches the first image, obtaining second position information of the unmanned device, and adjusting the position information of the work plot according to the offset amount of the first position information and the second position information.

[0075] Since the current height of the unmanned device is the same as the first height when the unmanned device takes the first image, the real-time preview picture collected by the ground camera is the same as the first image when the unmanned device moves to the same horizontal position as the deviation reference point, and thus whether the unmanned device moves to above the deviation reference point can be determined by comparing whether the real-time preview picture is the same as the first image.

[0076] In an embodiment, a worker compares whether the real-time preview picture is the same as the first image or whether the cross center point of the real-time preview picture is above the deviation reference point as the cross center point of the first image by naked eyes. Referring to Figure 6 , the worker can know from the first image 19 that the deviation reference point is the boundary point A, and accordingly, when the worker observes that the cross center point of the real-time preview picture 16 coincides with the boundary point A, it can be determined that the real-time preview picture is approximately the same as the first image, and further that the unmanned device is located above the deviation reference point. At this time, the worker can click the confirmation control 20, and the remote control device determines that the real-time preview picture is the same as the first image in response to the triggering operation of the confirmation control 20, and then sends a positioning instruction to the unmanned device. After receiving the positioning instruction, the unmanned device determines its second position information by the self-difference technology and sends the second position information to the remote control device. The remote control device takes the received second position information as the working position information of the deviation reference point. The second position information can be understood as the horizontal position information of the unmanned device when it is located above the deviation reference point.

[0077] In another embodiment, since the worker may make a mistake when comparing the real-time preview picture and the first image by naked eyes, the worker can automatically determine whether the real-time preview picture and the first image match by comparing the features of the real-time preview picture and the first image, so as to avoid introducing a mistake by the worker. Exemplarily, Figure 8 is a flowchart provided by an embodiment of the present application for determining whether the first image and the real-time preview picture match. As Figure 8 indicated, the step of determining whether the first image and the real-time preview picture match specifically includes S1501-S1503:

[0078] S1501, performing feature matching on the real-time preview picture and the first image to determine a feature matching degree of the real-time preview picture and the first image.

[0079] S1502, in a case where the feature matching degree is greater than or equal to a preset matching threshold, determining that the real-time preview picture and the first image match.

[0080] S1503, in a case where the feature matching degree is less than the preset matching threshold, determining that the real-time preview picture and the first image do not match.

[0081] Exemplarily, the first feature vector is extracted from the real-time preview picture and the second feature vector is extracted from the first image, a cosine distance of the first feature vector and the second feature vector is calculated, and the cosine distance is taken as a feature matching degree of the real-time preview picture and the first image. In the embodiment, the preset matching threshold can be regarded as a minimum feature matching degree when the first image is matched with the real-time preview picture. If the feature matching degree of the current real-time preview picture and the first image is greater than or equal to the preset matching threshold, it is determined that the current real-time preview picture is matched with the first image, and at this time, the remote control device can automatically send a positioning instruction to the unmanned device to obtain second position information of the unmanned device. If the feature matching degree of the current real-time preview picture and the first image is less than the preset matching threshold, it is determined that the current real-time preview picture is not matched with the first image, and then the remote control device can further adjust the position of the unmanned device until the unmanned device moves above the deviation correction reference point.

[0082] In the embodiment, when the work position information of the deviation correction reference point is obtained, an offset amount of the first position information of the deviation correction reference point and the work position information is calculated, and each position information of the work plot is adjusted based on the offset amount to obtain the position information of the work plot that can be collected by the unmanned device when the work plot is worked, and then the flight route of the unmanned device is planned according to the position information of the work plot when the work plot is worked, so as to ensure the work accuracy of the unmanned device.

[0083] In summary, the job plot deviation correction method provided in the embodiments of the present application controls the unmanned device to fly to the vicinity of the deviation correction reference point based on the first position information obtained when the unmanned device dots the deviation correction reference point, adjusts the unmanned device to the first height when the unmanned device dots the deviation correction reference point, and displays the first image captured by the ground camera when the unmanned device dots the deviation correction reference point on the remote control interface. At the same time, the ground camera of the unmanned device is turned on, and the real-time preview image of the ground camera is displayed on the remote control interface together with the first image. The user can control the unmanned device to move by referring to the first image and the real-time preview image, or the remote control device controls the unmanned device to move by itself according to the matching result. When the unmanned device accurately moves above the deviation correction reference point, the first image is the same as the real-time preview image, and the second position information of the unmanned device obtained at this time is the job position that can be collected by the unmanned device when positioning the deviation correction reference point during work. According to the offset between the dotting position and the job position of the deviation correction reference point, the position information of the job plot generated by dotting is corrected, and the position information of the job plot that can be positioned by the unmanned device during work is obtained. When the unmanned device works according to the position information of the job plot after correction, the unmanned device can accurately fly to the job plot and work on the job plot, realizing accurate work of the unmanned device. Through the above technical means, the user can quickly control the unmanned device to move above the deviation correction reference point by referring to the first image and the real-time preview image, greatly reducing the difficulty of deviation correction and improving the efficiency of deviation correction. Moreover, under the assistance of the first image, the unmanned device can accurately move to the position where the first image is captured, ensuring that the unmanned device is located at the same position during dotting and deviation correction. Based on the offset between the dotting position and the actual position collected at the same position before and after, the job plot is corrected, ensuring the position accuracy of the job plot after correction, and further ensuring the work accuracy of the unmanned device.

[0084] On the basis of the above embodiments, Figure 9 A structural schematic diagram of a job plot deviation correction device provided in the embodiments of the present application is shown in FIG. 1. Referring to Figure 9 The job plot deviation correction device provided in the embodiments of the present application specifically includes a first flight control module 31, a reference data acquisition module 32, an interface display module 33, a second flight control module 34, and a position correction module 35.

[0085] The first flight control module is configured to obtain first position information of a deviation correction reference point in a job plot, and control an unmanned device to fly above the deviation correction reference point according to the first position information.

[0086] The reference data acquisition module is configured to acquire a first image and a first height associated with the deviation correction reference point, the first image being an image captured by the ground camera when the unmanned device collects the first position information, and the first height being a flight height when the unmanned device collects the first image;

[0087] The interface display module is configured to control the unmanned device to fly to the first height and display the first image and a real-time preview image of the ground camera on a remote control interface.

[0088] The second flight control module is configured to adjust the position of the unmanned device to match the real-time preview image with the first image.

[0089] The position deviation correction module is configured to acquire second position information of the unmanned device when it is determined that the real-time preview image matches the first image, and adjust the position information of the work plot according to the offset between the first position information and the second position information.

[0090] On the basis of the above embodiment, the work plot deviation correction device further comprises a dotting module, which comprises: a dotting control unit configured to control the unmanned device to fly above the boundary point of the work plot, and control the unmanned device to dot to collect the position information of the boundary point; an auxiliary data acquisition unit configured to select the boundary point as a deviation correction reference point, and control the unmanned device to capture the first image through the ground camera, and determine the first height when the unmanned device captures the first image; and an auxiliary data storage unit configured to determine the first position information of the deviation correction reference point according to the position information of the boundary point, and store the first image, the first height and the first position information in association with the deviation correction reference point.

[0091] On the basis of the above embodiment, the dotting control unit comprises: a first preview display subunit configured to display a real-time preview image of the ground camera on a remote control interface; and a first movement control subunit configured to generate a first movement instruction in response to a first input operation, and send the first movement instruction to the unmanned device, so that the unmanned device moves according to the first movement instruction and the boundary point is located at the center of the real-time preview image.

[0092] On the basis of the above embodiment, the second flight control module comprises: a remote control unit configured to generate a second movement instruction in response to a second input operation before it is determined that the real-time preview image is the same as the first image, and send the second movement instruction to the unmanned device, so that the unmanned device moves according to the second movement instruction.

[0093] On the basis of the above-mentioned embodiments, the second flight control module comprises: a translation parameter determination unit configured to determine a feature matching pair between the first image and the real-time preview picture before determining that the preview picture is the same as the first image, and determine a translation parameter between the first image and the preview picture according to the feature matching pair; a movement parameter determination unit configured to determine a movement distance and a movement direction of the unmanned device according to the translation parameter, and send the movement distance and the movement direction to the unmanned device to make the unmanned device move according to the movement distance and the movement direction.

[0094] On the basis of the above-mentioned embodiments, the position correction module comprises: a matching degree determination unit configured to perform feature matching between the real-time preview picture and the first image, and determine a feature matching degree of the real-time preview picture and the first image; a judgment unit configured to determine that the real-time preview picture is the same as the first image in the case that the feature matching degree is greater than or equal to a preset matching threshold.

[0095] On the basis of the above-mentioned embodiments, the positioning system of the unmanned device adopts a self-difference technology for positioning.

[0096] On the basis of the above-mentioned embodiments, the work plot correction device further comprises a correction judgment module configured to, before obtaining the first position information of the correction reference point in the work plot, judge whether a preset first correction condition is met before the work plot is worked, and the correction condition is that the positioning system is restarted after the dotting is completed or the running time of the positioning system after the dotting is completed exceeds a preset time threshold.

[0097] According to the above, the work plot rectification device provided by the embodiments of the present application controls the unmanned device to fly to the vicinity of the rectification reference point based on the first position information obtained when the unmanned device dots the rectification reference point, adjusts the unmanned device to the first height when the unmanned device dots the rectification reference point, and displays the first image captured by the ground camera when the unmanned device dots the rectification reference point on the remote control interface. At the same time, the ground camera of the unmanned device is turned on, and the real-time preview image of the ground camera is displayed on the remote control interface together with the first image. The user can control the unmanned device to move by referring to the first image and the real-time preview image, or the remote control device controls the unmanned device to move by itself according to the matching result. When the unmanned device accurately moves above the rectification reference point, the first image is the same as the real-time preview image. At this time, the second position information of the unmanned device obtained is the work position that can be collected by the unmanned device when the rectification reference point is positioned during work. According to the offset between the dotting position of the rectification reference point and the work position, the position information of the work plot generated by dotting is rectified to obtain the position information of the work plot that can be positioned by the unmanned device during work. When the unmanned device works according to the position information of the rectified work plot, the unmanned device can accurately fly to the work plot and work on the work plot, realizing accurate work of the unmanned device. Through the above technical means, the user can quickly control the unmanned device to move above the rectification reference point by referring to the first image and the real-time preview image, greatly reducing the rectification difficulty and improving the rectification efficiency. Moreover, with the aid of the first image, the unmanned device can accurately move to the position where the first image is captured, ensuring that the unmanned device is located at the same position during dotting and rectification. Based on the offset between the dotting position and the actual position collected at the same position before and after, the work plot is rectified, ensuring the position accuracy of the rectified work plot and further ensuring the work accuracy of the unmanned device.

[0098] The work plot rectification device provided by the embodiments of the present application can be used to execute the work plot rectification method provided by the above embodiments, and has corresponding functions and advantages.

[0099] Figure 10 FIG. 1 is a structural schematic diagram of a remote control device provided by an embodiment of the present application, which is used to execute the work plot rectification method provided by the above embodiments. Figure 10 The remote control device includes a processor 41, a memory 42, a communication device 43, an input device 44, and an output device 45. The number of processors 41 in the remote control device can be one or more, and the number of memories 42 in the remote control device can be one or more. The processor 41, the memory 42, the communication device 43, the input device 44, and the output device 45 of the remote control device can be connected through a bus or other means.

[0100] The memory 42, as a computer readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the working field rectification method of any embodiment of the present application (for example, the first flight control module 31, the reference data acquisition module 32, the interface display module 33, the second flight control module 34 and the position rectification module 35 in the working field rectification device). The memory 42 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required by a function; the data storage area can store data created according to the use of the device, etc. In addition, the memory 42 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state memory device. In some examples, the memory can further include a memory remotely arranged with respect to the processor, which can be connected to the device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0101] The communication device 43 is used for data transmission.

[0102] The processor 41 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 42, that is, realizes the above-mentioned working field rectification method.

[0103] The input device 44 can be used to receive input digital or character information, and generate key signal input related to user settings and function control of the device. The output device 45 can include a display device such as a display screen.

[0104] The remote control device provided above can be used to execute the working field rectification method provided by the above-mentioned embodiments, and has corresponding functions and beneficial effects.

[0105] The embodiment of the present application further provides a storage medium containing computer executable instructions, which are used to execute a job plot rectification method when executed by a computer processor, and the job plot rectification method comprises the following steps: obtaining first position information of a rectification reference point in a job plot, and controlling an unmanned device to fly above the rectification reference point according to the first position information; obtaining a first image associated with the rectification reference point and a first height, the first image being an image captured by an aerial camera when the unmanned device collects the first position information, and the first height being a flight height when the unmanned device collects the first image; controlling the unmanned device to fly to the first height, and displaying the first image and a real-time preview picture of the aerial camera on a remote control interface; adjusting the position of the unmanned device to match the real-time preview picture with the first image; and obtaining second position information of the unmanned device under the condition that the real-time preview picture is matched with the first image, and adjusting the position information of the job plot according to the offset between the first position information and the second position information.

[0106] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include an installation medium, e.g., a CD-ROM, floppy disks, or tape device; computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; or a non-volatile memory such as a magnetic medium (e.g., a hard disk or optical storage); registers, or other similar types of memory elements, etc. The storage medium can also include other types of storage medium and combinations thereof. In addition, the storage medium can be located in a first computer system that executes a program containing the instructions described herein, or the storage medium can be located in a second different computer system that connects to the first computer system over a network such as the Internet. The second computer system can provide program instructions to the first computer system for execution. The term "storage medium" can include two or more storage mediums that reside in different locations, e.g., in different computer systems that are connected over a network. The storage medium can store program instructions (e.g., as an installed program) that can be executed by one or more processors.

[0107] Of course, the storage medium containing computer executable instructions provided by the embodiment of the present application is not limited to the job plot rectification method as above, and can also execute the related operations in the job plot rectification method provided by any embodiment of the present application.

[0108] The job plot rectification device, the storage medium and the remote control device provided in the above embodiments can execute the job plot rectification method provided by any embodiment of the present application, and the technical details not described in the above embodiments can be referred to the job plot rectification method provided by any embodiment of the present application.

[0109] The above merely describes the preferred embodiments of the present application and the technical principles applied. The present application is not limited to the specific embodiments herein, and various obvious changes, modifications and replacements made by those skilled in the art without departing from the scope of the present application shall not be excluded. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and more other equivalent embodiments can be included without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A method for correcting deviations in a work area, characterized in that, include: Obtain the first position information of the correction reference point in the work area, and control the unmanned equipment to fly above the correction reference point based on the first position information; Acquire a first image and a first altitude associated with the correction reference point. The first image is an image captured by the unmanned device through a ground-facing camera when it collects the first position information. The first altitude is the flight altitude of the unmanned device when it collects the first image. Control the unmanned device to fly to the first altitude, and display the first image and the real-time preview of the ground camera on the remote control interface; Adjust the position of the unmanned device so that the real-time preview image matches the first image; If the real-time preview image matches the first image, the second location information of the unmanned device is obtained, and the location information of the work site is adjusted according to the offset between the first location information and the second location information.

2. The method for correcting deviations in a work area according to claim 1, characterized in that, Before obtaining the first location information of the correction reference point in the work area, the method further includes: Control the unmanned equipment to fly above the boundary point of the work site, and control the unmanned equipment to mark points to collect the location information of the boundary point; The boundary point is selected as the correction reference point, and the unmanned device is controlled to capture a first image through the ground-facing camera, and the first height of the unmanned device when capturing the first image is determined. The first position information of the correction reference point is determined based on the position information of the boundary point, and the first image, the first height, the first position information and the correction reference point are associated and saved.

3. The method for correcting deviations in a work area according to claim 2, characterized in that, Controlling the unmanned equipment to fly above the boundary point of the work site includes: The remote control interface displays a real-time preview of the ground camera. In response to a first input operation, a first movement command is generated and sent to the unmanned device so that after the unmanned device moves according to the first movement command, the boundary point is located at the center of the real-time preview screen.

4. The method for correcting deviations in a work area according to claim 1, characterized in that, Adjusting the position of the unmanned equipment includes: In response to a second input operation, a second movement command is generated and sent to the unmanned device so that the unmanned device moves according to the second movement command.

5. The method for correcting deviations in a work area according to claim 1, characterized in that, Adjusting the position of the unmanned equipment includes: Determine the feature matching pair between the first image and the real-time preview screen, and determine the translation parameter between the first image and the preview screen based on the feature matching pair; The moving distance and moving direction of the unmanned device are determined according to the translation parameters, and the moving distance and moving direction are sent to the unmanned device so that the unmanned device moves according to the moving distance and moving direction.

6. The method for correcting deviations in a work area according to claim 1, characterized in that, The step of determining that the real-time preview screen matches the first image includes: The real-time preview image is matched with the first image to determine the feature matching degree between the real-time preview image and the first image. If the feature matching degree is greater than or equal to a preset matching threshold, the real-time preview screen is determined to match the first image.

7. The method for correcting deviations in a work area according to any one of claims 1-6, characterized in that, The positioning system of the unmanned equipment uses self-differential technology for positioning.

8. The method for correcting deviations in a work area according to claim 7, characterized in that, Before obtaining the first location information of the correction reference point in the work area, the method further includes: Before carrying out operations on the work site, it is determined whether a preset first correction condition is met. The correction condition is that the positioning system is restarted after the marking is completed or the running time of the positioning system exceeds a preset time threshold after the marking is completed.

9. A field correction device, characterized in that, include: The first flight control module is configured to acquire the first position information of the correction reference point in the work area, and control the unmanned equipment to fly above the correction reference point according to the first position information. The reference data acquisition module is configured to acquire a first image and a first altitude associated with the correction reference point. The first image is an image captured by the ground-facing camera when the unmanned device collects the first position information, and the first altitude is the flight altitude of the unmanned device when it collects the first image. The interface display module is configured to control the unmanned device to fly to the first altitude and display the first image and the real-time preview of the ground camera on the remote control interface; The second flight control module is configured to adjust the position of the unmanned device so that the real-time preview image matches the first image; The position correction module is configured to, when it is determined that the real-time preview screen matches the first image, obtain the second position information of the unmanned device, and adjust the position information of the work site according to the offset between the first position information and the second position information.

10. A remote control device, characterized in that, include: One or more processors; A memory that stores one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the work site correction method as described in any one of claims 1-8.

11. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the site correction method as described in any one of claims 1-8.

Images