UAV inspection method, device and UAV
By adjusting the camera's field of view during the drone inspection, the problem that the shooting screen does not include the equipment to be inspected due to shaking is solved, and the patrol success rate is improved.
Patent Information
- Application Number
- CN202080004231.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-03-27
- Publication Date
- 2025-03-04
- Estimated Expiration
- 2040-03-27
AI Technical Summary
Due to control errors, external interference and other factors, the drone shook during the inspection, resulting in the shooting screen not including the equipment to be inspected, resulting in the failure of the inspection.
When the drone flies to the inspection location point and adjusts to the preset state according to the preset inspection data, obtain the current picture collected by the camera and determine whether the equipment to be inspected is included. If not included, adjust the camera's field of view to include the equipment to be inspected.
By expanding the camera's field of view, the probability of the current picture collected by the camera including the equipment to be inspected is increased, and the probability of patrol failure is reduced.
Smart Images

Figure CN112585554B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of drone technology, and in particular to a drone inspection method, device and drone. Background Art
[0002] With the continuous development of drone technology, drones are being used more and more widely. For example, they can be used for power inspection, bridge inspection, oil and gas pipeline inspection, etc.
[0003] At present, the automated inspection of drones can be realized in the following ways: first, the drone is manually operated to complete the inspection task, and the drone attitude data of the waypoint, the attitude data of the gimbal set on the drone, and the camera parameters of the camera set on the gimbal are recorded during the task completion process; then, the automated inspection task for the automated inspection of the drone is generated based on these data; finally, the drone can realize automated inspection based on the automated inspection task. In the process of the drone performing automated inspection based on the automated inspection task, specifically, after flying to a single waypoint, the drone can adjust the attitude of the drone, the attitude of the gimbal, and the parameters of the camera according to the data of the waypoint, and take pictures after the adjustment is completed.
[0004] However, due to the influence of various factors, such as the control error of the gimbal posture, the control error of the drone posture, external interference causing the drone to shake, etc., the equipment to be inspected is not included in the captured image, resulting in the problem of inspection failure. Summary of the invention
[0005] The embodiments of the present application provide a drone inspection method, device and drone, which are used to solve the problem in the prior art that due to the influence of various factors, the equipment to be inspected is not included in the captured image, thereby causing inspection failure.
[0006] In a first aspect, an embodiment of the present application provides a drone inspection method, wherein a camera is provided on the drone, and the method comprises:
[0007] When the drone arrives at an inspection location and is adjusted to a preset state according to preset inspection data of the inspection location, a current picture captured by the camera is obtained;
[0008] Determine whether the current screen includes the equipment to be inspected;
[0009] If the current image does not include the device to be inspected, the first target parameter is adjusted to expand the field of view of the camera.
[0010] In a second aspect, an embodiment of the present application provides a drone inspection device, wherein the drone is provided with a camera, and the device comprises: a memory and a processor;
[0011] The memory is used to store program code;
[0012] The processor calls the program code, and when the program code is executed, is used to perform the following operations:
[0013] When the drone arrives at an inspection location and is adjusted to a preset state according to preset inspection data of the inspection location, a current picture captured by the camera is obtained;
[0014] Determine whether the current screen includes the equipment to be inspected;
[0015] If the device to be inspected is not included in the current picture, the first target parameter is adjusted to expand the field of view of the camera so that the device to be inspected can be included in the current picture captured by the camera.
[0016] In a third aspect, an embodiment of the present application provides a drone, the drone comprising a fuselage, a power system disposed on the fuselage, a camera, and an inspection device;
[0017] The power system is used to provide power for the UAV;
[0018] The camera is used to take inspection photos during the inspection of the drone;
[0019] The inspection device includes a memory and a processor;
[0020] The memory is used to store program code;
[0021] The processor calls the program code, and when the program code is executed, is used to perform the following operations:
[0022] When the drone arrives at an inspection location and is adjusted to a preset state according to preset inspection data of the inspection location, a current picture captured by the camera is obtained;
[0023] Determine whether the current screen includes the equipment to be inspected;
[0024] If the device to be inspected is not included in the current picture, the first target parameter is adjusted to expand the field of view of the camera so that the device to be inspected can be included in the current picture captured by the camera.
[0025] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes at least one segment of code, and the at least one segment of code can be executed by a computer to control the computer to execute any of the methods described in the first aspect above.
[0026] In a fifth aspect, an embodiment of the present application provides a computer program, which, when executed by a computer, is used to implement the method described in any one of the first aspects above.
[0027] The embodiments of the present application provide a drone inspection method, device and drone, which obtains the current picture captured by the camera when the drone arrives at an inspection position and adjusts to a preset state according to preset inspection data at the inspection position, determines whether the current picture includes the equipment to be inspected, and if not, adjusts the first target parameter to expand the camera's field of view. This achieves that during the drone inspection process, when the drone is adjusted to a preset state according to preset inspection data, if the current picture captured by the camera does not include the equipment to be inspected, the camera's field of view can be expanded by adjusting the first target parameter, thereby increasing the probability that the current picture captured by the camera includes the equipment to be inspected, reducing the probability that the equipment to be inspected is not included in the image or video taken at the inspection position, and helping to reduce the probability of inspection failure. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0029] Figure 1 A schematic diagram of an application scenario of the focusing method provided in an embodiment of the present application;
[0030] Figure 2 A schematic diagram of a flow chart of a drone inspection method provided in one embodiment of the present application;
[0031] Figure 3 A flowchart of a drone inspection method provided by another embodiment of the present application;
[0032] Figure 4A and Figure 4B A schematic diagram of adjusting drone control parameters to expand the camera field of view provided in an embodiment of the present application;
[0033] Figure 5 A flowchart of a drone inspection method provided in another embodiment of the present application;
[0034] Figure 6 A schematic diagram of adjusting the gimbal control parameters to expand the camera field of view provided in an embodiment of the present application;
[0035] Figure 7A flowchart of a drone inspection method provided in another embodiment of the present application;
[0036] Fig. 8A and Figure 8B A schematic diagram of changing the position of a device to be inspected in the current picture provided in an embodiment of the present application;
[0037] Fig. 9 A schematic diagram of the structure of a drone inspection device provided in one embodiment of the present application;
[0038] Fig.10 A schematic diagram of the structure of a drone provided in one embodiment of the present application. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0040] The drone inspection method provided in the embodiment of the present application can be applied to Figure 1 The inspection system 10 shown may include a camera 11 and an inspection device 12, wherein the camera 11 is arranged on a drone. The camera 11 is used to collect images; the inspection device 12 may obtain the current image collected by the camera 11, and process the current image using the drone inspection method provided in the embodiment of the present application.
[0041] Optionally, the drone may include a patrol device 12, or the control terminal may include a patrol device 12, and the control terminal is used to control the drone. The control terminal may be, for example, a smart phone, a tablet computer, a wearable device, a remote control with a screen, etc.
[0042] The drone inspection method provided in the embodiment of the present application obtains the current picture captured by the camera when the drone arrives at an inspection position and adjusts to a preset state according to preset inspection data of the inspection position, determines whether the current picture includes the equipment to be inspected, and if the current picture does not include the equipment to be inspected, adjusts the first target parameter to expand the field of view of the camera. This achieves that during the drone inspection process, when the drone is adjusted to a preset state according to preset inspection data, if the current picture captured by the camera does not include the equipment to be inspected, the first target parameter can be adjusted to expand the field of view of the camera, thereby increasing the probability that the current picture captured by the camera includes the equipment to be inspected.
[0043] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.
[0044] Figure 2 This is a flow chart of a drone inspection method provided by an embodiment of the present application. The execution subject of this embodiment may be the inspection device 12, and specifically may be the processor of the inspection device 12. Figure 2 As shown, the method of this embodiment may include:
[0045] Step 201, when the drone arrives at an inspection location and is adjusted to a preset state according to preset inspection data of the inspection location, a current picture captured by the camera is obtained.
[0046] In this step, the inspection position point refers to the waypoint where the drone is located when the camera set on the drone needs to take an image or video of the equipment to be inspected. Taking the category of the equipment to be inspected as power equipment as an example, the equipment to be inspected can be one or more of wires, poles, solar panels of photovoltaic power stations, insulators of electric towers, anti-drop pins, hanging point bolts, anti-vibration hammers, etc. Of course, in other embodiments, the equipment to be inspected can also be other equipment, and this application does not limit this.
[0047] The preset inspection data refers to the relevant data pre-set for controlling the camera's field of view when the drone arrives at the inspection location. The preset inspection data corresponds to the preset field of view of the camera when the drone arrives at the inspection location. The preset field of view can theoretically include the equipment to be inspected. The preset inspection data can specifically be any type of data that can affect the camera's field of view. For example, the preset inspection data can include one or more of the drone's attitude data, the gimbal's attitude data, and the camera parameters.
[0048] It should be noted that the method for obtaining the preset inspection data can be flexibly implemented according to the needs. Optionally, the preset inspection data can be obtained by manually operating the drone to complete the inspection task. For example, in the process of manually operating the drone to complete the inspection task, for the waypoints that need to be imaged or videoed, the drone attitude data of the waypoint, the attitude data of the gimbal set on the drone, and the camera parameters of the camera set on the gimbal can be recorded. The waypoint is the inspection position point, and the drone attitude data, gimbal attitude data and camera parameters of the waypoint are the preset inspection data of the inspection position point.
[0049] Among them, manually operating a drone to complete an inspection task can be understood as a teaching stage of the drone's automated inspection, generating an automated inspection task based on the data recorded in the teaching stage, and the drone automatically completing the inspection task based on the automated inspection task can be understood as the replay stage corresponding to the teaching stage.
[0050] Theoretically, when the preset state is adjusted according to the preset inspection data of the inspection position, the current picture collected by the camera will include the device to be inspected. However, in actual applications, due to the influence of various factors, there is a situation where the device to be inspected is not included in the current picture collected by the camera when the camera arrives at an inspection position and adjusts to the preset state according to the preset inspection data of the inspection position, resulting in the inability of the automated inspection to obtain the scene content of the device to be inspected at the inspection position, thereby causing the inspection to fail.
[0051] The influencing factors include but are not limited to: 1. There are positioning errors in the drone attitude data, gimbal attitude data, camera parameters, etc. stored in the teaching phase; 2. There are control errors when the drone arrives at the inspection location; 3. After the drone arrives at the inspection location, there is an attitude error when the gimbal attitude is adjusted according to the gimbal attitude data included in the preset inspection data; 4. The drone may shake due to interference from external forces (such as wind) during the shooting process; 5. Different models of drones perform the same inspection task, and errors are caused by differences in drone structures, etc.
[0052] Step 202: determine whether the current image includes equipment to be inspected.
[0053] In this step, it can be determined based on the current screen whether a certain condition is met. If the certain condition is met, it can be indicated that the current screen includes the device to be inspected. If the certain condition is not met, it can be indicated that the current screen does not include the device to be inspected. The specific method for determining whether the current screen includes the device to be inspected can be flexibly implemented according to needs. It can be understood that in a certain scenario, when the current screen only includes part of the device to be inspected, it can be identified that the current screen does not include the device to be inspected.
[0054] Step 203: If the current image does not include the device to be inspected, adjust the first target parameter to expand the field of view of the camera.
[0055] In this step, in case that the device to be inspected is not included in the current picture, the first target parameter needs to be adjusted to expand the field of view of the camera. Specifically, the first target parameter can be any type of parameter that can expand the field of view of the camera. Exemplarily, the first target parameter includes one or more of the following: drone control parameters, gimbal control parameters, or focal length parameters. Drone control parameters refer to parameters used to control drones, gimbal control parameters refer to parameters used to control gimbals, and focal length parameters refer to parameters used to control the focal length of the camera.
[0056] It should be noted that, in practical applications, expanding the camera's field of view may refer to updating the camera's field of view from the original field of view A to a field of view B including the original field of view A and a certain range around the original field of view A. Since the new field of view B includes not only the original field of view A but also a certain range around it, the camera's field of view is expanded, and the expanded field of view is the new field of view B. Moreover, by including a certain range around the original field of view A in the new field of view B, the field of view can be expanded to the surrounding areas of the original field of view A at the same time, which is conducive to improving the efficiency of expanding the camera's field of view in a single time so that the current picture collected by the camera includes the equipment to be inspected.
[0057] Furthermore, expanding the camera's field of view may also refer to updating the camera's field of view from an original field of view C to a field of view D that includes part of the original field of view C and a certain range in a certain direction of the original field of view C. Since the original field of view C and the new field of view D are both field of view that the camera can observe, and the new field of view D includes a certain range in a certain direction of the original field of view A, the camera's field of view is expanded, and the expanded field of view is the union of the original field of view C and the new field of view D.
[0058] Since expanding the camera's field of view can increase the content of the pictures captured by the camera and include content that was not originally captured in the current picture, it can increase the probability that the current picture captured by the camera includes the equipment to be inspected, and reduce the probability that the equipment to be inspected is not included in the image or video taken at the inspection location, which is beneficial to reduce the probability of inspection failure.
[0059] Optionally, a single adjustment can be made to expand the camera's field of view. After the single adjustment, the current image captured by the camera can be directly photographed; or, multiple adjustments can be made to expand the camera's field of view. The specific parameters adjusted during multiple adjustments can be the same or different.
[0060] Optionally, after the device to be inspected is found in the picture, the device to be inspected can be tracked, and after adjusting parameters, the device to be inspected can be focused and photographed.
[0061] In the present embodiment, when the UAV arrives at an inspection position and adjusts to a preset state according to preset inspection data of the inspection position, the current picture captured by the camera is obtained, and it is determined whether the current picture includes the equipment to be inspected. If not, the first target parameter is adjusted to expand the camera's field of view. This achieves that during the UAV inspection process, when the UAV is adjusted to a preset state according to preset inspection data, if the current picture captured by the camera does not include the equipment to be inspected, the camera's field of view can be expanded by adjusting the first target parameter, thereby increasing the probability that the current picture captured by the camera includes the equipment to be inspected, reducing the probability that the equipment to be inspected is not included in the image or video taken at the inspection position, and helping to reduce the probability of inspection failure.
[0062] Figure 3 A flowchart of a drone inspection method provided by another embodiment of the present application. Figure 2 Based on the embodiment shown, the optional implementation method of the first target parameter including the drone control parameter is mainly described. Figure 3 As shown, the method of this embodiment may include:
[0063] Step 301, when the drone arrives at an inspection location and is adjusted to a preset state according to preset inspection data of the inspection location, a current picture captured by the camera is obtained.
[0064] It should be noted that step 301 is similar to step 201 and will not be described in detail here.
[0065] Step 302: determine whether the current image includes equipment to be inspected.
[0066] In this step, it can be determined whether the current picture includes the device to be inspected according to the target feature of the device to be inspected, wherein the target feature of the device to be inspected can be a feature that can represent the characteristics of the device to be inspected.
[0067] Exemplarily, feature matching can be performed on the current screen according to the target feature of the device to be inspected to determine whether there is an area in the current screen whose similarity with the target feature is greater than a certain threshold. If there is no area in the current screen whose similarity with the target feature is greater than a certain threshold, the device to be inspected is not included in the current screen; if there is an area in the current screen whose similarity with the target feature is greater than the threshold, the device to be inspected is included in the current screen. It can be understood that the area in the current screen whose similarity with the target feature is greater than a certain threshold can be understood as the area in the current screen where the device to be inspected is located.
[0068] When the devices to be inspected in the inspection task are of the same type, for example, when all the devices to be inspected in the inspection task are electric poles, the target feature can correspond to the inspection task. When the devices to be inspected in the inspection task include different types of devices, for example, when the devices to be inspected in the inspection task include insulators and anti-drop pins, the target feature can correspond to the inspection position point.
[0069] Exemplarily, the target feature can be obtained through the preset inspection data. Optionally, the preset inspection data can include the target feature. For example, during the teaching phase when the drone is manually operated for inspection, when the drone arrives at the inspection location and is adjusted to clearly obtain the preset state of the device to be inspected and capture a preset image including the device to be inspected, the user can select the target area where the device to be inspected is located in the preset image, further perform feature extraction on the target area to obtain the target feature, and record the target feature as part of the preset inspection data.
[0070] Alternatively, the preset inspection data may include a preset image, the preset image includes the equipment to be inspected, and the target features are extracted from the preset image. For example, during the teaching phase, when the manually operated drone is used for inspection, when the manually operated drone arrives at the inspection location and is adjusted to clearly obtain the preset state of the equipment to be inspected and capture a preset image including the equipment to be inspected, the preset image can be recorded as part of the preset inspection data. Based on this, before executing step 302, feature extraction can also be performed on the preset image to obtain the target features. Considering that the equipment to be inspected is usually used as a foreground object in the image during manual operation, in order to improve the accuracy of the target features, a preset algorithm can be used to first identify the foreground pixels in the preset image, and then feature extraction can be performed on the area occupied by the foreground pixels in the preset image to obtain the target features.
[0071] Step 303: If the current image does not include the device to be inspected, the drone control parameters are adjusted to control the drone to fly a certain distance in a preset direction until a first stop condition is met.
[0072] In this step, the preset direction is consistent with the opposite direction of the camera field of view. The preset direction is consistent with the opposite direction of the camera field of view, which can specifically mean that the angle between the preset direction and the opposite direction of the camera field of view is less than the angle threshold. Among them, the angle threshold is greater than or equal to 0°, which can be flexibly implemented according to needs. The drone control parameter can specifically be any type of parameter that can control the drone to fly a certain distance in a preset direction. For example, a certain distance can be 0.5 meters, 1 meter, etc., which can be flexibly implemented according to needs.
[0073] Since the preset direction is consistent with the opposite direction of the camera's field of view, the distance between the object and the camera can be increased by controlling the drone to fly a certain distance in the preset direction, and since the camera's field of view is certain, the camera's field of view can be updated from the original field of view A to a field of view B including the original field of view A and a certain range around the original field of view A by controlling the drone to fly a certain distance in the preset direction. In one embodiment, the camera is disposed below the drone, and the preset direction includes a vertical upward direction. In the case where the camera is disposed below the drone, the camera is usually used to photograph objects below the drone, that is, the camera's field of view is usually below the drone. By including a vertical upward direction in the preset direction, that is, controlling the drone to fly vertically upward for a certain distance, the field of view can be expanded and the drone can be easily controlled.
[0074] like Figure 4A As shown in the figure, assuming that the camera's field of view is direction O, before the drone is controlled to fly a certain distance in the preset direction, the camera's field of view is as follows: Figure 4A The preset direction is the opposite direction of the camera's field of view. After the drone is controlled to fly a certain distance in the preset direction, the camera's field of view will be as follows: Figure 4A From the range B in the figure, it can be seen that the field of view B is larger than the field of view A, thereby expanding the field of view of the camera.
[0075] like Figure 4B As shown in the figure, assuming that the camera's field of view is direction O', before the drone is controlled to fly a certain distance in the preset direction, the camera's field of view is as follows: Figure 4B The range A' in the preset direction is vertical upward. After the drone is controlled to fly a certain distance in the preset direction, the camera's field of view is as follows: Figure 4B From the range B' in the image, it can be seen that the field of view B' is larger than the field of view A', thereby expanding the field of view of the camera.
[0076] The first stop condition is used to determine the timing of stopping adjusting the control parameters of the drone. The first stop condition can be flexibly implemented according to demand. When the first stop condition is met, the control parameters of the drone are no longer adjusted, that is, the drone is no longer controlled to fly in the preset direction. Exemplarily, the first stop condition may specifically include one or more of the following: the number of times the drone control parameters are adjusted is greater than a number threshold, the total flight distance along the preset direction is greater than a distance threshold, or the current screen includes equipment to be inspected. It should be noted that, when the first stop condition includes multiple conditions, satisfying any one of the multiple conditions can represent that the first stop condition is satisfied, that is, the multiple conditions can be in a logical "or" relationship.
[0077] Among them, the number of times the drone control parameters are adjusted is greater than the number threshold, which is used to limit the maximum number of times the drone control parameters are adjusted, avoiding problems caused by excessive adjustment of the drone control parameters, such as inspection taking too long. It should be noted that the distances of the drone control parameters adjusted multiple times to control the drone to fly in the preset direction can be the same or different. For example, the first adjustment of the drone control parameters can control the drone to fly 0.5 meters in the preset direction, and the second adjustment of the drone control parameters can control the drone to fly 1 meter in the preset direction. The total flight distance along the preset direction is greater than the distance threshold, which is used to limit the maximum distance of the drone flight along the preset direction, avoiding problems caused by excessive flight distance along the preset direction, such as deviation from the preset route too far.
[0078] If the first stop condition is satisfied that the current image includes the device to be inspected, it can be indicated that the inspection of the position to be inspected can capture the device to be inspected, that is, the inspection of the position to be inspected can be successful. If the first stop condition is satisfied that the current image does not include the device to be inspected, it can be indicated that the inspection of the position to be inspected cannot capture the device to be inspected, that is, the inspection of the position to be inspected has failed.
[0079] Exemplarily, when the drone control parameters are adjusted for the first time to expand the camera's field of view, the current picture captured by the camera (which can be recorded as current picture 1) can be obtained, and it can be determined whether the current picture 1 includes the equipment to be inspected. When the current picture 1 includes the equipment to be inspected, the adjustment of the drone control parameters can be terminated. When the current picture 1 does not include the equipment to be inspected and the number of adjustments and / or the flight distance do not meet the stop condition, the drone control parameters can be adjusted for a second time to further expand the camera's field of view; in the case of further expanding the camera's field of view, the current picture captured by the camera after the second expansion of the camera's field of view can be obtained (which can be recorded as current picture 2), and it can be determined whether the current picture 2 includes the equipment to be inspected. When the current picture 2 includes the equipment to be inspected, the adjustment of the drone control parameters can be terminated. When the current picture 2 does not include the equipment to be inspected and the number of adjustments and / or the flight distance do not meet the stop condition, the drone control parameters can be adjusted for a third time to further expand the camera's field of view; ..., and so on until the current picture includes the equipment to be inspected or the number of adjustments or the flight distance meet the stop condition.
[0080] It should be noted that the specific method for determining whether the current screen 1, the current screen 2, etc. include the device to be inspected is similar to the method of the aforementioned steps 202 and 302, and will not be repeated here.
[0081] Optionally, in the event of inspection failure at the location to be inspected, in order to facilitate the user to obtain relevant information of the inspection event, when the first stop condition satisfied at the end of the adjustment is that the current screen does not include the device to be inspected, it may also include: sending inspection failure information to the control terminal, and the inspection failure information is used to indicate the inspection failure of the inspection location point.
[0082] In the present embodiment, when the UAV arrives at an inspection position and is adjusted to a preset state according to preset inspection data of the inspection position, the current picture captured by the camera is obtained, and it is determined whether the current picture includes the equipment to be inspected. If not, the UAV control parameters are adjusted to control the UAV to fly a certain distance in a preset direction, thereby expanding the camera's field of view. This achieves that during the UAV inspection process, when the UAV is adjusted to a preset state according to preset inspection data, if the current picture captured by the camera does not include the equipment to be inspected, the camera's field of view can be expanded by controlling the UAV to fly a certain distance in a preset direction, thereby increasing the probability that the current picture captured by the camera includes the equipment to be inspected, reducing the probability that the equipment to be inspected is not included in the image or video taken at the inspection position, and helping to reduce the probability of inspection failure.
[0083] Figure 5 A flowchart of a drone inspection method provided by another embodiment of the present application is shown in FIG. Figure 2 Based on the embodiment shown, the optional implementation method of the first target parameter including the PTZ control parameter is mainly described. Figure 5 As shown, the method of this embodiment may include:
[0084] Step 501, when the drone arrives at an inspection location and is adjusted to a preset state according to preset inspection data of the inspection location, a current picture captured by the camera is obtained.
[0085] It should be noted that step 501 is similar to step 201 and will not be described in detail here.
[0086] Step 502: determine whether the current image includes equipment to be inspected.
[0087] It should be noted that step 502 is similar to step 202 and step 302 and will not be described in detail here.
[0088] Step 503: If the current image does not include the device to be inspected, adjust the pan / tilt control parameters to control the pan / tilt to rotate around a preset axis by a certain angle until a second stop condition is met.
[0089] In this step, the camera is set on the drone through the gimbal, and the field of view of the camera can be changed by controlling the gimbal. The preset rotation axis can be, for example, one or more of the roll axis, the pitch axis, and the heading axis. The gimbal control parameter can be any type of parameter that can control the gimbal to rotate around the preset rotation axis at a certain angle. The certain angle can be, for example, 5°, 10°, etc., and can be flexibly implemented according to needs. Since the field of view of the camera can be changed by controlling the gimbal to rotate around the preset rotation axis at a certain angle, the field of view of the camera can be updated from the original field of view C to a field of view D including part of the original field of view C and a certain range in a certain direction of the original field of view C by controlling the gimbal to rotate around the preset rotation axis at a certain angle.
[0090] like Figure 6 As shown in the figure, assuming that the camera's field of view is in the direction O, before the gimbal is controlled to rotate around the preset axis for a certain angle, the camera's field of view is as follows: Figure 6 In the range C, the preset rotation axis is the pitch axis. After the gimbal is controlled to rotate a certain angle along the preset rotation axis, the camera's field of view is expanded from only range C to the union of range C and range D, thereby expanding the camera's field of view.
[0091] The second stop condition is used to determine the timing to stop adjusting the pan-tilt control parameters. The second stop condition can be flexibly implemented according to the needs. When the second stop condition is met, the pan-tilt control parameters are no longer adjusted, that is, the pan-tilt is no longer controlled to rotate around the preset axis. Exemplarily, the second stop condition may specifically include one or more of the following: the number of times the pan-tilt control parameters are adjusted is greater than a number threshold, the sum of the rotation angles around the preset axis is greater than an angle threshold, or the current screen includes equipment to be inspected. It should be noted that, when the second stop condition includes multiple conditions, satisfying any one of the multiple conditions can represent that the second stop condition is satisfied, that is, the multiple conditions can be in a logical "or" relationship.
[0092] Among them, the number of times the pan-tilt control parameters are adjusted is less than the number threshold, which is used to limit the maximum number of times the pan-tilt control parameters are adjusted, so as to avoid problems caused by excessive adjustment of the pan-tilt control parameters, such as inspection taking too long. It should be noted that the preset axis around which the pan-tilt control parameters are adjusted multiple times to control the rotation of the pan-tilt can be the same or different, the angles around the preset axis can be the same or different when the pan-tilt control parameters are adjusted multiple times, and the directions of rotation around the preset axis can be the same or different when the pan-tilt parameters are adjusted multiple times. The sum of the rotation angles around the preset axis is greater than the angle threshold, which is used to limit the maximum angle of the pan-tilt rotation along the preset axis, so as to avoid problems caused by excessive rotation angles along the preset axis, such as deviating too far from the correct direction.
[0093] Exemplarily, when the pan-tilt control parameters are adjusted for the first time to expand the camera's field of view, the current picture captured by the camera (which can be recorded as current picture 1) can be obtained, and it can be determined whether the current picture 1 includes the device to be inspected. When the current picture 1 includes the device to be inspected, the adjustment of the pan-tilt control parameters can be terminated. When the current picture 1 does not include the device to be inspected and the number of adjustments and / or the rotation angle do not meet the stop condition, the pan-tilt control parameters can be adjusted for a second time to further expand the camera's field of view; when further expanding the camera's field of view, the current picture captured by the camera (which can be recorded as current picture 2) can be obtained, and it can be determined whether the current picture 2 includes the device to be inspected. When the current picture 2 includes the device to be inspected, the adjustment of the pan-tilt control parameters can be terminated. When the current picture 2 does not include the device to be inspected and the number of adjustments and / or the rotation angle do not meet the stop condition, the pan-tilt control parameters can be adjusted for a third time to further expand the camera's field of view; ..., and so on until the current picture includes the device to be inspected or the number of adjustments or the rotation angle meet the stop condition.
[0094] Similarly, when the second stop condition satisfied for ending the adjustment is not that the current screen includes the device to be inspected, the method may further include: sending inspection failure information to the control terminal, wherein the inspection failure information is used to indicate that the inspection of the inspection location point has failed.
[0095] In the present embodiment, when the UAV arrives at an inspection position and adjusts to a preset state according to preset inspection data of the inspection position, the current picture captured by the camera is obtained, and it is determined whether the current picture includes the equipment to be inspected. If not, the pan-tilt control parameters are adjusted to control the pan-tilt to rotate a certain angle around a preset axis, thereby expanding the camera's field of view. This achieves that during the UAV inspection process, when the UAV is adjusted to a preset state according to preset inspection data, if the current picture captured by the camera does not include the equipment to be inspected, the camera's field of view can be expanded by controlling the pan-tilt to rotate a certain angle around the preset axis, thereby increasing the probability that the current picture captured by the camera includes the equipment to be inspected, reducing the probability that the equipment to be inspected is not included in the image or video taken at the inspection position, and helping to reduce the probability of inspection failure.
[0096] Figure 7 A flowchart of a drone inspection method provided by another embodiment of the present application is shown in FIG. Figure 2 Based on the embodiment shown, the optional implementation method of the first target parameter including the focal length parameter is mainly described. Figure 7 As shown, the method of this embodiment may include:
[0097] Step 701, when the drone arrives at an inspection location and is adjusted to a preset state according to preset inspection data of the inspection location, a current picture captured by the camera is obtained.
[0098] It should be noted that step 701 is similar to step 201 and will not be described in detail here.
[0099] Step 702: determine whether the current image includes equipment to be inspected.
[0100] It should be noted that step 702 is similar to step 202 and step 302 and will not be described in detail here.
[0101] Step 703: If the current image does not include the device to be inspected, adjust the focal length parameter to adjust the focal length of the camera by a certain value in the direction of focal length reduction until the third stop condition is met.
[0102] In this step, the focal length refers to the range of variation of the focal length of the zoom lens. The focal length of the lens is inversely proportional to the angle of view. The longer the focal length, the smaller the angle of view, and the shorter the focal length, the larger the angle of view. Therefore, by reducing the focal length of the camera, the camera's field of view can be expanded. The certain value can be, for example, 5 millimeters (mm), 10 mm, etc., and can be flexibly implemented according to needs. Since the shorter the focal length, the larger the angle of view, the larger the angle of view, the larger the camera's field of view, and the camera's field of view expands around the original field of view, so by controlling the camera's focal length to adjust in the direction of reducing the focal length parameter, the camera's field of view can be updated from the original field of view A to a field of view B including the original field of view A and a certain range around the original field of view A.
[0103] The third stop condition is used to determine the timing of stopping adjusting the focal length parameters. The third stop condition can be flexibly implemented according to the needs. When the third stop condition is met, the focal length parameters are no longer adjusted, that is, the focal length parameters are no longer controlled to adjust in the direction of focal length reduction. Exemplarily, the third stop condition may specifically include one or more of the following: the current focal length is the minimum focal length or the current picture includes the equipment to be inspected. It should be noted that, when the third stop condition includes multiple conditions, satisfying any one of the multiple conditions can represent satisfying the third stop condition, that is, the multiple conditions can be in a logical "or" relationship.
[0104] The minimum focal length may be the theoretical minimum focal length of the camera, or may be a pre-set focal length, and the current focal length is the minimum focal length, which is used to limit the minimum focal length of the camera to capture the image. It should be noted that the values of adjusting the focal length parameters multiple times to adjust the focal length of the camera in the direction of focal length reduction may be the same or different. For example, the first adjustment of the focal length parameters may control the focal length of the camera from 135mm to 200mm to 100mm to 135mm, and the second adjustment of the focal length parameters may control the focal length of the camera from 100mm to 135mm to 35mm-100mm.
[0105] Exemplarily, when the focal length parameters are adjusted for the first time to expand the camera's field of view, the current picture captured by the camera (which can be recorded as current picture 1) can be obtained, and it can be determined whether the current picture 1 includes the device to be inspected. When the current picture 1 includes the device to be inspected, the adjustment of the focal length parameters can be ended. When the current picture 1 does not include the device to be inspected and the current focal length is not the minimum focal length, the focal length parameters can be adjusted for a second time to further expand the camera's field of view; when the camera's field of view is further expanded, the current picture captured by the camera (which can be recorded as current picture 2) can be obtained, and it can be determined whether the current picture 2 includes the device to be inspected. When the current picture 2 includes the device to be inspected, the adjustment of the focal length parameters can be ended. When the current picture 2 does not include the device to be inspected and the current focal length is not the minimum focal length, the pan / tilt control parameters can be adjusted for a third time to further expand the camera's field of view; ..., and so on until the current picture includes the device to be inspected or the current focal length is the minimum focal length.
[0106] Similarly, when the third stop condition satisfied by the end adjustment is not that the current screen includes the device to be inspected, it may also include: sending inspection failure information to the control terminal, the inspection failure information is used to indicate the inspection failure of the inspection position point.
[0107] Optionally, the preset inspection parameters may include a preset focal length. The preset focal length refers to a preset parameter for controlling the focal length of the camera when the drone arrives at the inspection position. For example, the preset focal length may be a focal length parameter used when shooting at the inspection position in the instruction mode. In order to make the size of the device to be inspected in the image or video taken during the automatic inspection of the drone consistent with the preset setting, when the current picture of the camera includes the device to be inspected, it may also include: adjusting the focal length parameter to the preset focal length. It is understandable that the process of adjusting the focal length parameter to the preset focal length needs to ensure that the device to be inspected is in the current picture, so as to avoid the device to be inspected from being out of the current picture due to adjusting the focal length parameter to the preset focal length.
[0108] Optionally, after finding the device to be inspected in the picture, adjust the focal length parameters to make them the same as the focal length parameters in the teaching stage and then focus and shoot.
[0109] In the present embodiment, when the UAV arrives at an inspection position and is adjusted to a preset state according to preset inspection data of the inspection position, the current picture captured by the camera is obtained, and it is determined whether the current picture includes the equipment to be inspected. If not, the focal length parameters are adjusted to adjust the focal length of the camera by a certain value in the direction of focal reduction, thereby expanding the camera's field of view. This achieves that during the UAV inspection process, when the UAV is adjusted to a preset state according to preset inspection data, if the current picture captured by the camera does not include the equipment to be inspected, the camera's field of view can be expanded by controlling the focal length of the camera to be adjusted by a certain value in the direction of focal reduction, thereby increasing the probability that the current picture captured by the camera includes the equipment to be inspected, reducing the probability that the equipment to be inspected is not included in the image or video taken at the inspection position, and helping to reduce the probability of inspection failure.
[0110] It should be noted that Figure 3 , Figure 5 , Figure 7 The methods for adjusting the first target parameter shown can be used alone or in combination.
[0111] by Figure 3 and Figure 7 Taking this as an example, assuming that the first stop condition includes adjusting the control parameters of the drone for a number of times greater than a threshold number and the current picture includes the device to be inspected, and the third stop condition includes the current focal length being the minimum focal length and the current picture including the device to be inspected, the adjustment process may be, for example: when adjusting the focal length parameters for the first time to expand the field of view of the camera, the current picture captured by the camera (which may be recorded as current picture 1) may be obtained, and it may be determined whether the current picture 1 includes the device to be inspected. When the current picture 1 includes the device to be inspected, the adjustment of the focal length parameters may be terminated. When the current picture 1 does not include the device to be inspected and the current focal length is not the minimum focal length, the focal length parameters may be adjusted a second time to further expand the field of view of the camera; after further expanding the field of view, the focal length parameters may be adjusted. In the case of the camera field of view, the current picture captured by the camera (which can be recorded as current picture 2) can be obtained, and it is determined whether the current picture 2 includes the equipment to be inspected; further assuming that the current picture 2 does not include the equipment to be inspected and the current focal length is the minimum focal length, the drone control parameters can be adjusted for the first time to expand the camera field of view, and the gimbal control parameters can be adjusted for the third time to further expand the camera field of view; when the camera field of view is further expanded, the current picture captured by the camera (which can be recorded as current picture 3) can be obtained, and it is determined whether the current picture 3 includes the equipment to be inspected..., and so on until the current picture includes the equipment to be inspected or the number of times the drone control parameters are adjusted is greater than the number threshold.
[0112] It should be noted that the above Figure 3 and Figure 7The combined methods are only examples, and the specific combination methods can be flexibly implemented according to needs.
[0113] On the basis of the above embodiment, when the current picture captured by the camera includes the device to be inspected, the method may further include: adjusting the second target parameter to change the position of the device to be inspected in the current picture so that the device to be inspected can be located at a preset position in the current picture. By adjusting the second target parameter, the position of the device to be inspected in the current picture can be adjusted, so that the device to be inspected in the image or video taken by the automatic inspection of the drone can be in a desired specific position, which is conducive to improving the shooting effect.
[0114] The second target parameter may be any type of parameter that can adjust the position of the device to be inspected in the current image. For example, the second target parameter includes a drone attitude parameter and / or a gimbal control parameter. A drone attitude parameter refers to a parameter used to adjust the attitude of a drone, such as a parameter for controlling the pitch angle of a drone, or a parameter for controlling the roll angle of a drone. A gimbal control parameter refers to a parameter used to control a gimbal, such as a parameter for controlling the rotation of a gimbal around a pitch axis, or a parameter for controlling the rotation of a gimbal around a heading axis.
[0115] The preset position can be flexibly implemented according to the needs. In one embodiment, the preset position can include a central position. By adjusting the second parameter so that the device to be inspected can be located in the central position of the current picture, the device to be inspected can be located in the central position in the image or video taken by the automatic inspection of the drone, which is conducive to highlighting the device to be inspected in the picture. Taking the device to be inspected as a solar panel as an example, before adjusting the second target parameter, the position of the device to be inspected in the current picture can be as follows: Fig. 8A As shown, reference Fig. 8A Before adjusting the second target parameter, the position of the device to be inspected in the current screen can be as follows: Figure 8B As shown, reference Figure 8B After adjusting the second target parameters, the device to be inspected is located in the center of the current screen.
[0116] It should be noted that, in view of the aforementioned adjustment of adjusting the focal length parameter to the preset focal length, the focal length parameter can be further adjusted to the preset focal length after adjusting the second target parameter so that the device to be inspected is located at the center of the current picture. Based on this, in the process of adjusting the focal length parameter to the preset focal length, it is possible to ensure that the device to be inspected is in the current picture without further processing, which is conducive to simplifying the implementation.
[0117] Optionally, the adjusting of the second target parameter may specifically include: adjusting the second target parameter according to the preset position and the current position of the device to be inspected in the current picture when the adjustment of the first target parameter is completed. Based on this, it is possible to adjust the second target parameter once so that the device to be inspected is located at the preset position in the current picture, which is conducive to simplifying the implementation.
[0118] Alternatively, the adjustment of the second target parameter may specifically include: adjusting the second target parameter according to the preset position and the current position of the device to be inspected in the current picture determined by target tracking of the device to be inspected, until the device to be inspected is located at the preset position in the current picture. Among them, the target tracking is used to continuously determine the current position of the device to be inspected in the current picture during the process of adjusting the second target parameter, so as to realize the dynamic adjustment of the adjustment amount of the second target parameter based on the target tracking. Based on this, it is possible to adjust the second target parameter multiple times so that the position to be inspected is located at the preset position in the current picture, which is conducive to improving the adjustment accuracy.
[0119] On the basis of the above method embodiment, when the current picture captured by the camera includes the device to be inspected, it can further include: judging whether the area ratio of the device to be inspected in the current picture satisfies a certain ratio condition; and if the area ratio of the device to be inspected in the current picture does not satisfy the ratio condition, adjusting the focal length parameter of the camera to adjust the focal length parameter of the camera in the direction of increasing the focal length until the area ratio of the device to be inspected in the current picture satisfies the ratio condition.
[0120] Among them, satisfying a certain proportion condition may specifically refer to a proportion being greater than or equal to a proportion threshold. Among them, the proportion threshold can be flexibly implemented according to demand, and the larger the proportion threshold, the larger the area occupied by the equipment to be inspected in the current picture, and the smaller the proportion threshold, the smaller the area occupied by the equipment to be inspected in the current picture. By adjusting the focal length parameters of the camera when the proportion of the area of the equipment to be inspected in the current picture does not meet the proportion condition, the size of the equipment to be inspected in the image or video taken during the automatic inspection of the drone can meet certain requirements, avoiding problems caused by the area occupied by the equipment to be inspected in the taken image or video being too small, such as the inability to clearly obtain detailed information of the equipment to be inspected.
[0121] On the basis of the above embodiment, after adjusting the first target parameter, the method may further include: shooting the current screen captured by the camera. The shooting may specifically be image shooting or video shooting. By shooting the current screen captured by the camera, an image or video including the device to be inspected can be finally obtained, and the inspection of the inspection location point can be completed. The image or video obtained by shooting can be stored in the camera, and / or can be sent to the control terminal in real time so that the control terminal can store it.
[0122] Based on the above embodiment, after the equipment to be inspected is found in the picture, the current picture and the picture taken during the teaching stage can be compared. According to the position, size and angle of the equipment to be inspected in the picture, the parameters of the drone, gimbal or shooting device can be adjusted to make the position, size and angle of the equipment to be inspected in the current picture consistent with the position, size and angle of the picture taken during the teaching stage.
[0123] For adjusting the position, the first average position of the equipment to be inspected in the current picture and the second average position of the equipment to be inspected in the shooting picture during the teaching stage are obtained respectively, and a control instruction is generated according to the pixel difference between the first average position and the second average position. The posture of the gimbal or drone is adjusted according to the control instruction to adjust the position of the equipment to be inspected in the current picture.
[0124] For resizing, respectively obtain the first number of pixels occupied by the equipment to be inspected in the current picture and the second number of pixels occupied by the equipment to be inspected in the shooting picture during the teaching phase, generate a control instruction based on the pixel difference between the first number of pixels and the second number of pixels, and adjust the posture of the gimbal or drone or the focal length of the shooting device according to the control instruction to adjust the size of the equipment to be inspected in the current picture.
[0125] For adjusting the angle, the first angle of the device to be inspected in the current picture with respect to the preset reference axis and the second angle of the device to be inspected in the shooting picture during the teaching phase with respect to the preset reference axis can be obtained according to the preset reference axis of the device to be inspected, and a control instruction can be generated according to the difference between the first angle and the second angle, and the position of the gimbal or drone can be adjusted according to the control instruction to adjust the angle of the device to be inspected in the current picture. It should be noted that the reference axis can also be determined in real time according to the shape of the device to be inspected. For example, when the device to be inspected is an electric wire, the straight line segment detected in the foreground pixel is used as the reference axis; and when the device to be inspected is a symmetrical shape, the central axis of the symmetrical shape is detected as the reference axis.
[0126] In one embodiment, the three can be adjusted in the order of position, size and angle to improve the accuracy of the adjustment, ensure that the picture taken in the automated inspection stage is consistent with the picture taken in the teaching stage, which is beneficial to the reliability of the inspection results and provides a reference for subsequent judgment of whether the equipment to be inspected has a fault.
[0127] Fig. 9 This is a schematic diagram of the structure of a drone inspection device provided in an embodiment of the present application, wherein the drone is provided with a camera, such as Fig. 9 As shown, the device 900 may include: a processor 901 and a memory 902 .
[0128] The memory 902 is used to store program codes;
[0129] The processor 901 calls the program code, and when the program code is executed, is used to perform the following operations:
[0130] When the drone arrives at an inspection location and is adjusted to a preset state according to preset inspection data of the inspection location, a current picture captured by the camera is obtained;
[0131] Determine whether the current screen includes the equipment to be inspected;
[0132] If the device to be inspected is not included in the current picture, the first target parameter is adjusted to expand the field of view of the camera so that the device to be inspected can be included in the current picture captured by the camera.
[0133] The drone inspection device provided in this embodiment can be used to execute the technical solution of the aforementioned method embodiment. Its implementation principle and technical effect are similar to those of the method embodiment and will not be repeated here.
[0134] Fig.10 A schematic diagram of the structure of a drone provided in one embodiment of the present application is shown in FIG. Fig.10 As shown, the drone 100 may include: a fuselage 101, a power system 102 disposed on the fuselage 101, a camera 103 and an inspection device 104;
[0135] The power system 102 is used to provide power for the UAV;
[0136] The camera 103 is used to take inspection photos during the inspection of the drone;
[0137] The inspection device 104 includes a memory and a processor;
[0138] The memory is used to store program code;
[0139] The processor calls the program code, and when the program code is executed, is used to perform the following operations:
[0140] When the drone arrives at an inspection location and is adjusted to a preset state according to preset inspection data of the inspection location, a current picture captured by the camera is obtained;
[0141] Determine whether the current screen includes the equipment to be inspected;
[0142] If the device to be inspected is not included in the current picture, the first target parameter is adjusted to expand the field of view of the camera so that the device to be inspected can be included in the current picture captured by the camera.
[0143] Optionally, the drone 100 may further include a gimbal 105, and the camera 102 may be disposed on the fuselage 101 via the gimbal 105. Of course, in addition to the above-listed devices, the drone may also include other elements or devices, which are not listed here one by one.
[0144] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.
[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A drone inspection method, characterized in that: The drone is provided with a camera, and the method comprises: When the drone arrives at an inspection location and is adjusted to a preset state according to preset inspection data of the inspection location, a current picture captured by the camera is obtained; Determine whether the current screen includes the equipment to be inspected; If the current image does not include the device to be inspected, adjusting the focal length parameter to expand the field of view of the camera; The preset inspection data includes a preset focal length, and the method further includes: When the current picture of the camera includes the device to be inspected, the target parameters are adjusted to change the position of the device to be inspected in the current picture so that the device to be inspected can be located at a preset position in the current picture, and the focal length parameters are adjusted to a preset focal length.
2. The method according to claim 1, characterized in that The method further comprises: If the current picture does not include the device to be inspected, the drone control parameters and / or the gimbal control parameters are adjusted to expand the field of view of the camera.
3. The method according to claim 2, characterized in that The adjusting of the drone control parameters includes: adjusting the drone control parameters to control the drone to fly a distance along a preset direction until a first stop condition is met, and the preset direction is consistent with the opposite direction of the camera field of view.
4. The method according to claim 3, characterized in that The first stop condition includes one or more of the following: The number of times the drone control parameters are adjusted is greater than the number threshold, the total flight distance along the preset direction is greater than the distance threshold, or the current screen includes the equipment to be inspected.
5. The method according to claim 3, characterized in that: The camera is arranged below the drone, and the preset direction includes a vertical upward direction.
6. The method according to claim 2, characterized in that The camera is arranged on the drone via a gimbal; The adjusting of the pan / tilt control parameters includes: adjusting the pan / tilt control parameters to control the pan / tilt to rotate a certain angle around a preset rotation axis until a second stop condition is met.
7. The method according to claim 6, characterized in that The second stop condition includes one or more of the following: The number of times the PTZ control parameters are adjusted is greater than the number threshold, the sum of the rotation angles around the preset axis is greater than the angle threshold, or the current screen includes the device to be inspected.
8. The method according to claim 1, characterized in that The adjusting of the focal length parameters includes: adjusting the focal length parameters to adjust the focal length of the camera by a certain value in a direction of reducing the focal length until a third stop condition is met.
9. The method according to claim 8, characterized in that The third stop condition includes one or more of the following: The current focal length is the minimum focal length or the current image includes the device to be inspected.
10. The method according to any one of claims 4, 7 and 9, characterized in that: When the stop condition satisfied by the adjustment is not that the current screen includes the equipment to be inspected, the method further includes: Sending inspection failure information to the control terminal, wherein the inspection failure information is used to indicate the inspection failure of the inspection location point.
11. The method according to claim 1, characterized in that The determining whether the current screen includes the device to be inspected includes: Perform feature matching on the current image according to the target feature of the device to be inspected, so as to determine whether there is an area in the current image whose similarity with the target feature is greater than a certain threshold; If there is no area in the current picture whose similarity with the target feature is greater than a certain threshold, the current picture does not include the device to be inspected.
12. The method according to claim 11, characterized in that The preset inspection data includes the target feature; or, the preset inspection data includes a preset image, the preset image includes the device to be inspected, and the target feature is extracted from the preset image.
13. The method according to claim 1, characterized in that The target parameter is a second target parameter, and the second target parameter includes a drone attitude parameter and / or a gimbal control parameter.
14. The method according to claim 1, characterized in that The preset position includes a central position.
15. The method according to claim 1, characterized in that The target parameter is a second target parameter, and the adjusting target parameter includes: According to the preset position and the current position of the device to be inspected in the current picture determined by target tracking for the device to be inspected, the second target parameter is adjusted until the device to be inspected is located at the preset position in the current picture.
16. The method according to claim 1, characterized in that In the case where the current picture captured by the camera includes the device to be inspected, the method further includes: Determine whether the area ratio of the device to be inspected in the current picture meets a certain ratio condition; If the area ratio of the device to be inspected in the current picture does not meet the ratio condition, adjust the focal length parameter of the camera to increase the focal length until the area ratio of the device to be inspected in the current picture meets the ratio condition.
17. The method according to claim 1, characterized in that After adjusting the focal length parameters, the method further includes: The current picture captured by the camera is photographed.
18. An unmanned aerial vehicle inspection device, characterized in that: The drone is provided with a camera, and the device comprises: a memory and a processor; The memory is used to store program code; The processor calls the program code, and when the program code is executed, is used to perform the following operations: When the drone arrives at an inspection location and is adjusted to a preset state according to preset inspection data of the inspection location, a current picture captured by the camera is obtained; Determine whether the current screen includes the equipment to be inspected; If the current image does not include the device to be inspected, adjusting the focal length parameter to expand the field of view of the camera; The preset inspection data includes a preset focal length. When the current picture of the camera includes the device to be inspected, the target parameters are adjusted to change the position of the device to be inspected in the current picture so that the device to be inspected can be located at the preset position in the current picture, and the focal length parameters are adjusted to the preset focal length.
19. The device according to claim 18, characterized in that The processor is further configured to: If the current picture does not include the device to be inspected, the drone control parameters and / or the gimbal control parameters are adjusted to expand the field of view of the camera.
20. The device according to claim 19, characterized in that The processor is used to adjust the drone control parameters, specifically including: adjusting the drone control parameters to control the drone to fly a distance along a preset direction until a first stop condition is met, and the preset direction is consistent with the opposite direction of the camera field of view.
21. The device according to claim 20, characterized in that The first stop condition includes: The number of times the drone control parameters are adjusted is greater than the number threshold, the total flight distance along the preset direction is greater than the distance threshold, or the current screen includes the equipment to be inspected.
22. The device according to claim 21, characterized in that The camera is arranged below the drone, and the preset direction includes a vertical upward direction.
23. The device according to claim 20, characterized in that The camera is arranged on the drone via a gimbal; The processor is used to adjust the pan-tilt control parameters, specifically including: adjusting the pan-tilt control parameters to control the pan-tilt to rotate a certain angle around a preset rotation axis until a second stop condition is met.
24. The device according to claim 23, characterized in that The second stop condition includes: The number of times the PTZ control parameters are adjusted is greater than the number threshold, the sum of the rotation angles around the preset axis is greater than the angle threshold, or the current screen includes the device to be inspected.
25. The device according to claim 18, characterized in that The processor is used to adjust the focal length parameters, specifically including: adjusting the focal length parameters to adjust the focal length of the camera by a certain value in the direction of focal length reduction until a third stop condition is met.
26. The device according to claim 25, characterized in that The third stop condition includes: The current focal length is the minimum focal length or the current image includes the device to be inspected.
27. The device according to any one of claims 21, 24 and 26, characterized in that When the stop condition satisfied by the adjustment is not that the current picture includes the device to be inspected, the processor is further used to: Sending inspection failure information to the control terminal, wherein the inspection failure information is used to indicate the inspection failure of the inspection location point.
28. The device according to claim 18, characterized in that The processor is used to determine whether the current picture includes the device to be inspected, specifically including: Perform feature matching on the current image according to the target feature of the device to be inspected, so as to determine whether there is an area in the current image whose similarity with the target feature is greater than a certain threshold; If there is no area in the current picture whose similarity with the target feature is greater than a certain threshold, the current picture does not include the device to be inspected.
29. The device according to claim 28, characterized in that The preset inspection data includes the target feature; or, the preset inspection data includes a preset image, the preset image includes the device to be inspected, and the target feature is extracted from the preset image.
30. The device according to claim 18, characterized in that The target parameter is a second target parameter, and the second target parameter includes a drone attitude parameter and / or a gimbal control parameter.
31. The device according to claim 18, characterized in that The preset position includes a central position.
32. The device according to claim 18, characterized in that The target parameter is a second target parameter, and the processor is used to adjust the target parameter, specifically including: According to the preset position and the current position of the device to be inspected in the current picture determined by target tracking for the device to be inspected, the second target parameter is adjusted until the device to be inspected is located at the preset position in the current picture.
33. The device according to claim 18, characterized in that In the case where the current picture captured by the camera includes the device to be inspected, the processor is further configured to: Determine whether the area ratio of the device to be inspected in the current picture meets a certain ratio condition; If the area ratio of the device to be inspected in the current picture does not meet the ratio condition, adjust the focal length parameter of the camera to increase the focal length until the area ratio of the device to be inspected in the current picture meets the ratio condition.
34. The device according to claim 18, characterized in that The processor is also used to shoot the current picture captured by the camera after adjusting the focal length parameter.
35. A drone, characterized in that: The drone includes a fuselage, a power system arranged on the fuselage, a camera and an inspection device; The power system is used to provide power for the UAV; The camera is used to take inspection photos during the inspection of the drone; The inspection device includes a memory and a processor; The memory is used to store program code; The processor calls the program code, and when the program code is executed, is used to perform the following operations: When the drone arrives at an inspection location and is adjusted to a preset state according to preset inspection data of the inspection location, a current picture captured by the camera is obtained; Determine whether the current screen includes the equipment to be inspected; If the current image does not include the device to be inspected, adjusting the focal length parameter to expand the field of view of the camera; The preset inspection data includes a preset focal length. When the current picture of the camera includes the device to be inspected, the target parameters are adjusted to change the position of the device to be inspected in the current picture so that the device to be inspected can be located at the preset position in the current picture, and the focal length parameters are adjusted to the preset focal length.
36. The drone according to claim 35, characterized in that: The processor is further configured to: If the current picture does not include the device to be inspected, the drone control parameters and / or the gimbal control parameters are adjusted to expand the field of view of the camera.
37. The drone according to claim 36, characterized in that: The processor is used to adjust the drone control parameters, specifically including: adjusting the drone control parameters to control the drone to fly a distance along a preset direction until a first stop condition is met, and the preset direction is consistent with the opposite direction of the camera field of view.
38. The drone according to claim 37, characterized in that: The first stop condition includes: The number of times the drone control parameters are adjusted is greater than the number threshold, the total flight distance along the preset direction is greater than the distance threshold, or the current screen includes the equipment to be inspected.
39. The drone according to claim 38, characterized in that: The camera is arranged below the drone, and the preset direction includes a vertical upward direction.
40. The drone according to claim 36, characterized in that: The camera is arranged on the drone via a gimbal; The processor is used to adjust the pan-tilt control parameters, specifically including: adjusting the pan-tilt control parameters to control the pan-tilt to rotate a certain angle around a preset rotation axis until a second stop condition is met.
41. The drone according to claim 40, characterized in that The second stop condition includes: The number of times the PTZ control parameters are adjusted is greater than the number threshold, the sum of the rotation angles around the preset axis is greater than the angle threshold, or the current screen includes the device to be inspected.
42. The drone according to claim 35, characterized in that: The processor is used to adjust the focal length parameters, specifically including: adjusting the focal length parameters to adjust the focal length of the camera by a certain value in the direction of focal length reduction until a third stop condition is met.
43. The drone according to claim 42, characterized in that: The third stop condition includes: The current focal length is the minimum focal length or the current image includes the device to be inspected.
44. The drone according to any one of claims 38, 41 and 43, characterized in that: When the stop condition satisfied by the adjustment is not that the current picture includes the device to be inspected, the processor is further used to: Sending inspection failure information to the control terminal, wherein the inspection failure information is used to indicate the inspection failure of the inspection location point.
45. The drone according to claim 35, characterized in that The processor is used to determine whether the current picture includes the device to be inspected, specifically including: Perform feature matching on the current image according to the target feature of the device to be inspected, so as to determine whether there is an area in the current image whose similarity with the target feature is greater than a certain threshold; If there is no area in the current picture whose similarity with the target feature is greater than a certain threshold, the current picture does not include the device to be inspected.
46. The drone according to claim 45, characterized in that The preset inspection data includes the target feature; or, the preset inspection data includes a preset image, the preset image includes the device to be inspected, and the target feature is extracted from the preset image.
47. The drone according to claim 35, characterized in that: The target parameter is a second target parameter, and the second target parameter includes a drone attitude parameter and / or a gimbal control parameter.
48. The drone according to claim 35, characterized in that: The preset position includes a central position.
49. The drone according to claim 35, characterized in that: The target parameter is a second target parameter, and the processor is used to adjust the target parameter, specifically including: According to the preset position and the current position of the device to be inspected in the current picture determined by target tracking for the device to be inspected, the second target parameter is adjusted until the device to be inspected is located at the preset position in the current picture.
50. The drone according to claim 35, characterized in that In the case where the current picture captured by the camera includes the device to be inspected, the processor is further configured to: Determine whether the area ratio of the device to be inspected in the current picture meets a certain ratio condition; If the area ratio of the device to be inspected in the current picture does not meet the ratio condition, adjust the focal length parameter of the camera to increase the focal length until the area ratio of the device to be inspected in the current picture meets the ratio condition.
51. The drone according to claim 35, characterized in that: The processor is also used to shoot the current picture captured by the camera after adjusting the focal length parameter.
52. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes at least one section of code, and the at least one section of code can be executed by a computer to control the computer to execute the method according to any one of claims 1 to 17.
53. A computer program, characterized in that When the computer program is executed by a computer, it is used to implement the method according to any one of claims 1 to 17.
Citation Information
Patent Citations
Cruising robot pan-tilt adjustment method based on visual feedback
CN107042511A
Robot inspection method and device, computer readable storage medium and robot
CN110850872A