Airplane defect detection method, system, device, and medium

Abstract

The present disclosure provides an aircraft defect detection method, system, device and medium, relating to the technical field of defect detection. The method comprises: positioning the aircraft to be detected to obtain real-time position information; determining the correction position information of each shooting point on the flight route of the unmanned aerial vehicle according to the preset position information and the real-time position information of each shooting point on the flight route of the unmanned aerial vehicle; controlling the unmanned aerial vehicle to collect images of the aircraft to be detected at the corresponding shooting point according to the correction position information; and determining the defect information of the aircraft to be detected according to the images of the aircraft to be detected at the corresponding shooting point. The present disclosure ensures the accurate matching of the automatic flight route of the unmanned aerial vehicle and the actual parking geographical position of the aircraft by positioning the aircraft to be detected, can start the automatic flight of the unmanned aerial vehicle around the aircraft with one key, automatically shoot the pictures of the upper surface of the aircraft, and thus improves the efficiency of aircraft defect detection.

Description

Technical Field

[0001] This disclosure relates to the field of defect detection technology, and in particular to an aircraft defect detection method, an aircraft defect detection system, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Before leaving the site after completing scheduled maintenance, maintenance personnel need to walk around the aircraft once to conduct a complete external inspection to ensure that the aircraft is in a suitable condition for flight and that no maintenance has been missed.

[0003] In related technologies, due to the altitude of aircraft, the inspection method for the upper surface of an aircraft involves maintenance personnel visually inspecting it using a lift platform vehicle. However, this inspection method suffers from poor effectiveness and a high rate of missed inspections.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] This disclosure provides a method, system, equipment, and medium for aircraft defect detection, which at least to some extent overcomes the problems of poor effectiveness and high missed detection rate in the inspection of aircraft upper surfaces in related technologies.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] According to one aspect of this disclosure, an aircraft defect detection method is provided, comprising: locating the aircraft to be detected to obtain real-time position information; determining corrected position information for each shooting point on the flight path of the UAV based on preset position information of each shooting point and the real-time position information; controlling the UAV to acquire images of the aircraft to be detected at corresponding shooting points based on the corrected position information; and determining defect information of the aircraft to be detected based on the images of the aircraft to be detected at corresponding shooting points.

[0008] In one embodiment of this disclosure, the real-time location information includes at least one of the following in the UAV coordinate system: the position coordinates of the left main landing gear corresponding to the left rear wheel, the position coordinates of the right main landing gear corresponding to the right rear wheel, and the position coordinates of the front wheel of the aircraft under test; wherein, the step of locating the aircraft under test to obtain the real-time location information includes: locating the aircraft under test using a real-time dynamic RTK positioning method to obtain the real-time location information.

[0009] In one embodiment of this disclosure, the step of using RTK positioning to locate the aircraft under test and obtain the real-time location information includes: acquiring the position coordinates of any two wheels of the aircraft under test in the UAV coordinate system, wherein the any two wheels are any two of the left rear wheel, the right rear wheel, and the front wheel; determining the position coordinates of the remaining wheels of the aircraft under test based on preset installation parameters and the position coordinates of the any two wheels, thereby obtaining the real-time location information.

[0010] In one embodiment of this disclosure, the preset location information of the shooting point includes at least one of the following: shooting point identifier, a first distance between the shooting point and the left rear wheel, a second distance between the shooting point and the right rear wheel, and a third distance between the shooting point and the front wheel.

[0011] In one embodiment of this disclosure, determining the corrected position information of each shooting point on the UAV flight path based on the preset position information of each shooting point on the UAV flight path and the real-time position information includes: using the maximum likelihood estimation method, the least squares method, or the triangle centroid algorithm to determine the corrected position information of each shooting point on the UAV flight path, wherein the corrected position information includes the position coordinates of the corresponding shooting point in the UAV coordinate system.

[0012] In one embodiment of this disclosure, each shooting point has multiple shooting areas arranged in an array, and each shooting area corresponds to a set of UAV operating parameters; wherein, controlling the UAV to acquire images of the aircraft to be detected at the corresponding shooting point according to the corrected position information includes: controlling the UAV to the corresponding shooting point; determining the UAV's operating parameters based on the shooting area, taking pictures of the corresponding shooting area to obtain a regional image of the aircraft to be detected in the corresponding shooting area, and storing the regional image on a server, wherein the storage information of the regional image includes the shooting point to which the regional image belongs and the shooting area identifier corresponding to the regional image.

[0013] In one embodiment of this disclosure, determining the defect information of the aircraft to be tested based on the image of the aircraft to be tested at the corresponding shooting point includes: identifying defects in the image of the aircraft to be tested at the corresponding shooting point based on a pre-trained defect detection model.

[0014] According to another aspect of this disclosure, an aircraft defect detection system is also provided, including a drone and a controller, wherein the drone is used to locate the aircraft to be inspected and obtain real-time position information; the controller is used to determine corrected position information for each shooting point on the drone's flight path based on preset position information of each shooting point on the drone's flight path and the real-time position information; control the drone to acquire images of the aircraft to be inspected at the corresponding shooting points based on the corrected position information; and determine the defect information of the aircraft to be inspected based on the images of the aircraft to be inspected at the corresponding shooting points.

[0015] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the above-described aircraft defect detection method by executing the executable instructions.

[0016] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described aircraft defect detection method.

[0017] In this embodiment, the aircraft to be inspected is located to obtain real-time location information; based on the preset location information and real-time location information of each shooting point along the UAV's flight path, corrected location information of each shooting point along the UAV's flight path is determined; the UAV is controlled to acquire images of the aircraft to be inspected at the corresponding shooting points based on the corrected location information; and the defect information of the aircraft to be inspected is determined based on the images of the aircraft to be inspected at the corresponding shooting points. This disclosure, by locating the aircraft to be inspected, ensures a precise match between the UAV's automatic flight path and the aircraft's actual parking location, allowing for one-click initiation of automatic UAV circling flight and automatic image capture of the aircraft's upper surface, thereby improving the efficiency of aircraft defect detection.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0020] Figure 1 A schematic diagram of an exemplary system architecture for an aircraft defect detection method according to an embodiment of this disclosure is shown.

[0021] Figure 2 A flowchart of an aircraft defect detection method according to an embodiment of this disclosure is shown.

[0022] Figure 3 This diagram illustrates a flight path of a drone according to an embodiment of the present disclosure.

[0023] Figure 4 A schematic diagram of the drone scanning points in an embodiment of this disclosure is shown.

[0024] Figure 5 A flowchart of another aircraft defect detection method in an embodiment of this disclosure is shown.

[0025] Figure 6 A flowchart of a method for locating an aircraft to be detected is shown in an embodiment of this disclosure.

[0026] Figure 7 This diagram illustrates a principle diagram of aircraft positioning under test in an embodiment of this disclosure.

[0027] Figure 8 A flowchart of a method for determining corrected location information is shown in an embodiment of this disclosure.

[0028] Figure 9 This diagram illustrates a principle for determining corrected location information according to an embodiment of the present disclosure.

[0029] Figure 10 This document presents a flowchart illustrating a method for capturing regional images of an aircraft to be detected in a corresponding shooting area, as described in an embodiment of this disclosure.

[0030] Figure 11 This diagram illustrates a plurality of arrayed shooting areas at the tail of the aircraft in an embodiment of the present disclosure.

[0031] Figure 12 Show Figure 11 A schematic diagram of the area images captured in each shooting area.

[0032] Figure 13 A schematic diagram of an aircraft defect detection system according to an embodiment of this disclosure is shown.

[0033] Figure 14 A schematic diagram of the interface of an aircraft defect detection system according to an embodiment of this disclosure is shown.

[0034] Figure 15 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0035] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0036] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0037] Figure 1 A schematic diagram of an exemplary system architecture that can be applied to an aircraft defect detection method according to embodiments of this disclosure is shown.

[0038] like Figure 1 As shown, the system architecture 100 may include a drone 110, a network 120, and a server 130.

[0039] Network 120 is a medium used to provide a communication link between drone 110 and server 130, and may be a wireless network.

[0040] Optionally, the aforementioned wireless network uses standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to any combination of Local Area Network (LAN), Metropolitan Area Network (MAN), Wide Area Network (WAN), mobile, wireless network, private network, or Virtual Private Network. In some embodiments, technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0041] The drone 110 can be any electronic device with RTK positioning and image acquisition capabilities.

[0042] For example, the drone 110 is equipped with an industrial-grade drone latitude and longitude RTK positioning system to achieve the positioning of the aircraft to be tested.

[0043] For example, the drone 110 is equipped with a gimbal camera or other image acquisition device capable of capturing images of the surface of the aircraft to be inspected.

[0044] Server 130 can be a server that provides various services, such as a back-end management server that supports the device used to operate the drone 110. The back-end management server can analyze and process data such as received requests (e.g., images of the surface of the aircraft to be inspected collected by the drone 110) to obtain processing results (e.g., defect information on the surface of the aircraft to be inspected).

[0045] Optionally, server 130 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0046] Those skilled in the art will know that Figure 1 The number of drones, networks, and servers shown is merely illustrative; any number of drones, networks, and servers can be used depending on actual needs. This disclosure does not limit the scope of the embodiments.

[0047] In related technologies, maintenance personnel drive a lift platform vehicle to conduct visual inspections of the aircraft. However, because some areas of the aircraft are far from the inspectors, such as the top of the vertical tail, the effectiveness of visual inspections is poor, and some areas may be missed.

[0048] On the other hand, due to the large area of ​​the aircraft's upper surface, both using a driving lift platform vehicle for inspection and manually operating drones for inspection require a significant amount of manpower and time.

[0049] To address at least some of the aforementioned problems, this disclosure provides an aircraft defect detection method that enables a drone to automatically fly around the aircraft and automatically capture images of the aircraft's upper surface, thereby accurately identifying aircraft defects.

[0050] The aircraft defect detection method disclosed herein involves locating the aircraft to be inspected to obtain its real-time location information; determining corrected location information for each shooting point along the UAV's flight path based on preset and real-time location information; controlling the UAV to acquire images of the aircraft to be inspected at the corresponding shooting points based on the corrected location information; and determining the defect information of the aircraft to be inspected based on the images at the corresponding shooting points. This method, by locating the aircraft to be inspected, ensures a precise match between the UAV's automatic flight path and the aircraft's actual parking location, enabling one-click initiation of automatic UAV circling flight and automatic image capture of the aircraft's upper surface, thereby improving the efficiency of aircraft defect detection.

[0051] The following detailed description of this exemplary implementation method is provided in conjunction with the accompanying drawings and embodiments.

[0052] First, this disclosure provides an aircraft defect detection method, which can be executed by any electronic device with computing capabilities. In some embodiments, the method can be executed by a drone; in other embodiments, the method can be executed by a server; in addition, it can be implemented through interaction between a drone and a server.

[0053] Figure 2 This diagram illustrates a flowchart of an aircraft defect detection method according to an embodiment of the present disclosure, as follows: Figure 2 As shown, the aircraft defect detection method provided in this embodiment includes the following steps:

[0054] S202. Locate the aircraft to be tested and obtain its real-time location information.

[0055] The aircraft to be inspected is the subject of a drone's camera, meaning it is a target for which it is necessary to determine whether there are any defects.

[0056] It should be noted that the real-time position information mentioned above includes, but is not limited to, the position coordinates of the left main landing gear corresponding to the left rear wheel, the position coordinates of the right main landing gear corresponding to the right rear wheel, the position coordinates of the front wheel, the center position coordinates of the aircraft under test, and the yaw angle, etc., in the UAV coordinate system. Among them, the center position coordinates of the aircraft under test can be the center coordinates of the triangle formed by the left rear wheel, the right rear wheel, and the front wheel.

[0057] The aforementioned yaw angle refers to the difference between the parking position of the aircraft under test and its preset position in the memory.

[0058] S204. Based on the preset position information and real-time position information of each shooting point on the drone's flight path, determine the corrected position information of each shooting point on the drone's flight path.

[0059] In one embodiment, when the drone flies around the aircraft to be tested, because different models of the aircraft to be tested have different sizes, the drone's flight path needs to be pre-configured for the aircraft to be tested, and the corresponding drone flight path is different for different models of the aircraft to be tested. For example... Figure 3 As shown, a drone flight path is provided. The drone's takeoff point is set near the nose of the aircraft to be tested, and both the left and right sides can be used as takeoff points. The horizontal distance between the drone's flight path and the aircraft to be tested is maintained at more than 5 meters, thereby ensuring the safety of both the drone and the aircraft to be tested.

[0060] like Figure 3 As shown, a point in the nose of the aircraft can be used as the takeoff point, and the upper surface of the aircraft to be tested can be photographed in a counterclockwise direction or in a clockwise direction.

[0061] In one embodiment, such as Figure 4 As shown, multiple shooting points can be set along the drone's flight path, for example... Figure 4 The system has 1 to 36 shooting points, which correspond to different parts of the aircraft to be inspected. This allows the drone's shooting area to cover the upper surface of the aircraft. The shooting points are selected so that the captured images are clear enough for users to identify defects with the naked eye. At the same time, the distance between the drone and the aircraft to be monitored is far enough to ensure safety.

[0062] In one implementation, the preset position information of each shooting point on the aforementioned drone flight path may include at least one of the following: shooting point identifier, first distance between the shooting point and the left rear wheel, second distance between the shooting point and the right rear wheel, third distance between the shooting point and the front wheel, position coordinates of the shooting point, and center position coordinates of the aircraft to be detected.

[0063] The shooting location markers can be represented by text, numbers, letters, symbols, etc., and are used to distinguish different shooting locations on the drone's flight path.

[0064] It should be noted that the preset position information of each shooting point along the drone's flight path can be stored in the server in advance. By retrieving the preset position information and combining it with the real-time position information, the corrected position information of each shooting point can be determined. The corrected position information is used to determine the position coordinates of each shooting point of the drone in the drone coordinate system at the current parking position of the aircraft under test.

[0065] S206. Control the UAV to collect images of the aircraft to be detected at the corresponding shooting points based on the corrected position information.

[0066] In one embodiment, the user controls the drone through the software system interface to make the drone perform an automatic flight mission. The drone moves to the corresponding shooting point based on the corrected position information obtained in S204 and captures images of the aircraft to be detected.

[0067] Images of the aircraft to be inspected, captured by the drone, can be stored on a server, and these images can be identified by the shooting location.

[0068] S208. Determine the defect information of the aircraft to be inspected based on the images of the aircraft at the corresponding shooting points.

[0069] In one embodiment, defects in images from various shooting locations can be identified by the human eye.

[0070] In some embodiments, defects in images of the aircraft under inspection at corresponding shooting points can be identified based on a pre-trained defect detection model. That is, target defect detection is used to identify whether defects exist in the image. The aforementioned defect detection model can be trained using defect images collected during the maintenance or repair of the aircraft under inspection.

[0071] Defect detection models can include, but are not limited to, convolutional neural network models and backbone network models. Mean squared error or cross-entropy error can be used as the loss function to measure the difference between the defect prediction result and the defect label. The training process of the defect detection model will not be elaborated further.

[0072] In another embodiment, the presence or absence of defects in the images at each shooting point can be determined by comparing the images captured by the drone at each shooting point with pre-stored, defect-free images of the aircraft to be tested in the memory.

[0073] The defect information of the aircraft to be inspected can be classified into different defect levels according to the severity of the defect, and different processing methods can be adopted according to different defect levels.

[0074] For example, when the defect information of the aircraft to be inspected is at level one, level one defect level can mean that the aircraft has safety hazards and is not suitable for flight. In this case, a grounding control command is generated.

[0075] When the defect information of the aircraft to be inspected is at level two, level two defect level can be due to missed inspections or substandard maintenance, requiring maintenance personnel to address the aforementioned defects in a targeted manner.

[0076] When the defect information of the aircraft to be inspected is at level three, the level three defect level refers to a flaw that does not affect the aircraft to be inspected, the aircraft is suitable for flight, and no treatment is required.

[0077] It should be noted that the defect information of the aircraft to be tested can be set according to the actual situation, and this disclosure does not impose specific limitations.

[0078] In this embodiment, the aircraft to be inspected is located to obtain real-time location information; based on the preset location information and real-time location information of each shooting point along the UAV's flight path, corrected location information of each shooting point along the UAV's flight path is determined; the UAV is controlled to acquire images of the aircraft to be inspected at the corresponding shooting points based on the corrected location information; and the defect information of the aircraft to be inspected is determined based on the images of the aircraft to be inspected at the corresponding shooting points. This disclosure, by locating the aircraft to be inspected, ensures a precise match between the UAV's automatic flight path and the aircraft's actual parking location, allowing for one-click initiation of automatic UAV circling flight and automatic image capture of the aircraft's upper surface, thereby improving the efficiency of aircraft defect detection.

[0079] Figure 5 A flowchart of another UAV defect detection method according to an embodiment of this disclosure is shown. Figure 2 Based on the implementation example, S202 is further refined into S2022 to determine the positioning method of the aircraft to be detected. For example... Figure 5 As shown, the UAV defect detection method provided in this embodiment includes S2022, S204 to S208, and the method includes:

[0080] S2022. The RTK positioning method is used to locate the aircraft under test and obtain real-time location information.

[0081] It should be noted that the specific implementation of S204 to S208 in this embodiment is the same as that of S204 to S208 in the previous embodiment, and will not be repeated here.

[0082] In one embodiment, the UAV is equipped with an RTK (Real-time kinematic) positioning system, also known as RTK carrier phase differential technology. The RTK positioning system includes a base station and a rover station, both of which carry satellite receivers to observe and receive satellite data. The base station is a base station that provides a reference benchmark and can be installed on the UAV; the rover station is a continuously moving station. The rover station is the target object whose three-dimensional coordinates are to be measured, i.e., the left and right main landing gears in this disclosure.

[0083] During measurement, the base station serves as the measurement reference, and its coordinates are known. The base station first observes and receives satellite data. The base station then transmits the observed data to the rover station in real time via a nearby radio station (also known as a data link). Simultaneously, the rover station receives the base station's observation data and also observes and receives satellite data. Based on the base station's data and its own observation data, the rover station performs real-time differential calculations according to the principle of relative positioning to calculate its position coordinates and their accuracy.

[0084] In this embodiment, the RTK positioning system on the UAV is used to correct the UAV's automatic flight path, thereby achieving a precise match between the UAV's automatic flight path and the geographical location of the aircraft to be inspected, thus improving the accuracy of the aircraft defect detection method.

[0085] Figure 6 A flowchart illustrating a method for locating an aircraft to be detected, according to an embodiment of this disclosure, is shown. Figure 6 As shown, in one embodiment, S2022 above uses RTK positioning to locate the aircraft to be tested, determining the position coordinates of the front wheel of the aircraft to be tested in the UAV coordinate system, including:

[0086] S602. In the UAV coordinate system, collect the position coordinates of any two wheels of the aircraft to be tested, wherein any two wheels are any two of the left rear wheel, right rear wheel, and front wheel;

[0087] S604. Based on the preset installation parameters and the position coordinates of any two wheels, determine the position coordinates of the remaining wheels of the aircraft to be tested, and obtain real-time position information.

[0088] It should be noted that the order in which the coordinate positions of the left main landing gear and the right main landing gear are collected is unrelated. That is, S602 can be executed before or after S604.

[0089] In one embodiment, RTK positioning can be used to collect the position coordinates of any two of the left rear wheel, right rear wheel, and front wheel of the aircraft under test in the UAV coordinate system. Based on preset installation parameters, the position coordinates of the third wheel can be calculated to obtain real-time position information, providing a basis for automatic orbital point coordinate calculation.

[0090] For example, the position coordinates of the left and right rear wheels of the aircraft under test are collected in the UAV coordinate system using RTK positioning. Based on the preset installation parameters and the position coordinates of the left and right rear wheels, the position coordinates of the front wheel are calculated.

[0091] For example, the position coordinates of the left rear wheel and the front wheel of the aircraft under test are collected in the UAV coordinate system using RTK positioning. Based on the preset installation parameters and the position coordinates of the left rear wheel and the front wheel, the position coordinates of the right rear wheel are calculated.

[0092] It should be noted that the two wheels mentioned above are preferably the left rear wheel and the right rear wheel, which is convenient for drone operation.

[0093] In one embodiment, the center position coordinates of the aircraft to be tested can be determined based on the position coordinates of the three wheels, thereby obtaining the real-time position information of the aircraft to be tested, providing a basis for the automatic orbital point coordinate calculation. The coordinate transformation matrix can be determined according to the centroid algorithm, and then the position coordinates of the shooting point can be obtained according to the coordinate transformation matrix and the preset position coordinates of the shooting point.

[0094] like Figure 7 As shown, the preset installation parameters in S606 include, but are not limited to, the distance d between the left main landing gear and the right main landing gear, the distance l from the front wheel to the line connecting the left rear wheel and the right rear wheel, and the angle α between the line connecting the front wheel and the left rear wheel and the line connecting the front wheel and the right rear wheel.

[0095] In one embodiment, the yaw angle of the aircraft to be detected can be determined based on trigonometric relationships. The yaw angle θ is determined by the following formula:

[0096] θ=arctan((lng2-lng1) / (lat2-lat1))+π / 2 (Formula 1)

[0097] Where θ∈[0,π], the sign of θ is determined by the UAV's orientation angle, arctan() is the arctangent function, (lat1, lng1) are the coordinates of the left rear wheel in the UAV coordinate system, and (lat2, lng2) are the coordinates of the right rear wheel in the UAV coordinate system.

[0098] Since d and l are known, the coordinates of the front wheel (lat3, lng3) can be calculated.

[0099] The corrected position information can be obtained by performing corresponding translation and rotation operations based on the real-time position information and yaw angle of the aircraft under test.

[0100] In one embodiment, S204 above determines the corrected position information of each shooting point on the drone flight path based on the preset position information and real-time position information of each shooting point on the drone flight path, including: S802, using the maximum likelihood estimation method, the least squares method or the triangle centroid algorithm to determine the corrected position information of each shooting point on the drone flight path, the corrected position information including the position coordinates of the corresponding shooting point in the drone coordinate system.

[0101] Figure 9 This diagram illustrates a principle for determining corrected position information according to an embodiment of the present disclosure. After the real-time position information of the aircraft to be detected is determined, i.e., the position coordinates of the left main landing gear, the right main landing gear, and the nose wheel are determined, as follows... Figure 9 As shown, point C is the position of the front wheel, with coordinates (x1, y1); point B is the position of the left rear wheel, with coordinates (x2, y2); point A is the position of the right rear wheel, with coordinates (x3, y3); and point D is a scanning point on the UAV's flight path, where the coordinates are unknown.

[0102] Based on the preset position information of the shooting point D, the position coordinates of point D are determined. The preset position information includes the first distance R2 between shooting point D and left rear wheel B, the second distance R1 between shooting point D and right rear wheel A, and the third distance R3 between shooting point D and front wheel C. With left rear wheel B, right rear wheel A, and front wheel C as centers and R2, R1, and R3 as radii, three circles can be obtained, and the three circles intersect. The position coordinates of shooting point D are the coordinates of the intersection point of the three circles.

[0103] Due to measurement errors, the aforementioned circles may not intersect at a single point, but rather at a region. In this case, other algorithms can be used to solve the problem, such as maximum likelihood estimation, least squares estimation, or the triangle centroid algorithm.

[0104] The following describes how to use the least squares method to find an approximate solution and determine the coordinates of point D, the shooting location. The solution steps are as follows:

[0105] Establish a system of distance equations between beacon nodes (points A, B, and C) and the unknown node (point D):

[0106]

[0107] Where (x, y) are the position coordinates of the unknown node D, (x i y i Let be the position coordinates of the i-th beacon node, where i = 1, 2, ..., n, corresponding to the position coordinates of each beacon node (point A, point B, and point C), and n = 3 in this disclosure.

[0108] The above system of distance equations is a nonlinear system of equations. Subtracting the nth equation from the first n-1 equations yields the linearized equations:

[0109] Formula 2: AX = b

[0110] in,

[0111]

[0112] The coordinates of a point in the above linearized equation are obtained using the least squares method, as follows:

[0113] X = (A T A) -1 A T Formula 3

[0114] At this point, the location coordinates of the unknown node can be obtained, i.e., the corrected location information.

[0115] In other implementations, a triangle centroid algorithm can be used to determine the coordinates of the triangle center position of the real-time position information. Based on the pre-stored preset center position coordinates of the aircraft to be detected, a coordinate transformation matrix is ​​determined. Based on the coordinate transformation matrix and the position coordinates of the shooting point, the corrected position coordinates of the shooting point are determined.

[0116] In this embodiment, the corrected position information of each shooting point on the flight path of the UAV is determined by using the maximum likelihood estimation method, the least squares method, or the triangle centroid algorithm, so as to accurately determine the shooting point coordinates of the aircraft under inspection at the parking position, estimate the position coordinates of each shooting point, and improve the accuracy of aircraft defect detection.

[0117] It should be noted that each shooting location is configured with multiple shooting points, and each shooting location has multiple shooting areas distributed in an array. Each shooting area corresponds to a set of drone operating parameters.

[0118] To complete the shooting function, the drone is equipped with a gimbal system. When the drone moves to a shooting point, the gimbal system's pitch and horizontal rotation angles are controlled to ensure that the gimbal can capture area images of the aircraft to be inspected in each shooting area.

[0119] Figure 10 This diagram illustrates a method for capturing area images of the aircraft to be detected in a corresponding shooting area, as shown in an embodiment of this disclosure. Figure 10 As shown, in one embodiment, the controlled drone in S206 above acquires images of the aircraft to be detected at the corresponding shooting point based on the corrected position information, including:

[0120] Control the drone to the corresponding shooting location;

[0121] The operating parameters of the UAV are determined based on the shooting area. The corresponding shooting area is photographed to obtain the regional image of the UAV under test in the corresponding shooting area. The regional image is stored on the server. The storage information of the regional image includes the shooting point to which the regional image belongs and the shooting area identifier corresponding to the regional image.

[0122] like Figure 11 As shown, each shooting point is matrix-segmented, dividing the upper surface of the aircraft to be inspected corresponding to that shooting point into multiple small shooting areas. Figure 11 The system is divided into 36 shooting areas, arranged in 4 rows and 9 columns, with each shooting area corresponding to a region of the horizontal tail of the aircraft to be inspected.

[0123] During the drone's orbit, it hovers at each shooting point. The gimbal camera automatically captures images of the corresponding shooting area based on the drone's operating parameters determined for that area. The array shooting effect is as follows: Figure 12 As shown.

[0124] Each image captured by the gimbal camera is recorded in a JSON file, which contains information such as the shooting point and the position of the shooting area in the array, providing a basis for subsequent defect location.

[0125] The area identifier corresponding to the aforementioned area image can be represented by the row and column numbers of the area image in the array, or by the shooting sequence number of the area image at the shooting point. This disclosure does not impose any specific limitations.

[0126] In this embodiment of the disclosure, by dividing the surface of the aircraft to be inspected at each shooting point into multiple shooting areas, the location of defects can be quickly and conveniently identified, and the shooting areas can fully cover the upper surface of the aircraft to be inspected, thereby improving the comprehensiveness of aircraft defect detection and avoiding missed detections.

[0127] In one embodiment, determining the defect information of the aircraft to be inspected based on the image of the aircraft to be inspected at the corresponding shooting point in S208 includes: determining a preset area image based on the shooting point and the shooting area identifier corresponding to the area image; comparing the preset area image and the area image to determine the defect information of the aircraft to be inspected in the area image of the shooting area.

[0128] It should be noted that the above-mentioned preset area image is an image of the upper surface of the aircraft to be inspected that is defect-free when it is in the corresponding shooting area. It can be stored in the server in advance. The storage information of the preset area image also includes the shooting point and the shooting area identifier.

[0129] In this embodiment of the disclosure, by identifying the shooting points and shooting area markers, a preset area image stored in the server is determined. Based on the comparison results between the preset area image and the captured area image, defects in the aforementioned area image can be determined, thereby realizing automatic detection of aircraft defects.

[0130] Based on the same inventive concept, this disclosure also provides an aircraft defect detection system, as described in the following embodiments. Since the principle by which this system embodiment solves the problem is similar to that of the method embodiment described above, the implementation of this system embodiment can refer to the implementation of the method embodiment described above, and repeated details will not be repeated.

[0131] Figure 13 This diagram illustrates an aircraft defect detection system according to an embodiment of the present disclosure, such as... Figure 13 As shown, in one embodiment, an aircraft defect detection system includes: a drone 110 and a controller 1310.

[0132] Among them, UAV 110 is used to locate the aircraft to be tested and obtain real-time location information;

[0133] The controller 1310 is used to determine the corrected position information of each shooting point on the flight path of the UAV based on the preset position information and real-time position information of each shooting point; control the UAV to acquire images of the aircraft to be inspected at the corresponding shooting points based on the corrected position information; and determine the defect information of the aircraft to be inspected based on the images of the aircraft to be inspected at the corresponding shooting points.

[0134] It should be noted that the real-time location information includes at least one of the following in the UAV coordinate system: the position coordinates of the left main landing gear corresponding to the left rear wheel, the position coordinates of the right main landing gear corresponding to the right rear wheel, and the position coordinates of the front wheel.

[0135] In one embodiment, the drone 110 is used to locate the aircraft to be tested using RTK positioning to obtain real-time location information.

[0136] In one embodiment, the UAV 110 is used to collect the position coordinates of any two wheels of the aircraft to be tested in the UAV coordinate system, wherein any two wheels are any two of the left rear wheel, right rear wheel and front wheel; and to determine the position coordinates of the remaining wheels of the aircraft to be tested based on preset installation parameters and the position coordinates of any two wheels to obtain real-time position information.

[0137] It should be noted that the preset location information of the shooting point includes at least one of the following: shooting point marker, first distance between the shooting point and the left rear wheel, second distance between the shooting point and the right rear wheel, and third distance between the shooting point and the front wheel.

[0138] In one embodiment, the controller 1310 is configured to determine the corrected position information of each shooting point on the flight path of the UAV using at least one of the maximum likelihood estimation method, the least squares method, or the triangle centroid algorithm, wherein the corrected position information includes the position coordinates of the corresponding shooting point in the UAV coordinate system.

[0139] It should be noted that each shooting point has multiple shooting areas distributed in an array, and each shooting area corresponds to a set of UAV operating parameters; the controller 1310 is used to control the UAV to the corresponding shooting point; the operating parameters of the UAV are determined based on the shooting area; the UAV 110 is used to take pictures of the corresponding shooting area to obtain the area image of the aircraft to be detected in the corresponding shooting area, and store the area image on the server. The storage information of the area image includes the shooting point to which the area image belongs and the shooting area identifier corresponding to the area image.

[0140] In one embodiment, the controller 1310 is used to identify defects in images of the aircraft to be detected at corresponding shooting points based on a pre-trained defect detection model.

[0141] In the embodiments of this disclosure, the aircraft to be inspected is located to obtain real-time location information; based on the preset location information and real-time location information of each shooting point along the UAV flight path, the corrected location information of each shooting point along the UAV flight path is determined; the UAV is controlled to acquire images of the aircraft to be inspected at the corresponding shooting points based on the corrected location information; and the defect information of the aircraft to be inspected is determined based on the images of the aircraft to be inspected at the corresponding shooting points. This disclosure, by locating the aircraft to be inspected, ensures a precise match between the UAV's automatic flight path and the actual geographical location of the aircraft, and allows for one-click initiation of automatic UAV flight around the aircraft to automatically capture images of the aircraft's upper surface, thereby improving the efficiency of aircraft defect detection.

[0142] Figure 14 A schematic diagram of the interface of an aircraft defect detection system according to an embodiment of this disclosure is shown. Figure 14 As shown, in terms of the control system, a software system for automatic drone circling flight and automatic photography was designed based on the drone. The interface of the software system is as follows. Figure 14 As shown, the software system interface enables functions such as photo taking, gimbal adjustment, geofencing, pause, screenshot, automatic next, start from left, use modified data, stop the task by pushing the joystick, gimbal fine-tuning, and RTK integration. It should be noted that authorization and login to a DJI account are required upon first use of the software system.

[0143] During formal operations, the drone operator performs RTK positioning of the left and right main landing gears of the aircraft to be inspected. After the positioning data is collected, the drone is placed near the takeoff point, and it can be started with one click to automatically fly around the aircraft and automatically take pictures of the upper surface of the aircraft to be inspected.

[0144] The specific operating procedure is as follows:

[0145] Click the application icon, authorize and log in to your DJI account, select the aircraft type and set attribute parameters. The attribute parameters to be set include, but are not limited to, the aircraft model to be detected, confirming the drone's deregulation information, the drone's electronic fence information, and the drone's obstacle avoidance function switch.

[0146] Click on the FPV screen, move the drone to an open area, and turn on the RTK function. After 1 minute, when the signal is stable, move it near the aircraft to be tested and collect the position coordinates of the left and right main landing gears of the aircraft to be tested in the drone coordinate system. After collecting the position coordinates of the left and right main landing gears, click to generate flight path coordinates.

[0147] Move the drone to the vicinity of the takeoff point, wait for the RTK signal to stabilize, click the start button to start the flight route mission, check the return configuration and battery level, and click OK.

[0148] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0149] The following reference Figure 15 To describe an electronic device 1500 according to such an embodiment of the present disclosure. Figure 15 The electronic device 1500 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0150] like Figure 15 As shown, the electronic device 1500 is manifested in the form of a general-purpose computing device. The components of the electronic device 1500 may include, but are not limited to: at least one processing unit 1510, at least one storage unit 1520, and a bus 1530 connecting different system components (including storage unit 1520 and processing unit 1510).

[0151] The storage unit stores program code, which can be executed by the processing unit 1510, causing the processing unit 1510 to perform the steps described in the "Exemplary Methods" section above according to various exemplary embodiments of this disclosure.

[0152] For example, processing unit 1510 can perform the following: Figure 2 The method embodiment includes the following steps: locating the aircraft to be inspected to obtain real-time location information; determining corrected location information for each shooting point on the UAV's flight path based on preset location information and the real-time location information; controlling the UAV to acquire images of the aircraft to be inspected at corresponding shooting points based on the corrected location information; and determining defect information of the aircraft to be inspected based on the images of the aircraft to be inspected at corresponding shooting points.

[0153] Storage unit 1520 may include readable media in the form of volatile storage units, such as random access memory (RAM) 15201 and / or cache memory 15202, and may further include read-only memory (ROM) 15203.

[0154] Storage unit 1520 may also include a program / utility 15204 having a set (at least one) program module 15205, such program module 15205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0155] Bus 1530 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0156] Electronic device 1500 can also communicate with one or more external devices 1540 (e.g., keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 1500, and / or any device that enables electronic device 1500 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1550. Furthermore, electronic device 1500 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1560. As shown, network adapter 1560 communicates with other modules of electronic device 1500 via bus 1530. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0157] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0158] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. The computer-readable storage medium stores a program product capable of implementing the methods described above. In some possible implementations, various aspects of this disclosure may also be implemented as a program product including program code, which, when run on a user terminal, causes the user terminal to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0159] More specific examples of computer-readable storage media in this disclosure may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0160] In this disclosure, a computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device.

[0161] Optionally, the program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0162] In practical implementation, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0163] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0164] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0165] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0166] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A method for detecting aircraft defects, characterized in that, include: The aircraft to be tested is located to obtain real-time position information, which includes at least one of the following in the UAV coordinate system: the position coordinates of the left main landing gear corresponding to the left rear wheel, the position coordinates of the right main landing gear corresponding to the right rear wheel, and the position coordinates of the front wheel of the aircraft to be tested. Based on the preset position information of each shooting point on the drone flight path and the real-time position information, the corrected position information of each shooting point on the drone flight path is determined. The preset position information of the shooting point includes at least one of the following: shooting point identifier, first distance between shooting point and left rear wheel, second distance between shooting point and right rear wheel, and third distance between shooting point and front wheel. The drone is controlled to acquire images of the aircraft to be detected at the corresponding shooting points based on the corrected position information; Based on the images of the aircraft to be inspected at the corresponding shooting points, the defect information of the aircraft to be inspected is determined; The step of determining the corrected position information of each shooting point on the drone's flight path based on the preset position information and the real-time position information includes: The corrected position information of each shooting point on the UAV flight path is determined by using the maximum likelihood estimation method, the least squares method, or the triangle centroid algorithm. The corrected position information includes the position coordinates of the corresponding shooting point in the UAV coordinate system.

2. The method according to claim 1, characterized in that, The process of locating the aircraft to be tested and obtaining its real-time location information includes: The aircraft under test is located using real-time dynamic RTK positioning to obtain the real-time location information.

3. The method according to claim 2, characterized in that, The step of locating the aircraft under test using real-time dynamic RTK positioning to obtain the real-time location information includes: In the UAV coordinate system, the position coordinates of any two wheels of the aircraft to be tested are collected, wherein the any two wheels are any two of the left rear wheel, the right rear wheel, and the front wheel; Based on the preset installation parameters and the position coordinates of any two wheels, the position coordinates of the remaining wheels of the aircraft to be tested are determined, and the real-time position information is obtained.

4. The method according to claim 1, characterized in that, Each of the aforementioned shooting locations has multiple shooting areas distributed in an array, and each shooting area corresponds to a set of UAV operating parameters; The controlled drone, based on the corrected position information, acquires images of the aircraft to be detected at corresponding shooting points, including: Control the drone to the corresponding shooting location; The operating parameters of the UAV are determined based on the shooting area, and the corresponding shooting area is photographed to obtain the regional image of the UAV under test in the corresponding shooting area. The regional image is stored on the server, and the storage information of the regional image includes the shooting point to which the regional image belongs and the shooting area identifier corresponding to the regional image.

5. The method according to any one of claims 1-4, characterized in that, The step of determining the defect information of the aircraft under inspection based on the images of the aircraft under inspection at the corresponding shooting points includes: The pre-trained defect detection model identifies defects in images of the aircraft to be detected at corresponding shooting points.

6. An aircraft defect detection system, characterized in that, Defect detection is performed using the aircraft defect detection method as described in any one of claims 1-5, wherein the system includes a drone and a controller, wherein, The drone is used to locate the aircraft to be tested and obtain real-time location information; The controller is used to determine the corrected position information of each shooting point on the drone's flight path based on the preset position information of each shooting point and the real-time position information; control the drone to acquire images of the aircraft to be tested at the corresponding shooting points based on the corrected position information; and determine the defect information of the aircraft to be tested based on the images of the aircraft to be tested at the corresponding shooting points.

7. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the aircraft defect detection method according to any one of claims 1 to 5 by executing the executable instructions.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the aircraft defect detection method according to any one of claims 1 to 5.

Images