Unmanned aerial vehicle in-flight dynamic vibration detection method

By integrating a high-resolution camera and a generative adversarial network (GAN) onto an unmanned aerial vehicle (UAV), vibration state images are generated and evaluated in real time, solving the problems of real-time performance and accuracy in UAV vibration detection. This enables low-cost, high-efficiency vibration monitoring and control, suitable for various environments.

CN118982718AInactive Publication Date: 2024-11-19TIANBAO AVIATION TECHNOLOGY (HEBEI) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411119065.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-11-19
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing vibration detection methods for unmanned aerial vehicles (UAVs) cannot meet the high standards of real-time performance, accuracy, stability, and environmental adaptability. Especially when performing critical missions, they cannot monitor and control vibration in real time and accurately, and may affect UAV performance and increase costs.

Method used

Using a high-resolution camera and generative adversarial network (GAN) technology, images simulating vibration conditions are generated through image processing and generator training. The vibration performance of the unmanned aerial vehicle (UAV) is evaluated in real time. The image is transformed and adjusted using GAN to achieve real-time vibration detection of the UAV during flight.

Benefits of technology

It enables real-time and accurate vibration monitoring of unmanned aerial vehicles during actual flight, reduces testing costs, avoids impacting vehicle performance, and ensures stable operation in various environments, thereby improving the automation and accuracy of testing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982718A_ABST
    Figure CN118982718A_ABST
Patent Text Reader

Abstract

The invention relates to an unmanned aerial vehicle in-flight dynamic vibration detection method. The vibration performance of an unmanned aerial vehicle is evaluated by acquiring image data of the unmanned aerial vehicle in different states and utilizing a training generator and an adjustment function. The method comprises the following steps: firstly, capturing an initial image A in a non-vibration state, then generating an image A'in a simulated vibration state by using a generator GAB in a generative adversarial network framework, and calculating a change envelope line set L of an adjustment function; then, acquiring a vibration state image B in actual flight, converting the vibration state image B into a false image A in a non-vibration state by using another generator GBA, and similarly calculating a change envelope line set H of the adjustment function; and finally, comparing the two change envelope line sets, and if H is surrounded by L, judging that the vibration performance of the unmanned aerial vehicle meets the requirement. According to the method, image processing and machine learning technologies are effectively integrated, the vibration state of the unmanned aerial vehicle in actual flight can be accurately detected, and the flight performance and safety of the unmanned aerial vehicle are ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of unmanned aerial vehicles, and in particular to a method for detecting dynamic vibration of an unmanned aerial vehicle in flight. Background Art

[0002] In the field of modern aviation and drone technology, unmanned aerial vehicles (UAVs) are widely used in military reconnaissance, geographic surveying and mapping, agricultural monitoring, disaster relief and other fields due to their efficient and flexible operating characteristics. With the continuous expansion of the scope of application, the stability and safety issues of UAVs have become increasingly prominent, among which vibration control is the key link to ensure their stable flight and precise execution of tasks. The presence of vibration not only affects the structural integrity of the aircraft, but may also lead to a decrease in navigation accuracy, and in extreme cases may even cause mission failure or damage to the aircraft. Therefore, real-time and accurate detection and control of the vibration state of UAVs is an important technical challenge facing the current field of aerospace engineering.

[0003] At present, the vibration detection of unmanned aerial vehicles mainly relies on ground-based vibration detection platforms, which can perform static vibration tests on the aircraft before takeoff. However, this method has obvious limitations: First, the vibration detection platform cannot simulate the complex airflow, temperature changes and other dynamic environmental factors encountered by the aircraft in actual flight, so the test results often cannot fully reflect the vibration characteristics of the aircraft in actual flight. Secondly, ground detection cannot achieve real-time monitoring, and the sudden vibration after the aircraft takes off cannot be discovered and handled in time, which is particularly unfavorable for long-term and long-distance flight missions. In addition, ground detection usually requires the aircraft to stop running. Frequent testing not only affects the operating efficiency of the aircraft, but may also increase the maintenance cost and failure risk of the aircraft due to repeated disassembly and assembly.

[0004] In order to overcome the above defects, researchers have explored sensor-based real-time vibration detection methods in flight. These methods collect vibration data by installing sensors such as accelerometers and gyroscopes on the aircraft, and transmit the data to the ground station or cloud server through wireless transmission technology for processing and analysis. Although this method has improved in real-time and dynamic monitoring, it still has some shortcomings. The installation of sensors may change the mass distribution and aerodynamic characteristics of the aircraft, thereby affecting its flight performance. At the same time, the sensor itself may also be affected by environmental factors such as temperature changes, electromagnetic interference, etc., which may affect the accuracy and reliability of the data. In addition, a large amount of sensor data requires efficient data processing algorithms and powerful computing resources to achieve rapid analysis, which undoubtedly increases the complexity and cost of the system.

[0005] In addition to sensor methods, non-contact detection technologies such as laser Doppler vibrometers are also used in vibration detection of unmanned aerial vehicles. This type of equipment can measure the vibration characteristics of the aircraft without contacting it, thus avoiding affecting the performance of the aircraft. However, equipment such as laser Doppler vibrometers are usually expensive and have strict requirements on environmental conditions, such as avoiding strong light interference and maintaining stable temperature and humidity conditions. These limitations make it difficult for non-contact detection technology to be widely used in the field or in harsh environments.

[0006] In summary, the existing vibration detection methods for unmanned aerial vehicles have certain limitations and cannot meet the high standards for real-time, accuracy, stability and environmental adaptability. Especially when unmanned aerial vehicles perform critical tasks, real-time detection and control of vibration are crucial to ensure the successful completion of the mission. Therefore, the development of a new type of vibration detection system for unmanned aerial vehicles, which can not only realize real-time and accurate monitoring of the vibration state of the aircraft in actual flight, but also has no negative impact on the performance of the aircraft, and can work stably under various environmental conditions, has become an urgent need in the field of aeronautical engineering. Summary of the invention

[0007] The present invention aims to design a method for detecting the dynamic vibration of an unmanned aerial vehicle in flight, so as to solve the problems that the traditional test method has high cost and low accuracy of the detection result.

[0008] The present invention provides a method for detecting dynamic vibration of an unmanned aerial vehicle in flight, comprising: Step 1: Obtain an initial image A of the UAV in a balanced hovering and vibration-free state; Step 2: By training the generator G AB and an adjustment function to generate a series of images A' under simulated vibration state from the initial image A, where A'∈Q{A 1 '、A 2 '、A 3 '......A n '}, n is a positive integer, and obtains the change of the adjustment function in the transition process from A to each A', and obtains the change envelope set L of the adjustment function; Step 3: Obtain an image B of the UAV in a vibration state during actual flight; Step 4: By training the generator G BA and an adjustment function, converting the image B in the vibration state into the false image A in the non-vibration state, and calculating the change of the adjustment function in the transition process from B to the false image A, and obtaining a set of change envelopes H of the adjustment function; Step 5: Compare the change envelope set H with the change envelope set L. If the envelope H is enclosed by the envelope L, it is determined that the vibration performance of the unmanned aerial vehicle meets the requirements. Otherwise, it is determined that the vibration performance of the unmanned aerial vehicle does not meet the requirements.

[0009] Preferably, the step 1 comprises: The setup includes at least one high-resolution camera for capturing image data in real time during flight; Integrate the control system and image processing system of unmanned aerial vehicles to achieve real-time data transmission; Warm up and fly the UAV under no or minimal vibration conditions until a stable hover is achieved; An initial image A of the unmanned aerial vehicle in a balanced hovering and vibration-free state is obtained as reference data.

[0010] Preferably, step 2 includes collecting multiple image data, including images of the unmanned aerial vehicle in different vibration states and corresponding vibration-free images; and preprocessing the collected image data, including image size standardization, denoising, brightness and contrast adjustment.

[0011] Preferably, in step 2, the training generator G AB A generative adversarial network framework is adopted, including a generator G, which is used to generate an image A' in a simulated vibration state; Discriminator D, used to distinguish the image in the real vibration state from the image A' generated by the generator; The performance of the discriminator D and the generator G is optimized through iterative training.

[0012] Preferably, in step 4, a generative adversarial network framework is used to train G BA ,include: The generator G accepts the actual vibration image B as input and outputs a fake vibration-free image A; The discriminator D distinguishes between the real vibration-free image A and the fake image A produced by the generator; Through the standard generative adversarial network training process, the performance of the generator G and the discriminator D is optimized.

[0013] Preferably, in step 5, The change envelope L of the adjustment function of the transition process from the initial image A to a series of images A' under simulated vibration state is: L={l 1 ,l 2 ,l 3 ,...,l n}; The set of changes in the image adjustment function during the transition from the actual flight vibration state image B to the vibration-free state image A is: H={h 1 ,h 2 ,h 3 ,...,h m}, Wherein, n and m are both natural integers.

[0014] Preferably, the adjustment function is adjusted according to the actual captured image to implement the auxiliary generator G AB or G BA , generate a simulated vibration state diagram A' or a non-vibration state pseudo diagram A, and the change of the point can be determined by adjusting the function. The specific implementation formula is:

[0015] Where (x, y) is the point in the original image, (x', y') is the adjusted point, (a, b, c, d) control rotation and scaling, and tx, ty control translation.

[0016] Beneficial effects of the present invention: 1. Enhanced real-time and dynamic performance: The present invention can monitor and evaluate the vibration state of the UAV in real time during its actual flight. Compared with the traditional static detection method of the ground test bench, the present invention can provide more real-time and dynamic data analysis, thereby more accurately reflecting the vibration characteristics of the UAV in the actual flight environment.

[0017] 2. Cost-effectiveness: Compared with the use of high-cost professional equipment such as laser Doppler vibrometers, the present invention uses image processing technology and generative adversarial network (GAN) technology for vibration analysis. The application of these technologies greatly reduces the complexity and cost of detection equipment, making vibration detection more economical and efficient.

[0018] 3. Non-invasiveness: The present invention does not require the installation of additional physical sensors on the UAV, thus avoiding possible impacts on the performance and aerodynamic characteristics of the aircraft, while simplifying the detection process and reducing the need for aircraft modifications.

[0019] 4. Automation and intelligence: By using the generative adversarial network to automatically learn and adjust, the present invention reduces the reliance on human intervention and improves the accuracy and reliability of detection. In addition, the automated process speeds up data processing and improves efficiency.

[0020] 5. Quickly obtain the range of variation of the adjustment function: Use the pre-trained generator (G AB and G BA ) and adjustment function, the present invention can quickly generate the required simulated vibration image when the drone is hovering and flying, so as to quickly obtain the variation range of the adjustment function.

[0021] 6. Direct evaluation of vibration performance: By comparing the transition process of the vibration state image in actual flight with the range of changes in the simulated state, the present invention can directly evaluate whether the vibration performance of the UAV in the actual environment meets the expected design standards, and effectively determine whether the aircraft meets the operational safety and performance standards.

[0022] 7. Easy to implement and widely used: The present invention does not rely on complex hardware equipment. Based on the existing UAV equipment, the function upgrade can be achieved through software update and algorithm optimization, which has high economic benefits and broad application prospects. This makes the present invention flexibly applicable to various types of UAVs, providing a new and efficient detection method for the development and maintenance of UAVs. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0024] Figure 1 : Flow chart of the detection method provided by the present invention Figure 2 : Flow chart of the method for obtaining a reference image provided by the present invention; Figure 3 : Logic diagram of the present invention; Figure 4 : Correspondence diagram from the original image point to the changed image point; Figure 5 : Comparative analysis diagram of envelope sets L and H. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the following is a brief description of the present invention in conjunction with the attached Figure 1-5 The present application is further described in detail with specific implementation methods.

[0027] The present invention provides a method for detecting the dynamic vibration of an unmanned aerial vehicle in flight, which aims to evaluate the vibration performance of the unmanned aerial vehicle in actual flight in real time through advanced image processing and deep learning technology. This embodiment will describe in detail the specific operating steps, technical solutions and expected effects of the method to ensure that those skilled in the art can implement the present invention according to this specification.

[0028] First, it is necessary to ensure that the UAV is equipped with high-performance image capture equipment, such as a high-resolution camera, to capture image data in real time during flight. In addition, the control system of the UAV needs to be integrated with the image processing system to facilitate real-time data transmission and processing.

[0029] The UAV should be preheated under the condition of no vibration or minimal vibration to reach a stable hovering state. At this time, an image capture device is used to obtain an initial image A of the UAV in a balanced hovering and vibration-free state as the baseline data.

[0030] Secondly, prepare the data set, specifically: To train the generator G AB and G BA , it is necessary to collect a large amount of image data, including images of the UAV in different vibration states and the corresponding images without vibration. These data can be obtained from historical flight records or generated by artificially creating vibrations in a simulated vibration environment on the ground.

[0031] All collected image data need to be preprocessed, including image size standardization, denoising, brightness and contrast adjustment, etc., to ensure data consistency and quality.

[0032] like Figure 3 As shown, then the generator G AB The training of the generator G AB The main task of the algorithm is to learn the image conversion from the non-vibration state to the vibrating state. The generator GAB is trained using the Generative Adversarial Network (GAN) framework. GAN consists of two parts: the generator G and the discriminator D. The generator G is responsible for generating images A' in a simulated vibrating state. The discriminator D tries to distinguish between images in a real vibrating state and images A' generated by the generator. During the training process, the generator G and the discriminator D play a game and gradually optimize through iterative training. The goal of the discriminator D is to correctly distinguish between real images and generated images. Its loss function can be shown as formula (1): (1) Among them, D(x) is the probability that the discriminator judges the real image as real, and D(G(z)) is the probability that the discriminator judges the generated image as real.

[0033] The goal of the generator G is to generate images that are realistic enough to deceive the discriminator D. Its loss function can be expressed as: (2) Here, D(G(z)) is the probability that the discriminator judges the generated image as real.

[0034] By alternately training the discriminator D and the generator G, the probability of the discriminator correctly distinguishing images is minimized, thereby optimizing the performance of the generator.

[0035] At the same time, for the generator G BA Train the generator G BA Responsible for converting the image B in the actual vibration state into a fake image A in the non-vibration state. Generator G BA The main goal of is to learn the transformation from an image B of an actual vibration state to a fake image A of a vibration-free state. This requires that the generated fake image A should not only be visually close to the real vibration-free image, but also maintain consistency in semantic content in order to accurately evaluate the vibration performance of the unmanned aerial vehicle. To achieve this goal, we use a generative adversarial network (GAN) framework, which consists of a generator and a discriminator. In this framework, we do not introduce additional cycle consistency losses, but optimize the performance of the generator through the standard GAN training process. The generator G accepts the actual vibration image B as input and outputs a fake vibration-free image A. The discriminator D distinguishes between the real vibration-free image A and the fake image A generated by the generator. In standard GAN training, we have two main loss functions: Generator loss: Encourage the generator to produce images that can deceive the discriminator. This can be expressed as follows: (3) Among them, D(G(B)) is the probability that the discriminator judges the fake image A output by the generator as real.

[0036] Discriminator loss: aims to improve the discriminator's ability to distinguish between real and fake images. This can be expressed by the following formula (4): (4) Where D(A) is the probability that the discriminator judges the real vibration-free image as real, and D(G(B)) is the probability that the discriminator judges the generated image as real.

[0037] By alternately optimizing the generator G and the discriminator D, the quality of the generated image is gradually improved, making the generated fake image A closer and closer to the real vibration-free image.

[0038] For G AB and G BA After the training is completed, the unmanned aerial vehicle is tested for dynamic vibration during flight. Specifically: In the actual flight of the UAV, the image B in flight is acquired in real time through the image capture device. At the same time, the generator G AB A series of images A' under simulated vibration state are continuously generated, and the change of the adjustment function in the transition process from the initial image A to each A' is calculated to obtain the change envelope set L of the adjustment function.

[0039] For the image B acquired in real time, use the trained generator G BA Convert it into a fake image A in a vibration-free state, and calculate the change of the adjustment function in the transition process from B to the fake image A, and obtain a set of change envelopes H of the adjustment function; The adjustment function maps a point (x,y) in one image to a point (x',y') in another image through a mathematical transformation. This process involves rotation (controlled by a,b,c,d) and translation (controlled by tx,ty). In practical applications of unmanned aerial vehicles, these adjustments are based on the image B captured in real time during flight, converted to an image A in a vibration-free state, or generate an image A' under simulated vibration from the initial image A.

[0040] In the generator G AB and G BA During the training process of the reshaping function, the parameters (a, b, c, d, tx, ty) are optimized as part of the generator. The generator takes as input the real or reference image and outputs the reshaped image. The discriminator D receives these two images (real and generated) and tries to distinguish them. Through iterative training, the parameters of the reshaping function are refined so that the generated image is as close as possible to the real target image in terms of visual and semantics.

[0041] In actual flight, the UAV’s image processing system uses the adjustment function in real time to transform the captured image B. By applying the trained parameters (a, b, c, d, tx, ty), the system transforms the image B in the vibration state into a fake image A in the non-vibration state. This transformation allows for a quick assessment of the vibration performance of the aircraft in the actual flight environment and comparison with the performance in the simulated vibration state.

[0042] The generator is used to generate a fake image A' based on the actual in-flight vibration state image B. This process involves applying a resizing function controlled by rotation, scaling, and translation parameters (a, b, c, d, tx, ty). Figure 4 , the origin on the vibration state image B can be converted into image B by adjusting the function parameter settings and generator. Figure 4 Middle b image.

[0043] The degree of coincidence is quantified by calculating the similarity or overlapping area between the fake image A' (set H) and the standard reference image A (set L). Figure 5 As shown, the vibration state of the UAV at the point where the envelope set H is surrounded by the envelope line L meets the vibration performance requirements; wherein, the point beyond the envelope set L indicates that the vibration state of the UAV at this time does not meet the vibration performance requirements. In the present application, it is stipulated that within 2s, the proportion of the image envelope line H of the vibration state of the UAV that exceeds the envelope line set L is not greater than 10%, and it is determined that the vibration performance of the UAV meets the requirements.

[0044] After completing the vibration performance evaluation, all test results should be recorded and fed back to the user. If it is found that the vibration performance of the UAV does not meet the requirements, the vibration control parameters or structural design of the aircraft should be adjusted according to the test results to optimize its performance.

[0045] It should be noted that, in this article, terms such as "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or apparatus.

[0046] The embodiments described above are only descriptions of the preferred methods of the present application, and are not intended to limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements made to the technical solutions of the present application by ordinary technicians in this field should fall within the protection scope determined by the claims of the present application.

Claims

1. A method for detecting dynamic vibration of an unmanned aerial vehicle in flight, characterized in that: include: Step 1: Obtain an initial image A of the UAV in a balanced hovering and vibration-free state; Step 2: By training the generator G AB and an adjustment function to generate a series of images A' in a simulated vibration state from the initial image A, where A'∈Q{A1', A2', A3'...A n '}, n is a positive integer, and obtains the change of the adjustment function in the transition process from A to each A', and obtains the change envelope set L of the adjustment function; Step 3: Obtain an image B of the UAV in a vibration state during actual flight; Step 4: By training the generator G BA and an adjustment function, converting the image B in the vibration state into the false image A in the non-vibration state, and calculating the change of the adjustment function in the transition process from B to the false image A, and obtaining a set of change envelopes H of the adjustment function; Step 5: Compare the change envelope set H with the change envelope set L. If the envelope H is enclosed by the envelope L, it is determined that the vibration performance of the unmanned aerial vehicle meets the requirements. Otherwise, it is determined that the vibration performance of the unmanned aerial vehicle does not meet the requirements.

2. The method for detecting dynamic vibration of an unmanned aerial vehicle in flight according to claim 1, characterized in that: The step 1 comprises: The setup includes at least one high-resolution camera for capturing image data in real time during flight; Integrate the control system and image processing system of unmanned aerial vehicles to achieve real-time data transmission; Warm up and fly the UAV under no or minimal vibration conditions until a stable hover is achieved.

3. The method for detecting dynamic vibration of an unmanned aerial vehicle in flight according to claim 1, characterized in that: The step 2 includes collecting a plurality of image data, including images of the UAV in different vibration states and corresponding images without vibration; and preprocessing the collected image data, including image size standardization, denoising, brightness and contrast adjustment.

4. The method for detecting dynamic vibration of an unmanned aerial vehicle in flight according to claim 3, characterized in that: In step 2, the generator G is trained AB A generative adversarial network framework is adopted, including a generator G, which is used to generate an image A' in a simulated vibration state; Discriminator D, used to distinguish the image in the real vibration state from the image A' generated by the generator; The performance of the discriminator D and the generator G is optimized through iterative training.

5. The method for detecting dynamic vibration of an unmanned aerial vehicle in flight according to claim 1, characterized in that: In step 4, G is trained using a generative adversarial network framework. BA ,include: The generator G accepts the actual vibration image B as input and outputs a fake vibration-free image A; The discriminator D distinguishes between the real vibration-free image A and the fake image A produced by the generator; Through the standard generative adversarial network training process, the performance of the generator G and the discriminator D is optimized.

6. The method for detecting dynamic vibration of an unmanned aerial vehicle in flight according to claim 1, characterized in that: In step 5, The change envelope L of the adjustment function of the transition process from the initial image A to a series of images A' under simulated vibration state is: L={l1,l2,l3,...,l n}; The set of changes in the image adjustment function during the transition from the actual flight vibration state image B to the vibration-free state image A is: H={h1,h2,h3,...,h m }, Wherein, n and m are both natural integers.

7. The method for detecting dynamic vibration of an unmanned aerial vehicle in flight according to claim 6, characterized in that: The adjustment function is adjusted according to the actual captured image to realize the auxiliary generator G AB or G BA , generate a simulated vibration state diagram or a non-vibration state false diagram, and the change of the point can be determined by adjusting the function. The specific implementation formula is: 8.Where (x, y) is the point in the original image, (x', y') is the adjusted point, (a, b, c, d) control rotation and scaling, and tx, ty control translation.