Parachute opening failure discrimination method and device, and recovery landing point judgment method

Abstract

The application relates to a parachute opening failure discrimination method and device and a recovery landing point judgment method. An enhanced data set is obtained by performing enhancement processing on an obtained data set, and the generalization capability and robustness of a training model during training can be improved by using the enhanced data set; a threshold setting method is used to mark the data set to obtain a labeled data set, and the labeled data set comprises a first parachute image and a second parachute image; the actual projection area is corrected by reconstructing the opening area of the second parachute image, so that the actual opening area of the parachute can be more accurately obtained, and the accuracy of model training is improved; in the model construction stage, a lightweight failure discrimination model is used, the accuracy and operation efficiency are considered, the hardware requirement is reduced, the model has the characteristics of lightweight portability, and is more easily deployed in a small spacecraft recovery project.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a parachute opening failure discrimination method and device and a recovery landing point judgment method. BACKGROUND

[0002] The spacecraft deceleration and landing process contains a large number of uncertain factors. Among them, whether the parachute main parachute of the recovery system opens successfully has the greatest influence on the landing process trajectory. The existing parachute working condition detection and failure discrimination methods include manual detection and judgment, travel switch self-detection, and image recognition detection. Due to the low height, high speed and tight timing arrangement of the recovery system in operation, it is difficult to realize manual remote control detection and switch operation mode. The travel switch self-detection is to automatically detect and discriminate the working condition of the parachute through the signals sent by the travel switch and two height signalers. The image recognition detection is to process the actual shooting pictures of the parachute by using digital image technology to obtain the parachute cloth projection area, the attitude of the parachute system, the relative motion and other information in the opening process, so as to realize the judgment of the working state of the parachute. Compared with the manual method, the image recognition computer reading method takes less time, has high precision and can measure more parameters, but has high requirements for the image processing method; compared with the travel switch self-detection, the image recognition method can be used as an effective backup detection means to enhance the reliability of the opening discrimination through redundant backup. A well-known test is that the personnel of the NASA agency and the U.S. Army Research Center installed hundreds of reflective markers inside the parachute cloth to analyze the inflation and stable descent performance of the parachute, and then used cameras installed at different positions to shoot the position changes of the "point cloud" in the parachute working process, and finally obtained the three-dimensional shape of the parachute cloth, and finally obtained good measurement results.

[0003] However, the existing image processing method algorithm is difficult. In traditional image processing, the construction of data set and feature extraction are the keys, among which the feature extraction mainly depends on the artificially designed feature extractor. The parameter adjustment process is complex, and requires strong related professional knowledge. In addition, each method is different for specific applications, and the robustness and generalization ability of the method are poor. Therefore, there are many parameters to be considered, and the image task defined, designed and understood by man can only be used for some specific occasions. On this basis, there are also problems of large calculation resource consumption and high model deployment difficulty. Therefore, the traditional image recognition method is too bulky for some small spacecraft recovery projects, which makes the actual effect of discriminating the opening not meet the expectation. SUMMARY

[0004] Therefore, it is necessary to provide a lightweight parachute opening failure discrimination method and device which can construct data set accurately, reduce calculation amount, improve algorithm robustness and generalization ability, and provide a parachute recovery landing point judgment method.

[0005] A parachute opening failure discrimination method, the method comprises:

[0006] Obtain a data set, enhance the data set to obtain an enhanced data set;

[0007] Label the enhanced data set to obtain a labeled data set, the labeled data set comprising a first parachute image and a second parachute image; wherein the second parachute image is reconstructed to obtain the projected area of the parachute, and the projected area of the parachute is judged according to a set threshold, so as to label the second parachute image;

[0008] Construct a failure discrimination model, and train the failure discrimination model with the first parachute image and the second parachute image to obtain a trained failure discrimination model;

[0009] Discriminate the parachute opening condition through the trained failure discrimination model.

[0010] In one embodiment, the data set is obtained by Raspberry Pi; the Raspberry Pi is deployed on the sky side and the ground side respectively, and image transmission is performed between the Raspberry Pi on the sky side and the Raspberry Pi on the ground side to obtain the data set.

[0011] In one embodiment, reconstructing the second parachute image to obtain the projected area of the parachute comprises: reconstructing the second parachute image to obtain the projected area of the parachute by using a travel switch self-checking, restoring a circle according to a known arc, and a geometric relationship correction method.

[0012] In one embodiment, the travel switch self-checking comprises: setting a discrimination start time, starting a discrimination program when the drop time reaches the discrimination start time, and obtaining the second parachute image by the sky side Raspberry Pi camera and transmitting the second parachute image to the ground side Raspberry Pi.

[0013] In one embodiment, the restoring a circle according to a known arc comprises: calculating the radius of the parachute in the second parachute image, represented as:

[0014] r2 = (|O4A| + |O4B| + |O4C|) / 3;

[0015] Wherein, r2 represents the measured radius of the parachute in the second parachute image, A, B, and C represent randomly labeled points on the edge of the parachute in the parachute opening image, and O4 represents the estimated center of the circle.

[0016] In one embodiment, the geometric relationship correction method comprises:

[0017] The angle Θ of the parachute center deviating from the center axis of the field of view is calculated, that is:

[0018]

[0019] Solving by similar triangles, we get:

[0020]

[0021]

[0022] By the trigonometric relationship sin 2 Θ+cos 2 Θ=1, the opening percentage δ is obtained;

[0023] The actual drag area of the parachute in the second parachute image is calculated by the opening percentage δ to obtain the opening radius R1;

[0024] Where O represents the center of the field of view, O'2 represents the projection of O2 on the horizontal plane of the camera, r1 represents the estimated radius of the parachute in the second parachute image, r2 represents the measured radius of the parachute in the second parachute image, R full represents the radius corresponding to the drag area of the parachute when fully opened, l represents the distance from the sky-end Raspberry Pi camera to the opening section, and r full represents the radius of the second parachute image in the aligned and fully opened state.

[0025] In one embodiment, the fault discrimination model includes a convolutional neural network and a self-attention module.

[0026] The convolutional neural network has local perception ability, and the self-attention module can capture global receptive fields.

[0027] A parachute opening failure discrimination device, the device comprises:

[0028] A data acquisition module is configured to acquire a data set, perform enhancement processing on the data set, and obtain an enhanced data set;

[0029] A data labeling module is configured to label the enhanced data set to obtain a labeled data set, the labeled data set including first parachute images and second parachute images; wherein the second parachute images are reconstructed to obtain the projection area of the parachute, and the projection area of the parachute is judged according to a set threshold, so that the second parachute images are labeled;

[0030] A model training module is configured to construct a fault discrimination model, and train the fault discrimination model with the first parachute images and the second parachute images to obtain a trained fault discrimination model;

[0031] The umbrella opening discrimination module is configured to discriminate the parachute opening situation by using the trained fault discrimination model.

[0032] The parachute recovery landing point judgment method adopts the parachute opening fault discrimination method;

[0033] The parachute recovery landing point judgment method comprises:

[0034] The characteristic area of the parachute is calculated according to the parachute image information obtained by the sky-end Raspberry Pi camera;

[0035] Based on the three-degree-of-freedom dynamics equation, the speed of the parachute at different times is calculated according to the characteristic area of the parachute;

[0036] According to the speed of the parachute at different times, the parachute motion trajectory and the final landing point position are calculated.

[0037] In one embodiment, the three-degree-of-freedom dynamics equation is expressed as:

[0038]

[0039] wherein, represents the acceleration of the parachute in the x, y, and z directions; θ represents the velocity inclination angle; γ represents the velocity deflection angle; m represents the mass of the parachute system; g represents the acceleration of gravity; F D represents the aerodynamic drag; a 11 , a 22 , a 33 respectively represent three components of the additional mass.

[0040] The parachute opening failure discrimination method, device and equipment described above, by obtaining a data set and performing enhancement processing on the data set, obtain an enhanced data set, the enhanced data set improves the data volume under different scenarios, and can improve the generalization ability and robustness when training the model; the enhanced data set is labeled to obtain a labeled data set, which includes a first parachute image and a second parachute image; wherein the second parachute image is reconstructed to obtain the projection area of the parachute, and the projection area of the parachute is judged according to the set threshold, so as to label the second parachute image. By reconstructing the opening area of the second parachute image, the actual opening area of the parachute can be more accurately obtained, and the accuracy of model training is improved; a failure discrimination model is constructed, and the first parachute image and the second parachute image are used to train the failure discrimination model to obtain a trained failure discrimination model; the opening condition of the parachute is discriminated by the trained failure discrimination model. Since a lightweight failure discrimination model is used, the accuracy and computational efficiency are taken into account, the computational resource consumption of the model is reduced, the calculation time is shortened, the money cost is reduced, and the model has the characteristics of lightweight portability, which is more easily deployed in small spacecraft recovery projects. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 A parachute opening failure discrimination method flowchart is provided for embodiment 1.

[0042] Figure 2 A raspberry pi image transmission platform connection diagram is provided for embodiment 1.

[0043] Figure 3 A data enhancement diagram is provided for embodiment 1.

[0044] Figure 4 A first parachute image labeling diagram is provided for embodiment 1.

[0045] Figure 5 A normal parachute and camera position relationship diagram is provided for embodiment 1.

[0046] Figure 6 A parachute and camera position relationship diagram when the parachute is inclined is provided for embodiment 1.

[0047] Figure 7 A relative position diagram of the parachute and the camera when the camera can only capture part of the parachute is provided for embodiment 1.

[0048] Figure 8 A diagram of three possible situations for the first parachute image is provided for embodiment 1, where (a) represents the first situation, (b) represents the second situation, and (c) represents the third situation.

[0049] Figure 9Figure for calculating the chord randomly taken in Example 1;

[0050] Figure 10 Figure for correcting the geometric relationship in Example 1;

[0051] Figure 11 Figure for restoring the photographic image in the case of deflection in Example 1;

[0052] Figure 12 Figure for real-time judgment result, wherein (a) represents the image to be judged, and (b) represents the judgment result;

[0053] Figure 13 Structure block diagram of the parachute opening failure discrimination device;

[0054] Figure 14 Figure for the percentage of the parachute opening area at a certain time, its speed and coordinates in Example 2;

[0055] Figure 15 Figure for the height change curve of the parachute under three resistance areas in Example 2;

[0056] Figure 16 Figure for the speed change curve of the parachute under three resistance areas in Example 2;

[0057] Figure 17 Three-dimensional motion trajectory of the parachute under three resistance areas in Example 2. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical scheme and advantages of the present application more clear and understandable, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0059] The purpose of the present application is to reduce the complexity of the failure recognition model from the two dimensions of image transmission and deep learning-based image recognition, to meet the needs of discriminating opening failure, to make it more lightweight in the hardware and software algorithm levels relative to the traditional image processing scheme, and to be more easily deployed to small opening recovery projects. It is worth noting that the first parachute image in the present application represents the image of the parachute taken by the camera during the opening process, and the second parachute image represents the image of the parachute taken by the camera during the opening process. The image is only partially displayed; wherein the description of "first", "second" and the like is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of indicated technical features.

[0060] Example 1:

[0061] As Figure 1As shown, the parachute opening failure discrimination method provided by the application comprises the following steps:

[0062] Step 102, obtaining a data set, and performing enhancement processing on the data set to obtain an enhanced data set.

[0063] Specifically, the obtaining of the data set is mainly achieved by respectively deploying Raspberry Pis at the sky end and the ground end, and performing image transmission between the Raspberry Pis at the sky end and the ground end to obtain the data set.

[0064] In this step, Raspberry Pi 4B+8GB version *2, Raspberry Pi power supply base *1, Raspberry Pi camera *1, network card *2, antenna *2, and notebook computer *1 are used, and the device connection is as shown in Figure 2 The specific installation steps are as follows:

[0065] 1) Burn the OpenHD firmware into the two Raspberry Pis to realize real-time transmission of high-definition images. Use the Raspberry Pi charger to power the ground end Raspberry Pi, and use the Raspberry Pi power supply base to power the sky end device.

[0066] 2) Connect the two network cards to the two Raspberry Pis respectively, connect the antenna to the network card, and start the Raspberry Pi.

[0067] 3) Connect the computer to the wireless network "OpenHD" sent by the ground end, and you can get real-time transmission images on the computer.

[0068] The data set is enhanced, because the application is data-driven, and the image data is enhanced to increase the data volume under different scenes to improve the generalization ability and robustness of the trained model. The data enhancement methods that can be used include changing the brightness, contrast and color of the image, rotating, flipping and the like. It also includes adding Gaussian noise, using a generative adversarial network (GAN) to generate a new data set, and the like. The application preferably uses the method of changing the brightness and contrast of the image, and then flipping and splicing the image to enhance it, so that the data set is increased to 5 times the original size, and the data enhancement effect is as shown on the right side of Figure 3 After obtaining the enhanced data set, the enhanced data set is divided into a training set and a validation set in a ratio of 9:1, and a total of 875 training sets and 97 validation sets are obtained.

[0069] Step 104, labeling the enhanced data set to obtain a labeled data set, the labeled data set comprising a first parachute image and a second parachute image; wherein the second parachute image is reconstructed to obtain the projection area of the parachute, and the projection area of the parachute is judged according to the set threshold value, so as to label the second parachute image.

[0070] Generally, the projected area of the open parachute is not less than 75% of the average projected area of the fully inflated parachute. Therefore, the projected area of the fully inflated parachute is first calculated by OpenCV. Specifically, as shown in Figure 4 , first, the original image of the parachute is thresholded to obtain a binary image; the findContours function in OpenCV is used to find the contour of the binary image containing the parachute, and the total pixel points of the binary image containing the parachute contour are calculated to estimate the parachute area.

[0071] After calculating the projected area of the fully inflated parachute, the projected area of the parachute in other cases can be calculated using the same method, and the data set can be labeled. The success threshold of the parachute opening is designed to be 75%, and the projected area greater than 75% of the projected area of the fully inflated parachute is marked as "yes", and vice versa.

[0072] It is worth noting that when the parachute is dropped, the Raspberry Pi image transmission device is below the parachute, and the camera may not be directly below the parachute, i.e. it cannot obtain the first parachute image as shown in Figure 3 , the camera may capture a slanted parachute and only a part of the parachute, or even the parachute is not in the camera's field of view.

[0073] Normally, the relationship between the parachute and the camera should be as shown in Figure 5 , but when the camera captures a slanted parachute, the relationship between the parachute and the camera will be as shown in Figure 6 , but the basis for determining the fully inflated parachute is mainly the comparison between the projected area of the parachute and the projected area of the fully inflated parachute. At this time, the camera is perpendicular to the ground, and the parachute area captured by the camera is exactly the projected area of the parachute, i.e. the method shown in Figure 4 can still be used to determine whether the parachute is fully inflated.

[0074] In another case, during the parachute drop, due to shaking and wind disturbance, etc., the parachute may only be partially in or even not in the camera's field of view, as shown in Figure 7 . At this time, the relationship between the parachute and the camera has many cases, and this embodiment only discusses the case where the parachute is partially in the field of view, i.e. the camera obtains the second parachute image. During this process, because of the shape of the parachute and the complex situation of the parachute that can be captured by the camera, this embodiment first illustrates three cases as shown in Figure 8 , wherein, Figure 8 (a), Figure 8 (b), Figure 8 (c) the upper image in the three images represents the actual opening of the parachute, and the lower image represents the actual image taken by the camera.

[0075] Therefore, when calculating the projected area of the parachute in the second parachute image, the opening image of the parachute taken should be reconstructed first, and the complete opening area of the parachute is estimated from the part of the parachute taken.

[0076] Further specifically, the second parachute image is reconstructed by using the travel switch self-check, restoring the circle from the known circular arc, and the geometric relationship correction method in sequence to obtain the projected area of the parachute.

[0077] Firstly, the discrimination program is started by the travel switch self-check. The discrimination program is started when the air drop time reaches the start time of the discrimination program, the second parachute image is obtained by the sky-end Raspberry Pi camera, and is transmitted to the ground-end Raspberry Pi for image judgment.

[0078] It is worth noting that the present application uses a small parachute, and according to the experimental results, the time required from the start of air drop to the start of the second opening of the main parachute is about 23.5s, and the parachute is still in the opening state before this time, so image recognition is not required. The time delay from the sky-end Raspberry Pi image transmission to the ground-end computer is about 200ms, and the time required from receiving the picture to completing the picture recognition by the computer is about 340ms, which does not affect the final result. Therefore, the discrimination program is started from 23.5s to perform image transmission and recognition, which not only reduces the misjudgment caused by the incomplete opening of the parachute itself, but also saves the calculation cost.

[0079] Secondly, the estimated radius of the parachute in the second parachute image is obtained by the method of restoring the circle from the known circular arc. According to the geometric knowledge: two chords are randomly drawn on the known circular arc, and the intersection point of the perpendicular bisectors of the two chords is the center of the circle corresponding to the circular arc, and the distance from the intersection point to any point on the circular arc is the radius of the circle.

[0080] Further specifically, as shown in Figure 9 , according to the second parachute image transmitted by the sky-end Raspberry Pi, points A, B, and C are randomly marked on the second parachute image, then chords AB and BC are drawn, and then the midpoints M and N of AB and BC are drawn, and the perpendicular bisectors of the connecting line are drawn, the intersection point of the perpendicular bisectors is the estimated center O4, and O4A, O4B, and O4C are the radii of the corresponding circle. Thus, the measurement radius of the parachute in the second parachute image is obtained:

[0081] r2=(|O4A|+|O4B|+|O4C|) / 3.

[0082] ​Finally, the actual opening radius is obtained by correcting the opening radius through geometric relationship. Figure 7 It can be seen that the parachute is at an angle with the vertical direction at this time, and the field of view angle between the camera and the parachute also changes. Therefore, there is an error between the opening radius of the parachute estimated according to the photo taken by the camera and the actual opening radius, and the error is corrected through geometric relationship.

[0083] As shown in Figure 10 and Figure 11 , O1 represents the estimated opening center position, O2 represents the measured center of the parachute in the deflection state, O3 and O4 respectively represent the positions of O1 and O2 in the field of view picture, l represents the distance from the sky-end Raspberry Pi camera to the parachute opening section, O'1 and O'2 are the projections of O1 and O2 on the horizontal plane of the camera. Assuming that O'1 is located at the center of the field of view in the shooting image and coincides with O3 in the alignment case. r1 represents the estimated radius of the parachute in the second parachute image, r2 represents the measured radius of the parachute in the second parachute image, and r full represents the radius of the second parachute image in the alignment and full opening case. R1 is the actual opening radius corresponding to the resistance area of the parachute, and R full is the radius corresponding to the resistance area of the parachute when it is fully opened. Θ represents the angle of the parachute center deviating from the axis of the field of view center. Among the above parameters, l, r full , R full are known quantities of the system, and r2 are measured values in the image, Θ, δ and R1 are to-be-solved quantities.

[0084] The parachute opening percentage δ satisfies:

[0085]

[0086] First, the angle Θ of the parachute center deviating from the axis of the field of view center is calculated, that is:

[0087]

[0088] Among them, |OO2'| is the distance between the center O2 of the parachute opening circle and the axis of the field of view, which can be obtained by the geometric relationship of the shooting image:

[0089]

[0090] By solving the similar triangles, we get:

[0091]

[0092] According to the trigonometric relationship sin 2 Θ+cos 2Θ = 1, the equation can be solved to obtain the unknown opening percentage δ.

[0093] The opening percentage δ is used to determine whether the parachute is fully opened, and the actual drag area of the parachute in the second parachute image is calculated according to the opening percentage δ to obtain the opening radius:

[0094] R1 = δR full .

[0095] Step 106, constructing a fault discrimination model, using the first parachute image and the second parachute image to train the fault discrimination model, obtaining a trained fault discrimination model.

[0096] Specifically, when constructing the fault discrimination model, a lightweight CNN (Convolutional Neural Network) is used, which has the ability of local perception and relatively small calculation amount, and a self-attention module is used, which has the ability to capture global receptive field and has large parameter amount and calculation amount. Preferably, MobileViT can be used for model training, which combines the local perception ability of CNN and the global receptive field capturing ability of Transformer, and balances accuracy and operation efficiency. When training the fault discrimination model, the input is an RGB three-channel image with a size of 224*224, and the output is the parachute opening condition judgment (opening success / failure).

[0097] Step 108, discriminating the parachute opening condition through the trained fault discrimination model.

[0098] Specifically, as shown in Figure 12 , the picture transmitted by the Raspberry Pi is input into the trained fault discrimination model in real time to obtain a real-time judgment result. The result shows that when the parachute opening discrimination (class) is yes, the probability (prob) is 0.999, so the probability of the parachute opening successfully is 0.999, i.e. successful opening.

[0099] It is worth noting that when constructing the fault discrimination model, multiple fault discrimination models based on different deep learning neural networks can be constructed according to requirements, and the first parachute image and the second parachute image are used to train multiple fault discrimination models respectively, and then the fault discrimination model with the best performance on the validation set is selected from the trained models to discriminate the parachute opening condition.

[0100] In order to verify the advantages of the fault discrimination model in calculation efficiency and model accuracy, we use the same data set to train VGG16, ResNet50, and the fault discrimination model respectively, and the training speed and accuracy of each neural network are shown in Table 1.

[0101] Table 1: Comparison of training results of three kinds of neural networks

[0102] Neural Network Name VGG16 ResNet50 Fault detection model Training speed (frames / second) 3.78 5.19 10.71 Model accuracy 95.4％ 99.7％ 98.5％

[0103] According to Table 1, under the condition that the software and hardware used in the experiment and other conditions are the same, the fault discrimination model is the highest in the training speed of the model, far exceeding VGG16 and ResNet50, and the accuracy rate reaches 98.5%, meeting the actual demand.

[0104] The parachute opening failure discrimination method and device, through image detection and deep learning algorithm, judges the canopy projection area, parachute system posture and relative motion and other information, and realizes the judgment of the opening state. The scheme points include: obtaining a data set, performing data enhancement processing on the data set by using a data enhancement method to obtain an enhanced data set, the enhanced data set increases the data amount under different scenes, and the generalization ability and robustness of the training are improved when training the model; the enhanced data set is labeled to obtain a labeled data set, and the labeled data set includes a first parachute image and a second parachute image; wherein, the second parachute image is reconstructed to obtain the projection area of the parachute, and the projection area of the parachute is judged according to the set threshold, so as to label the second parachute image; the labeled parachute image extended by the data set is trained based on a convolutional neural network, the actual projection area is corrected by reconstructing the opening area of the second parachute image, the actual opening area of the parachute can be more accurately obtained, and the accuracy of model training is improved; a fault discrimination model is constructed, and the first parachute image and the second parachute image are used to train the fault discrimination model to obtain a trained fault discrimination model; the opening state of the parachute is discriminated by the trained fault discrimination model, since the lightweight fault discrimination model is used, the accuracy and operation efficiency are considered at the same time, the calculation resource consumption of the model is reduced, the calculation time is shortened, the hardware demand is reduced, the money cost is reduced, and the lightweight portable feature is achieved, which is more easily deployed in a small spacecraft recovery project.

[0105] It should be understood that, although Figure 1 The steps in the flowchart of the method are shown in sequence according to the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this application, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps in the method can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or sub-steps or stages of other steps.

[0106] In one embodiment, as shown in Figure 13 A parachute opening failure discrimination device is provided, comprising:

[0107] A data acquisition module is configured to acquire a data set, and perform enhancement processing on the data set to obtain an enhanced data set.

[0108] A data labeling module is configured to label the enhanced data set to obtain a labeled data set, the labeled data set comprising a first parachute image and a second parachute image; wherein the second parachute image is reconstructed to obtain a projection area of the parachute, and the projection area of the parachute is judged according to a set threshold value, so as to label the second parachute image.

[0109] A model training module is configured to construct a failure discrimination model, and train the failure discrimination model with the first parachute image and the second parachute image to obtain a trained failure discrimination model.

[0110] An opening discrimination module is configured to discriminate the parachute opening condition by using the trained failure discrimination model.

[0111] The specific limitations of the parachute opening failure discrimination device can be referred to the limitations of the parachute opening failure discrimination method in the foregoing, and will not be described here. Each module in the parachute opening failure discrimination device described above can be realized by software, hardware, and combinations thereof, in whole or in part. Each module described above can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0112] Embodiment 2:

[0113] The parachute recovery landing point judgment method provided in this embodiment uses the parachute opening failure discrimination method to judge the recovery landing point, comprising the following steps:

[0114] Step 202: The characteristic area of the parachute is calculated according to the parachute image information obtained by the sky-end Raspberry Pi camera, that is, the actual opening area of the parachute is obtained by the actual resistance area of the parachute corresponding to the opening radius R1 in Embodiment 1. It should be noted that the calculation process can be transmitting the parachute image information obtained by the sky-end to the ground-end for calculation, or directly completed in the sky-end.

[0115] Step 204: The speed of the parachute at different times is calculated according to the characteristic area of the parachute based on the three-degree-of-freedom dynamics equation.

[0116] Firstly, the parachute system is assumed to be a particle, and only the displacement of the system in space is considered. Considering the influence of the additional mass, a spherical empirical formula is used. For a three-degree-of-freedom motion model, the additional mass expression is:

[0117]

[0118] where a 11 , a 22 , and a 33 represent the three components of the additional mass, d p is the parachute canopy projection diameter, and p is the air density.

[0119] It is assumed that the atmospheric density changes with height, and the 1976 U.S. National Standard Atmospheric Model is used, that is, it satisfies:

[0120]

[0121] It is worth noting that during the landing stage, the recovery system is unpowered flight, and only subject to gravity, air resistance and atmospheric wind force; in the parachute descent area of the recovery system, the earth is assumed to be a flat force, without considering the curvature of the earth, and the gravitational acceleration can be considered as a constant.

[0122] Secondly, the drag area is calculated. The drag area reflects the drag characteristics of the canopy and is the product of the canopy drag coefficient c and the projection area A. The drag coefficient in this application is 0.875. The canopy drag area is calculated using an empirical formula. For a one-time closing ring parachute, the product of the canopy drag coefficient and the drag area during the inflation process is calculated as follows:

[0123]

[0124] where C D represents the drag coefficient, S represents the drag area, and the subscripts b, d, and f represent the opening time t b , the release time t d , and the stable descent time t f after full opening, respectively. In particular, at the stable descent time t f after full opening, the drag area is obtained by image recognition in step 202:

[0125] Finally, the three-degree-of-freedom dynamic equation is solved:

[0126]

[0127] where a represents the acceleration of the parachute system in the x, y, and z directions; and q represents the horizontal velocity component of the parachute and the airspeed is the angle between the horizontal velocity component and the velocity component in the x direction, i.e. the velocity inclination angle; γ represents the angle between the x direction velocity component and the horizontal velocity component, i.e. the velocity deflection angle; m represents the mass of the parachute system; g represents the gravitational acceleration; F D represents the aerodynamic drag, and the calculation formula is: wherein, V C represents the airspeed.

[0128] Considering the influence of the wind field, the airspeed vector The formula is:

[0129]

[0130] wherein, V x , V y , V z represent the velocities of the parachute in the x, y, z directions; ω x , ω y respectively represent two horizontal components of the wind speed; respectively represent unit vectors in the x, y, z directions.

[0131] The angle θ between the horizontal velocity component of the parachute and the airspeed is the velocity inclination angle, and the angle γ between the x direction velocity component and the horizontal velocity component is the velocity deflection angle, which respectively satisfy:

[0132]

[0133] wherein, V cy represents the component of the airspeed in the y direction.

[0134] Step 206, according to the velocities of the parachute at different times, the trajectory of the parachute and the final landing position are calculated.

[0135] According to the calculations of steps 202 and 204, the velocities of the parachute at different times are obtained, and on this basis, the trajectory of the parachute and the final landing position can be further calculated. The calculation results are shown in the following table: Figure 14 The opening area is 95.4%, v = 36.1 m / s, x = -8469.5 m, y = -510.3 m, and z = 869.2 m.

[0136] Specifically, the parachute recovery landing point simulation in this embodiment is based on three different parachute opening states, including ideal case (final drag area: 100%), approximate ideal case (final drag area: 95.4%), and main parachute failure case (final drag area: 18.3%). The initial height is set to 10 km, the landing area altitude is 1500 m, the initial combined velocity of the recovery system is 120 m / s, the velocity inclination angle θ = 80°, and the velocity deflection angle γ = 45°.

[0137] The height, speed, and position of the parachute system change over time as follows: Figure 15 , Figure 16 , Figure 17 As shown. Figure 15 As shown, under ideal conditions where the main parachute is fully deployed, an inflection point appears in the image at (25s, 7768.02m), indicating that the main parachute is fully inflated and the parachute system enters a stable descent. At this point, the resultant velocity of the parachute system is 31.74m / s, and the vertical velocity reaches its minimum value of 3.38m / s. Furthermore, the slope of the image near the inflection point is significantly different; the slope before the inflection point is approximately 80m / s, while the slope after the inflection point is approximately 36m / s, indicating that after the parachute is fully deployed, the parachute system descends at an approximately uniform speed. Figure 16 The diagram illustrates the resultant velocity change of the pilot parachute system, with the first 25 seconds representing the parachute inflation process. Within the initial 0.2 seconds, the pilot parachute fully inflates, achieving a drag area of ​​1.0 m². 2 The system began to decelerate and descend. At 15.5 seconds, the main parachute engaged, at which point the parachute system's speed was 110 m / s. 1.5 seconds later, the main parachute remained closed, with a closed area of ​​7.6 m². 2 After this, the parachute system rapidly decelerated. At 23.5 seconds, the closure was released, and the main parachute then fully inflated to a drag area of ​​40.2m². 2 Finally, the parachute landed stably at a speed of 36 m / s. The simulation shows that as the parachute deploys, the drag area of ​​the parachute system increases, so the simulation reflects the actual physical process.

[0138] Comparing the three sets of simulation results, the parachute system descends faster when the main parachute fails compared to when the main parachute is fully deployed. Furthermore, the drag area of ​​the main parachute has a significant impact on the motion of the parachute system. When the final drag area of ​​the main parachute is 95.4% of the drag area when fully deployed, its landing speed is almost the same as in the ideal case, while the landing speed under parachute failure is nearly twice that of the ideal case. This indicates that main parachute failure will lead to a deviation in the landing point, and the greater the degree of failure, the greater the deviation. The resultant velocity of the parachute system under the three cases is shown in Table 2. Compared to the ideal case, the calculated landing point deviations under the two parachute deployment states are: Δx1 = 24m, Δy1 = 222m; Δx2 = 565m, Δy2 = 5483m.

[0139] Table 2: Final Landing Point Parameters for Three Types of Drag Area Parachutes

[0140]

[0141] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] The above-described embodiments are merely illustrative of several embodiments of the present application, which are described in more detail and in a specific manner, but should not be construed as limiting the scope of the present application. It should be noted that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for detecting parachute deployment malfunctions, characterized in that, The method comprises: obtaining a data set, performing enhancement processing on the data set to obtain an enhanced data set; labeling the enhanced data set to obtain a labeled data set, the labeled data set comprising a first parachute image and a second parachute image; wherein the second parachute image is reconstructed to obtain a projection area of the parachute, and the projection area of the parachute is judged according to a set threshold, so that the second parachute image is labeled; constructing a fault discrimination model, and training the fault discrimination model with the first parachute image and the second parachute image to obtain a trained fault discrimination model; discriminating the parachute opening condition through the trained fault discrimination model; reconstructing the second parachute image to obtain the projection area of the parachute, comprising: reconstructing the second parachute image to obtain the projection area of the parachute by using a travel switch self-checking, restoring a circle according to a known circular arc, and a geometric relationship correction method; the restoring a circle according to a known circular arc comprises: calculating the radius of the parachute in the second parachute image, expressed as: ； the geometric relationship correction method comprises: Computing the angle of the parachute center from the center axis of the field of view i.e.: ； solving by similar triangles to obtain: ； ； From the trigonometric relationship , the opening percentage of the umbrella is obtained ; By the percentage of opening the parachute The actual drag area of the parachute in the second parachute image is calculated to correspond to the opening radius ; wherein, represents the measured radius of the parachute in the second parachute image, , , represents the points randomly marked on the parachute edge in the parachute opening image, represents the estimated center of the circle, represents the center of the field of view, represents the projection of the camera's lying horizontal plane, represents the estimated radius of the parachute in the second parachute image, represents the measured radius of the parachute in the second parachute image, represents the radius corresponding to the parachute drag area when fully inflated, represents the distance from the sky end Raspberry Pi camera to the opening cross section of the parachute, represents the radius of the second parachute image in the aligned and fully inflated condition.

2. The parachute deployment failure discrimination method according to claim 1, characterized in that, the data set is obtained through a Raspberry Pi; deploying the Raspberry Pi on the sky end and the ground end respectively, and performing image transmission through the sky end Raspberry Pi and the ground end Raspberry Pi to obtain the data set.

3. The parachute deployment failure discrimination method according to claim 1, characterized in that, The travel switch self-checking comprises: setting a discrimination start time, starting the discrimination program when the air drop time reaches the discrimination start time, and the sky end Raspberry Pi camera obtains the second parachute image and transmits it to the ground end Raspberry Pi.

4. The parachute deployment failure discrimination method according to any one of claims 1 to 3, characterized in that, The fault discrimination model comprises a convolutional neural network and a self-attention module; The convolutional neural network has local perception ability; and the self-attention module can capture a global receptive field.

5. A parachute opening failure determination device characterized by comprising: The parachute opening fault discrimination method comprises: a data acquisition module for obtaining a data set, performing enhancement processing on the data set to obtain an enhanced data set; a data labeling module for labeling the enhanced data set to obtain a labeled data set, the labeled data set comprising a first parachute image and a second parachute image; wherein the second parachute image is reconstructed to obtain a projection area of the parachute, and the projection area of the parachute is judged according to a set threshold, so that the second parachute image is labeled; a model training module for constructing a fault discrimination model, and training the fault discrimination model with the first parachute image and the second parachute image to obtain a trained fault discrimination model; an opening discrimination module for discriminating the parachute opening condition through the trained fault discrimination model.

6. A parachute recovery impact point determination method, characterized by, The parachute opening fault discrimination method comprises: calculating the characteristic area of the parachute according to the parachute image information obtained by the sky end Raspberry Pi camera; calculating the speed of the parachute at different times according to the characteristic area of the parachute based on a three-degree-of-freedom dynamics equation; ​ According to the speed of the parachute at different time, the parachute movement trajectory and the final landing position are calculated.

7. The parachute recovery impact point determination method according to claim 6, wherein The three-degree-of-freedom dynamics equation is expressed as: ； where , , denotes the acceleration of the parachute in , , three directions; denotes the velocity pitch angle; denotes the velocity yaw angle; denotes the mass of the parachute-canister system; denotes the gravitational acceleration; denotes the aerodynamic drag; , , denote the three components of the additional mass, respectively.

Images