Intelligent takeoff and landing DME measurement method

Abstract

This invention belongs to the field of intelligent takeoff and landing technology, specifically relating to an intelligent takeoff and landing DME measurement method, including the following steps: (1) In the YOLOv5 framework, a target prediction constraint mechanism based on runway image features is introduced; (2) A prediction layer loss function is designed to predict the runway profile; (3) The runway profile is detected, and DME distance feature data is constructed as input to the DME prediction model; (4) An RBF neural network structure is designed, and the constructed DME distance feature data is used to train the network. This invention is used for DME ranging in the intelligent takeoff and landing process of aircraft, improving the accuracy of runway detection, thereby constructing more realistic DME distance feature data and predicting more accurate DME distances.

Description

Technical Field

[0001] This invention belongs to the field of intelligent take-off and landing technology, and specifically relates to an intelligent take-off and landing DME measurement method. Background Technology

[0002] With the development of new-generation intelligent technologies, represented by image processing technology, machine vision has been widely applied. Visual navigation technology has advantages such as large space, non-contact, drift-free, and less susceptible to electronic warfare interference. It can play a very important role in the field of autonomous aircraft landing. Therefore, integrating machine vision as navigation source information can improve the accumulation of errors in inertial reference systems, solve the problem of GNSS being susceptible to environmental influences, and improve the accuracy, availability, continuity, and integrity of the original navigation system. This will enable it to meet the high requirements of aircraft takeoff, approach, and landing phases, and will provide a foundation for developing a fully autonomous aircraft takeoff and landing system that is not limited by the environment or dependent on external features. Summary of the Invention

[0003] In view of this, the present invention provides an intelligent take-off and landing DME measurement method that enables DME distance measurement during intelligent take-off and landing of aircraft.

[0004] The specific technical solution of this invention is as follows:

[0005] A smart takeoff and landing DME measurement method includes the following steps:

[0006] S100: Based on runway image features, introduce a target prediction constraint mechanism based on runway image features into the YOLOv5 framework;

[0007] Design the loss function L for the prediction layer. T The prediction layer loss function L Y =L GIoU +L T Among them, L GIoU To introduce the intersection-union ratio loss function of the minimum closure region, L T The runway profile ratio loss function;

[0008] S200: The neural network obtained by training S100 is used to detect the runway contour, and the four corner points of the obtained runway contour are used as feature points.

[0009] S300: Construct DME distance feature data, and construct 1*9 feature data by combining the feature points and runway image DME distance information;

[0010] S400: Design the RBF neural network structure and train the RBF neural network using the constructed DME distance feature data;

[0011] S500: DME is measured based on the trained RBF neural network.

[0012] Furthermore, the target prediction constraint mechanism in S100 is the coordinate relationship of the four corner feature points A1, A2, A3, A4 of the runway profile: x1≤x2&x4≤x3&y3≥y1&y3≥y2&y4≥y1&y4≥y2; where: A1(x1,y1), A2(x2,y2), A3(x3,y3), A4(x4,y4).

[0013] Furthermore, the prediction layer loss function L described in S100 Y for: Wherein: S R S represents the area of ​​the actual runway in the image. r S represents the predicted area of ​​the runway profile in the image. R ∩S r S represents the area of ​​the intersection of the two. R S r S represents the area of ​​the union of the two sets. B (x,y) represents the area of ​​the smallest closure region in the image; (x,y) represents the corner coordinates of the predicted runway profile, and (x′,y′) represents the corresponding corner coordinates of the actual runway.

[0014] Furthermore, the feature data mentioned in S300 is a 1*9 feature vector: [x1,y1,x2,y2,x3,y3,x4,y4,DME].

[0015] Furthermore, the RBF neural network structure described in S400 is as follows: the input layer is 8-dimensional feature data, which is first mapped to a 200-dimensional hidden layer, then mapped to a 1-dimensional hidden layer, and finally mapped to the output layer. The output layer is used to output the predicted DME.

[0016] The beneficial effects of this invention are:

[0017] This invention obtains the Direct Measure (DME) distance between the aircraft and the runway using machine vision methods, providing essential DME distance data for intelligent takeoff and landing. Furthermore, the DME measurement algorithm designed in this invention incorporates a target prediction constraint mechanism based on runway image features, and a designed prediction layer loss function L... T This solves the problems of insensitivity to the overlap direction and angle between the predicted bounding box and the ground truth bounding box, resulting in a more accurate runway profile. This allows for the construction of more realistic DME distance feature data and the prediction of more accurate DME distances. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the present invention;

[0020] Figure 2 These are schematic diagrams of the actual and predicted runway outlines in a specific embodiment of the present invention;

[0021] Figure 3 This is the RBF neural network structure in a specific embodiment of the present invention;

[0022] Figure 4 This is a schematic diagram of the DME prediction results of a specific embodiment of the present invention. Detailed Implementation

[0023] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0024] The following specific examples illustrate the implementation of this disclosure. Those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0025] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using other structures and / or functionalities besides one or more of the aspects set forth herein.

[0026] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The drawings only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0027] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0028] In one embodiment of the present invention, a smart takeoff and landing DME measurement method is proposed, comprising the following steps:

[0029] S100: Based on runway image features, introduce a target prediction constraint mechanism based on runway image features into the YOLOv5 framework;

[0030] Design the loss function L for the prediction layer. T The prediction layer loss function L Y =L GIoU +L T Among them, L GIoU To introduce the intersection-union ratio loss function of the minimum closure region, L T The runway profile ratio loss function;

[0031] S200: The neural network obtained by training S100 is used to detect the runway contour, and the four corner points of the obtained runway contour are used as feature points.

[0032] S300: Construct DME distance feature data, and construct 1*9 feature data by combining the feature points and runway image DME distance information;

[0033] S400: Design the RBF neural network structure and train the RBF neural network using the constructed DME distance feature data;

[0034] S500: DME is measured based on the trained RBF neural network.

[0035] In this embodiment, the target prediction constraint mechanism in S100 is the coordinate relationship of the four corner feature points A1, A2, A3, A4 of the runway profile: x1≤x2&x4≤x3&y3≥y1&y3≥y2&y4≥y1&y4≥y2; where: A1(x1,y1), A2(x2,y2), A3(x3,y3), A4(x4,y4).

[0036] In this embodiment, the prediction layer loss function L mentioned in S100 Y for: Wherein: S R S represents the area of ​​the actual runway in the image. r S represents the predicted area of ​​the runway profile in the image. R ∩S r S represents the area of ​​the intersection of the two. R ∪S r S represents the area of ​​the union of the two sets. B (x,y) represents the area of ​​the smallest closure region in the image; (x,y) represents the corner coordinates of the predicted runway profile, and (x′,y′) represents the corresponding corner coordinates of the actual runway.

[0037] In this embodiment, the feature data mentioned in S300 is a 1*9 feature vector: [x1,y1,x2,y2,x3,y3,x4,y4,DME].

[0038] In this embodiment, the RBF neural network structure described in S400 is as follows: the input layer is 8-dimensional feature data, which is first mapped to a 200-dimensional hidden layer, then mapped to a 1-dimensional hidden layer, and finally mapped to the output layer. The output layer is used to output the predicted DME.

[0039] This embodiment addresses the issues of airport facility limitations preventing commercial aircraft from achieving automatic landing at most airports, as well as the high workload and risk of human error for pilots during takeoff and landing. To improve operational safety and airspace efficiency, and meet the needs of smart civil aviation development, research on fully automated takeoff and landing is conducted based on next-generation information and intelligent technologies. Breakthroughs are achieved in machine vision-based navigation technology, high-availability autonomous navigation technology based on multi-source fusion, and intelligent airport surface surveillance technology based on machine vision. The development and technical verification of an autonomous navigation and surveillance prototype are completed, providing technical support for improving the performance of domestically produced commercial aircraft.

[0040] To achieve fully automated takeoff and landing, the first problem to be solved is to enable the aircraft to accurately perceive and navigate the takeoff and landing environment. This ensures that the aircraft can autonomously monitor and guide the environment, takeoff conditions, and landing conditions using its onboard perception and navigation system during taxiing, takeoff, and landing. Key technologies such as high-accuracy real-time reliable runway target recognition and tracking technology in dynamic and complex environments and intelligent monitoring technology supporting automated takeoff and landing need to be developed. A prototype of intelligent autonomous navigation and monitoring supporting fully automated takeoff and landing should be developed and integrated and verified in a ground laboratory. This will improve the ability to make correct decisions in complex environments, reduce workload, and provide an important technical foundation for achieving single-pilot operation.

[0041] Based on the characteristic that runway images are always convex quadrilaterals and that there are spatial order constraints between the four corner points, this embodiment specifies a clockwise order and introduces a target prediction constraint mechanism based on runway image features in the YOLOv5 framework.

[0042] Furthermore, the target prediction constraint mechanism based on runway image features is the coordinate relationship of the four corner feature points A1, A2, A3, and A4 of the runway contour:

[0043]

[0044] S200, Design the prediction layer loss function L T The prediction layer loss function L Y =L GIoU +L T Among them, L GIoU To introduce the intersection-union ratio loss function of the minimum closure region, L T This is the runway profile ratio loss function.

[0045] Furthermore, the prediction layer loss function L T In this context, the intersection-union ratio (IUU) loss function for the minimum closure region is introduced as follows:

[0046]

[0047] The runway profile ratio loss function is:

[0048]

[0049] That is, the loss function of the prediction layer is:

[0050]

[0051] Among them, S R S represents the area of ​​the actual runway in the image. r S represents the predicted area of ​​the runway profile in the image. R ∩S r S represents the area of ​​the intersection of the two. R ∪S r S represents the area of ​​the union of the two sets. B (x,y) represents the area of ​​the smallest closure region in the image; (x,y) represents the corner coordinates of the predicted runway profile, and (x′,y′) represents the corresponding corner coordinates of the actual runway.

[0052] S300, train the network, detect the runway outline, and obtain the four corner points of the runway outline as feature points.

[0053] Furthermore, the four corner feature points of the runway outline are:

[0054] A1(x1,y1),A2(x2,y2),A3(x3,y3),A4(x4,y4)

[0055] S400, construct DME distance feature data, and construct a 1*9 feature vector from the runway contour feature points and the runway image DME distance information.

[0056] Furthermore, the 1*9 feature vector is as follows:

[0057] [x1,y1,x2,y2,x3,y3,x4,y4,DME].

[0058] S500, design RBF neural network structure, and train the network using constructed DME distance feature data.

[0059] Furthermore, the RBF neural network structure is as follows: the input layer is 8-dimensional feature data, which is mapped to a 200-dimensional hidden layer, then mapped to a 1-dimensional hidden layer, and finally output to obtain the predicted DME distance.

[0060] Figure 1 The diagram shown illustrates the process of this invention, a smart takeoff and landing DME measurement method, comprising the following steps:

[0061] S100, such as Figure 2 As shown, the runway image is always a convex quadrilateral, and there is a spatial order constraint between the four corner points. Therefore, clockwise order is specified. In the YOLOv5 framework, a target prediction constraint mechanism based on runway image features is introduced.

[0062] The target prediction constraint mechanism based on runway image features is the coordinate relationship of the four corner feature points A1, A2, A3, and A4 of the runway contour:

[0063]

[0064] S200, Design the prediction layer loss function L T The prediction layer loss function L Y =L GIoU +L T Among them, L GIoU To introduce the intersection-union ratio loss function of the minimum closure region, L T Let be the runway profile ratio loss function. The intersection-union ratio loss function for the minimum closure region is introduced as follows:

[0065]

[0066] The runway profile ratio loss function is:

[0067]

[0068] That is, the loss function of the prediction layer is:

[0069]

[0070] Among them, S R S represents the area of ​​the actual runway in the image. r S represents the predicted area of ​​the runway profile in the image. R ∩S r S represents the area of ​​the intersection of the two. R ∪S r S represents the area of ​​the union of the two sets. B (x,y) represents the area of ​​the smallest closure region in the image; (x,y) represents the corner coordinates of the predicted runway profile, and (x′,y′) represents the corresponding corner coordinates of the actual runway.

[0071] S300, a training network, detects the runway contour and obtains the four corner points of the runway contour as feature points. The feature points of the four corner points of the runway contour are as follows:

[0072] A1(x1,y1),A2(x2,y2),A3(x3,y3),A4(x4,y4)

[0073] S400, construct DME distance feature data, and construct a 1*9 feature vector from the runway contour feature points and the DME distance information of the runway image:

[0074] [x1,y1,x2,y2,x3,y3,x4,y4,DME].

[0075] For the S500, an RBF neural network structure was designed and trained using constructed DME distance feature data. In this implementation, 280 DME feature data points were constructed; the first 200 features were used as the training set, and the remaining 80 features were used as the test set. The RBF neural network structure is as follows: Figure 3 As shown, the input layer consists of 8-dimensional feature data, which is mapped to a 200-dimensional hidden layer, then to a 1-dimensional hidden layer, and finally to the output layer to obtain the predicted DME distance.

[0076] like Figure 4 The figures shown are schematic diagrams illustrating the training set prediction results, test set prediction results, and prediction result evaluation metrics of the DME distance predicted by the trained RBF neural network.

[0077] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A method for intelligent takeoff and landing DME measurement, characterized in that, Includes the following steps: S100: Based on runway image features, introduce a target prediction constraint mechanism based on runway image features into the YOLOv5 framework; Design the loss function for the prediction layer. Loss function of prediction layer ;in, To introduce the intersection-union ratio loss function of the minimum closure region, The runway profile ratio loss function; S200: The neural network obtained by training S100 is used to detect the runway contour, and the four corner points of the obtained runway contour are used as feature points. S300: Construct DME distance feature data, and construct 1 from the feature points and runway image DME distance information.

9. Feature data; S400: Design the RBF neural network structure and train the RBF neural network using the constructed DME distance feature data; S500: DME is measured based on the trained RBF neural network; The target prediction constraint mechanism in S100 is the feature points of the four corner points of the runway profile. Coordinate relationship: ;in: ; The prediction layer loss function L mentioned in S100 Y for: in: This represents the area of ​​the actual runway in the image. This represents the predicted area of ​​the runway profile in the image. This represents the area of ​​the intersection of the two. This represents the area of ​​the union of the two sets. This represents the area of ​​the smallest closure region in the image. This represents the coordinates of the corner points of the predicted runway profile. This represents the coordinates of the corresponding corner points of the actual runway.

2. The intelligent takeoff and landing DME measurement method according to claim 1, characterized in that, The feature data mentioned in S300 is 1. eigenvectors of 9: .

3. The intelligent takeoff and landing DME measurement method according to claim 1, characterized in that, The RBF neural network structure described in S400 is as follows: the input layer is 8-dimensional feature data, which is first mapped to a 200-dimensional hidden layer, then to a 1-dimensional hidden layer, and finally to the output layer. The output layer is used to output the predicted DME.

Images