An aircraft de-icing fluid spray trajectory planning and control method, system and device

Abstract

The present application belongs to the technical field of aircraft automatic deicing, and discloses a kind of aircraft deicing fluid spraying trajectory planning and control method, system and equipment.The method includes: host computer uses gradient calculation method and double threshold detection method to extract the target profile in the depth image;Three-dimensional space coordinate system is constructed, and the depth feature of preselected deicing area is preselected, and the position coordinates of the preselected deicing area are obtained based on the depth feature;Drive the spraying device to move to the initial center position of preselected deicing area, and the host computer further adjusts the deicing fluid spraying angle and spraying flow of the spray head through closed-loop control;Real-time path planning is carried out by the depth image obtained by dTOF sensor, and the position coordinates of the next preselected deicing area are determined until the deicing is completed.The present application utilizes the characteristics of high imaging frame rate and strong integration of dTOF depth perception technology, establishes a closed-loop feedback channel for deicing fluid spraying angle and flow control, and realizes accurate control of deicing fluid landing point.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of automatic deicing of aircraft, and particularly relates to a method, system and device for planning and controlling a spraying trajectory of deicing fluid for an aircraft. BACKGROUND

[0002] Cold weather in winter can cause icing on the surface of an aircraft, increase air resistance, and damage the aerodynamic performance in flight. If the ice layer covers the wings, it will cause the wing load to be too large, thereby causing the lift to decrease sharply. If the rudder surface of the aircraft is blocked by ice and snow, the control quality of the aircraft will be reduced. The uneven weight distribution of the accumulated ice will affect the balance of the aircraft, causing the flight stability to decrease significantly. The internal power device and communication system of the aircraft can be affected by the accumulated ice and thus malfunction. As a result of accumulated ice, too many serious air disasters have occurred, and therefore, the icing of an aircraft is a major threat to civil aviation safety.

[0003] Currently, most airports use the method of spraying heated deicing fluid containing ethylene glycol or propylene glycol to deice aircraft. Usually, a deicing vehicle is manually driven and the spray head is manipulated to spray deicing fluid on the surface of the aircraft.

[0004] Through the above analysis, the problems and defects of the prior art are that the existing technology has low deicing efficiency, such as not timely deicing, deicing for too long, and not completely deicing. The existing technology has low accuracy in calculating the angle and flow rate of the spraying device, resulting in poor accuracy control effect of the deicing fluid drop point. Moreover, the existing technology has poor real-time monitoring of ice and snow and poor optimization effect of the spraying path, which easily causes an increase in deicing costs. SUMMARY

[0005] To overcome the problems in the related art, the present application provides a method, system and device for planning and controlling a spraying trajectory of deicing fluid for an aircraft.

[0006] The technical solution is as follows: a method for planning and controlling a spraying trajectory of deicing fluid for an aircraft, comprising the following steps:

[0007] S1, obtaining a depth image of an initial alignment area of a spraying device using a dTOF distance sensor and sending it to an upper computer, and the upper computer uses a gradient calculation method and a double threshold detection method to extract a target object contour in the depth image;

[0008] S2, constructing a three-dimensional space coordinate system, identifying depth features of a preselected deicing area in the target object contour using a convolutional neural network of a mask area, and obtaining position coordinates of the preselected deicing area based on the depth features;

[0009] S3, driving the spraying device to move to a preselected deicing area initial center position based on the obtained deicing area position coordinates, monitoring the environmental wind speed, deicing liquid spraying drop point and spraying flow based on the multi-element sensor, real-time solving the spraying device nozzle angle and spraying flow, and feeding the above information to the upper computer, and the upper computer further adjusting the deicing liquid spraying angle and spraying flow of the nozzle through closed loop control;

[0010] S4, the spraying device removes the ice and snow in the preselected deicing area using the initial path planning, and the depth image obtained by the dTOF sensor is used for real-time path planning to determine the position coordinates of the next preselected deicing area until the deicing is completed.

[0011] In step S1, the gradient calculation method comprises: calculating the gradients in the horizontal direction and the vertical direction using four gradient operators, respectively calculating the differences d x and d y in the horizontal direction and the vertical direction, determining the gradient G' and the direction θ of the pixel point; and the expression of the four gradient operators is:

[0012]

[0013]

[0014] d x = f(x, y) * Soble x (x, y)

[0015] d y = f(x, y) * Soble y (x, y)

[0016] The gradient G' calculation formula is:

[0017]

[0018] The direction θ calculation formula is:

[0019]

[0020] In the formula, f(x, y) is the partial derivative of a certain pixel point in the horizontal direction and the vertical direction, Soble X is the gradient of the depth image in the horizontal direction, Soble Y is the gradient of the depth image in the vertical direction, Soble Y (x, y) is the gradient of a certain pixel point in the horizontal direction, Soble X (x, y) is the gradient of a certain pixel point in the vertical direction, and M(x, y) is the gradient value of a certain pixel point.

[0021] The double threshold detection method comprises: screening the image screened by using the gradient calculation method by using a high threshold with a size ratio of 2:1 and a low threshold which is 1 / 3-1 / 2 times of the high threshold, and screening the gradient value of the edge pixel point; if the gradient value of the edge pixel point is greater than the high threshold, the image edge point is retained, the point greater than the low threshold and less than the high threshold is a fuzzy edge pixel point, and if there is, it is retained, and if not, it is filtered out; the edge pixel point with a gradient value less than the low threshold is filtered out; and the retained pixel points are connected to obtain the target object contour in the depth image, wherein the target object contour is an ice and snow coverage region contour.

[0022] In step S2, the three-dimensional space coordinate system is a world coordinate system, which comprises three-dimensional information of image coordinate system, pixel coordinate system and camera coordinate system conversion in the depth image.

[0023] In step S3, the multi-element sensor comprises an anemometer for measuring wind speed and wind direction, a flowmeter for measuring the spraying flow of the deicing liquid, and an infrared temperature sensor for measuring the spraying point of the deicing liquid; the closed loop control utilizes the difference between the preselected deicing area and the deicing liquid point coordinates to perform PID algorithm on the spraying flow and the spraying angle of the deicing liquid.

[0024] In step S4, the spraying device removes the ice and snow in the preselected deicing area by using the initial path planning, and specifically comprises the following steps:

[0025] The coordinates of the preselected deicing areas are connected with each other in a straight line, and the intersection between the connection lines is defined as a node of the spraying path; the spraying path between the nodes is defined as an edge, and each edge is described by a weight value to describe the complexity of the spraying path, and the expression is:

[0026] G=(V,E,W,P)

[0027] V={(V i |i=1,2…n)}

[0028] E={<V i ,V i >|V i ,v i ∈V}

[0029] W={W ij |<V i ,V j >∈E}

[0030] P={P ijk |<V i ,V j >,<v i ,v k >∈E}

[0031] Wherein, G represents a spraying path network; V represents a set of spraying path connection intersection nodes, V i represents a set of i spraying path connection intersection nodes, v i is a node with i quantity; V j is a node with j quantity, E represents an edge in the spraying path near the node, and the edge in the actual spraying path network is an attribute with two directions back and forth, represented as i , V j >; W represents the weight of the road segment, including the spraying path length and the spraying point moving time; W ij represents the weight of the road segment from i to j nodes, P represents the path node turning restriction; P ijk represents the speed limit value of the spraying path from i to j nodes at the turning node k, and k represents a certain value in the turning node; v k represents the speed of the k turning node.

[0032] Determine the position coordinates of the next preselected deicing area, comprising the following steps:

[0033] Step 1, obtain the speed of the spraying drop moving The length L of the path where the current moves i , the value of the given parameter a, p is taken

[0034] Step 2, initialize the road weight Solve the shortest spraying path to move to the last ice and snow covered coordinate in the current depth image and start driving according to the path;

[0035] Step 3, after a time interval T, obtain the average spraying drop moving speed Update the path weight according to the following formula;

[0036]

[0037]

[0038]

[0039] In the formula, is a threshold speed, a k is a weight factor, which is positive; from the above formula, is positive, and the spraying path weight only increases and does not decrease; when the average speed of the spraying drop is less than the threshold speed, update the weight, otherwise do not update, represents the weight of the k+1 turning node in the i road segment, represents the angular velocity of the k turning node in the i road segment, represents the weight of the k turning node in the i road segment, represents the weight of the initial turning node in the i-th section; L i Current path length where the mobile is located;

[0040] Step 4: According to the updated road weight and the new position information of the spraying point, the shortest spraying path is solved until the ice and snow in the current target area are removed.

[0041] The depth image obtained by the dTOF sensor is used for real-time path planning, including: when the ice and snow coverage position of the newly selected deicing area is preselected in the depth image detected by the dTOF sensor, the deicing path of the newly selected deicing area is reselected by using the initial path planning.

[0042] Another object of the present application is to provide an aircraft deicing liquid spraying trajectory planning and control system for implementing the aircraft deicing liquid spraying trajectory planning and control method, which comprises:

[0043] A target profile extraction module obtains a depth image of an initial alignment area of the spraying device by using a dTOF distance sensor and sends the depth image to an upper computer, and the upper computer extracts a target profile in the depth image by using a gradient calculation method and a double threshold detection method;

[0044] A preselected deicing area position coordinate acquisition module constructs a three-dimensional space coordinate system, identifies a depth feature of a preselected deicing area in a target profile by using a convolutional neural network of a mask area, and acquires a position coordinate of the preselected deicing area based on the depth feature;

[0045] A deicing liquid spraying angle and spraying flow adjustment module drives the spraying device to move to an initial center position of the preselected deicing area based on the acquired deicing area position coordinate, real-time calculates a spraying head angle and a spraying flow of the spraying device based on the monitoring of environmental wind speed, deicing liquid spraying point and spraying flow by a multi-element sensor, and feeds back the above information to the upper computer, and the upper computer further adjusts the deicing liquid spraying angle and spraying flow of the spraying head by closed-loop control.

[0046] A real-time path planning module removes ice and snow in a preselected deicing area by using initial path planning, and real-time path planning is performed by using a depth image obtained by a dTOF sensor to determine a position coordinate of a next preselected deicing area until deicing is completed.

[0047] Another object of the present application is to provide a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the aircraft deicing liquid spraying trajectory planning and control method.

[0048] In combination with all the above technical solutions, the present application has the following advantages and positive effects:

[0049] (1) The dTOF sensor is used to obtain depth information in real time, and the gradient calculation and double threshold detection are combined to realize target contour imaging. Compared with general image acquisition devices, the cost is reduced, the device volume is reduced, and the response speed is improved. The spraying trajectory is controlled in real time based on the Mask R CNN feature recognition network and the path planning algorithm, so that the use of deicing liquid is maximized, and the deicing efficiency of the device is significantly improved. The control of the spraying point is realized by the multi-element sensor in real time, which ensures the accurate spraying of the deicing liquid in severe weather environment and reduces the environmental pollution caused by the large spraying of the deicing liquid.

[0050] (2) Compared with the prior art, the advantages of the present application further include: when the aircraft deicing operation is carried out in China, a special deicing operator is needed to carry out high-altitude manual operation on the deicing vehicle, which is easily affected by the severe environment and has high danger. Manual operation during the peak period of flights also easily leads to low deicing efficiency and flight delays. Therefore, the aircraft deicing liquid spraying trajectory planning and control method provided by the present application improves the winter support efficiency of the airport and improves the safety of the deicing work.

[0051] (3) The aircraft deicing liquid spraying trajectory planning and control method provided by the present application has the characteristics of high imaging frame rate and strong integration of dTOF depth perception technology, establishes a closed-loop feedback channel for the deicing liquid spraying angle and flow control, and realizes accurate control of the deicing liquid drop point. The steps include: based on the dTOF depth information, the gradient calculation and double threshold detection are combined to realize the target contour imaging; based on the Mask R CNN feature recognition network, the preselected deicing area is locked; based on the dTOF three-dimensional information, the spatial coordinate system is constructed, and the spraying device is driven to move to the contour center point; based on the dTOF depth information, the ice and snow are preliminarily detected and the initial spraying path is planned; based on the multi-element sensor data, the nozzle angle and the spraying flow are calculated to realize accurate control of the deicing liquid drop point; based on the dTOF depth information, the ice and snow are monitored in real time and the spraying path is optimized.

[0052] (4) The technical scheme of the present application solves the technical problems in the design of the intelligent and automatic deicing scheme of the aircraft, improves the deicing efficiency of the aircraft, reduces the cost of the deicing device, and provides a technical basis for the development of the future airport automatic deicing system; the technical scheme of the present application fills the technical gap of the automatic deicing of the aircraft at home and abroad, and provides a reference for the intelligentization of the future airport. BRIEF DESCRIPTION OF DRAWINGS

[0053] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure.

[0054] Figure 1This is a flowchart of the aircraft de-icing fluid spray trajectory planning and control method provided in the embodiments of the present invention;

[0055] Figure 2 This invention relates to the principle of aircraft de-icing fluid spray trajectory planning and control method provided in this embodiment.

[0056] Figure 3 This is a schematic diagram of the operation of the dTOF depth sensor provided in the embodiment of the present invention;

[0057] Figure 4 This is a flowchart illustrating the control method steps used in the embodiments of the present invention;

[0058] Figure 5 This is a schematic diagram of an aircraft de-icing fluid spray trajectory planning and control system provided in an embodiment of the present invention;

[0059] In the diagram: 1. Target outline extraction module; 2. Pre-selected de-icing zone location coordinate acquisition module; 3. De-icing fluid spray angle and spray flow rate adjustment module; 4. Real-time path planning module. Detailed Implementation

[0060] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0061] Example 1, as Figure 1 As shown, an embodiment of the present invention provides a method for planning and controlling the spray trajectory of aircraft de-icing fluid, comprising the following steps:

[0062] S1. The depth image of the initial alignment area of ​​the spraying device is obtained using a dTOF distance sensor and sent to the host computer. The host computer uses gradient calculation method and dual threshold detection method to extract the contour of the target object in the depth image.

[0063] S2, construct a three-dimensional spatial coordinate system, use the convolutional neural network of the mask region to identify the depth features of the pre-selected de-icing zone in the outline of the target object, and obtain the position coordinates of the pre-selected de-icing zone based on the depth features;

[0064] S3, based on the obtained de-icing zone location coordinates, drive the spraying device to move to the initial center position of the pre-selected de-icing zone, monitor the ambient wind speed, de-icing fluid spray landing point, and spray flow rate based on multiple sensors, calculate the nozzle angle and spray flow rate of the spraying device in real time, and feed the above information back to the host computer. The host computer further adjusts the de-icing fluid spray angle and spray flow rate of the nozzle through closed-loop control.

[0065] S4, the spraying device removes ice and snow in the preselected deicing area using the initial path planning, and real-time path planning is performed based on the depth image obtained by the dTOF sensor to determine the position coordinates of the next preselected deicing area until the deicing is completed.

[0066] Further, in step S1, the gradient calculation method comprises: calculating the gradients in the horizontal direction and the vertical direction using four gradient operators, respectively calculating the differences d x and d y in the horizontal direction and the vertical direction, and determining the gradient G' and the direction θ of the pixel point; and the expression of the four gradient operators is:

[0067]

[0068]

[0069] d x = f(x, y) * Sobel x (x, y)

[0070] d y = f(x, y) * Sobel y (x, y)

[0071] The gradient G' calculation formula is:

[0072]

[0073] The direction θ calculation formula is:

[0074]

[0075] In the formula, f(x, y) is the partial derivative of a pixel point in the horizontal direction and the vertical direction, Sobel x is the gradient of the depth image in the horizontal direction, Sobel Y is the gradient of the depth image in the vertical direction, Sobel Y (x, y) is the gradient of a pixel point in the horizontal direction, Sobel x (x, y) is the gradient of a pixel point in the vertical direction, and M(x, y) is the gradient value of a pixel point.

[0076] Further, in step S1, the double threshold detection method comprises: using a high threshold and a low threshold with a size ratio of 2:1 and 1 / 3-1 / 2 times of the high threshold to screen the edge pixel point gradient value of the image screened by using the gradient calculation method; if the edge pixel point gradient value is greater than the high threshold, the image edge point is retained, the point greater than the low threshold and less than the high threshold is a fuzzy edge pixel point, if there is, it is retained, if not, it is filtered out; the edge pixel point with a gradient value less than the low threshold is filtered out; connecting the retained pixel points obtains the target object contour in the depth image, wherein the target object contour is the ice and snow covered area contour.

[0077] Further, in step S2, the three-dimensional space coordinate system is a world coordinate system, which comprises three-dimensional information of image coordinate system, pixel coordinate system and camera coordinate system conversion in the depth image.

[0078] In step S3, the multi-element sensor comprises an anemometer for measuring wind speed and wind direction, a flowmeter for measuring the deicing liquid spraying flow, and an infrared temperature sensor for measuring the deicing liquid spraying drop point; the closed loop control uses the difference between the preselected deicing area and the deicing liquid drop point coordinate to perform PID algorithm on the spraying flow and the deicing liquid spraying angle.

[0079] Further, in step S4, the spraying device removes the ice and snow in the preselected deicing area by using the initial path planning, which specifically comprises the following steps:

[0080] The coordinates of the preselected deicing areas are connected with each other in a straight line, and the intersection between the connection lines is defined as the node of the spraying path; the spraying path between the nodes is defined as the edge, and each edge is described by a weight value to describe the complexity of the spraying path, and the expression is:

[0081] G=(V,E,W,P)

[0082] V={(V i |i=1,2…n)}

[0083] E={<V i ,V i >|V i ,v i ∈V}

[0084] W={W ij |<V i ,V j >∈E}

[0085] P={P ijk |<V i ,V j >,<v i ,v k >∈E}

[0086] wherein G represents a spraying path network; V represents a set of spraying path link intersection nodes, V i represents a set of i spraying path link intersection nodes, v i is a node with i quantity; V j is a node with j quantity, E represents an edge in the spraying path near the node, the edge in the actual spraying path network is an attribute with two directions back and forth, represented as <V i , V j >; W represents the weight of the road segment, including the spraying path length and the spraying point moving time; W ij represents the weight of the road segment from i to j nodes, P represents the path node turning restriction; P ijk represents the speed limit value of the spraying path from i to j nodes at the turning node k, k represents a certain value in the turning node; v k represents the speed of k turning nodes.

[0087] Further, in step S4, the position coordinates of the next preselected deicing area are determined, including the following steps:

[0088] Step 1, obtain the speed of the spraying drop moving the length L of the path currently moving in i , the value of the given parameter a, p is taken as

[0089] Step 2, initialize the road weight solve the shortest spraying path to move to the last ice and snow coverage coordinate in the current depth image and start driving according to the path;

[0090] Step 3, after a time interval T, obtain the average spraying drop moving speed update the path weight according to the following formula;

[0091]

[0092]

[0093]

[0094] wherein, is a threshold speed, a k is a weight factor, which is positive; from the above formula, is positive, and the spraying path weight only increases and does not decrease; when the average speed of the spraying drop is less than the threshold speed, the weight is updated, otherwise it is not updated, represents the weight of the k+1 turning node in the i road segment, represents the angular speed of the k turning node in the i road segment, a weight value of an initial turning node in the i-th path segment; L a weight value of an initial turning node in the i-th path segment; L i a current path length of the moving;

[0095] Step 4: according to the updated road weight value and the new position information of the spraying point, solve the shortest spraying path until the ice and snow in the current target area is removed.

[0096] Further, in step S4, the depth image obtained by the dTOF sensor is used for real-time path planning, including: when the ice and snow coverage position of the newly selected deicing area is detected in the depth image detected by the dTOF sensor, the deicing path of the newly selected deicing area is reselected by using the initial path planning.

[0097] Embodiment 2: As another embodiment of the present application, the embodiment of the present application provides an aircraft deicing liquid spraying trajectory planning and control method comprising the following steps:

[0098] S101: based on dTOF (Direct time of flight) depth information, gradient calculation and double threshold detection are combined to realize target object contour imaging;

[0099] S102: based on Mask R CNN (Mask Region-based Convolutional Neural Network), a preselected deicing area is locked;

[0100] S103: a spatial coordinate system is constructed, and the spraying device is driven to move to the contour center point;

[0101] S104: based on dTOF depth information, ice and snow preliminary detection and initial spraying path planning are realized;

[0102] S105: based on multi-element sensor data monitoring environment wind speed, deicing liquid spraying point, and spraying flow, the spraying device angle and spraying flow are solved in real time, and deicing liquid drop point accurate control is realized;

[0103] S106: based on dTOF depth information, ice and snow coverage position is monitored and locked, and real-time spraying path is planned.

[0104] Further, in step S103, the spraying device includes a deicing liquid spray head with adjustable spraying angle and speed.

[0105] Further, in step S103, the contour center point is the center of the preselected deicing area in the spatial coordinate system.

[0106] Further, in step S104, the preliminary ice and snow detection uses the dTOF depth information to detect the area with abnormal depth information in the preselected deicing area as the ice and snow covered area.

[0107] Further, in step S106, for the movement of the spray head, the depth image position detected by the dTOF sensor is also moving, and new ice and snow covered positions may appear in the image, so the path needs to be re-planned.

[0108] In embodiment 3, as another possible implementation, the aircraft deicing liquid spray trajectory planning and control method provided by the embodiment of the application constructs a closed loop feedback channel with the dTOF depth perception device as the core, realizes real-time planning and accurate control of the deicing liquid spray trajectory through deviation control, and the depth information is the depth image detected by the dTOF depth sensor.

[0109] The pixel gradient is the change degree of the gray value of the edge feature. Therefore, the difference value can be used to represent the discrete data of the image, and the difference is the gray difference, that is, the difference of two pixels. The gradient calculation method is to use the Sobel operator to calculate the gradient.

[0110] The double threshold detection distinguishes the contour pixel points in the depth image detected by the dTOF depth sensor by setting a high threshold and a low threshold. The edge pixel gradient greater than the high threshold is retained, the pixel between the high threshold and the low threshold is considered as a connected region, and the pixel less than the low threshold is considered as noise and removed, wherein the ratio of the low threshold to the high threshold is 2:1. Finally, it is determined that the gradient pixel is in the threshold interval. If there is a pixel point with a gradient higher than the high threshold point in the connected pixel region of the edge pixel with a gradient between the high threshold and the low threshold, it is considered as a real contour pixel.

[0111] In the preferred embodiment of the application, Mask R-CNN is a target feature recognition algorithm. First, a feature map of the depth edge image obtained by the dTOF sensor is extracted based on a shared convolutional network; second, a target candidate frame is extracted based on a region generation network, the original image is taken as the input, and a series of target candidate frames are output, and then the feature map obtained in the previous step is mapped to two parallel convolutional layers through a sliding window. One of the convolutional layers corrects the candidate frame, and the other convolutional layer scores the foreground and background of the candidate region; finally, the candidate frame feature map is fixed based on the target region alignment network, a fixed size candidate frame feature map is obtained, the features in the image are extracted, and the position area of the image on the aircraft is recognized. The Mask R-CNN feature recognition network can accurately recognize the aircraft part in the pre-spraying area of the current spray head by using a large number of aircraft external depth edge images as a training set, so as to ensure the cooperative work of multiple deicing spray heads in different aircraft parts.

[0112] The contour center point is the initial position of the positioning of the spraying device, after the deicing area is identified by the Mask R-CNN, the deicing device is positioned to the center point of the contour outside the deicing area, and the contour center point is taken as the initial position of the deicing trajectory.

[0113] The spraying device is a deicing liquid spraying head with adjustable spraying angle and speed, the dTOF depth sensor and the spraying head are fixed on the same working plane and the detection direction and the spraying head direction are kept parallel, so as to ensure that the obtained depth image is updated in real time along with the working of the spraying head.

[0114] The ice and snow preliminary detection utilizes the dTOF depth information, detects the area with abnormal depth information in the preselected deicing area as the ice and snow covered area, identifies the ice and snow covered area in the initial depth information and adds it to the path list, determines the next ice and snow covered area with the shortest moving path through the relaxation operation on the surrounding ice and snow covered areas, and updates the depth information and the path list, and plans the deicing path in real time.

[0115] The shortest spraying path is solved according to the updated drop point moving path weight and the current new position coordinates of the drop point.

[0116] In the preferred embodiment of the present application, the drop point accurate control utilizes the difference between the ice and snow covered area position coordinates identified in the depth information and the deicing liquid drop point coordinates collected by the temperature sensor to perform closed-loop control on the deicing liquid spraying flow and angle through the PID algorithm, so as to realize the accurate coincidence of the deicing liquid spraying drop point and the ice and snow covered area.

[0117] In the preferred embodiment of the present application, the real-time monitoring judges whether the current ice and snow covered area has been cleared in real time through the dTOF depth information.

[0118] In Embodiment 4, as another possible implementation manner, as shown in Figure 2 the present application provides a deicing liquid spraying trajectory planning and control method based on the dTOF technology, which comprises the following steps:

[0119] (1) based on the dTOF depth information, gradient calculation and double threshold detection are combined to realize the contour imaging of the target object;

[0120] (2) a large number of aircraft surface depth pictures are used as a training set to train the Mask R CNN feature recognition network, so as to identify whether the current deicing area is the preset deicing part of the aircraft;

[0121] (3) based on the dTOF three-dimensional information, a space coordinate system is constructed to drive the spraying device to move to the contour center point;

[0122] (4) based on the dTOF depth information, ice and snow preliminary detection is realized and an initial spraying path is planned;

[0123] (5) Based on the multi-sensor data, the spraying device angle and the spraying flow are calculated to realize accurate control of the deicing liquid drop point.

[0124] (6) Based on the dTOF depth information, real-time ice and snow monitoring and spraying path optimization are realized.

[0125] As shown in Figure 3 , the dTOF depth sensor obtains the depth information of the deicing area in real time to identify the preselected deicing area and the ice and snow coverage position.

[0126] As shown in Figure 4 , the dTOF depth sensor obtains the depth information of the preselected deicing area, extracts the external contour of the preselected deicing area of the aircraft in the depth information, and identifies whether the preselected deicing area and the set area are consistent through the Mask R-CNN feature recognition network. The contour center point is located, the spraying device is driven to change the spraying angle, the nozzle position is aligned with the contour center point, and the initial speed of the deicing liquid is calculated according to the nozzle angle, the spraying distance and the wind speed detected by the anemometer. The deicing liquid spraying drop point is detected by the infrared temperature sensor, the drop point coordinates are compared with the ice and snow coverage area coordinates in the depth information, the spraying angle and the spraying flow are changed to form a closed loop control, and the control algorithm adopts the PID algorithm. The depth information obtained by the dTOF sensor changes in real time with the change of the spraying device angle, and the ice and snow coverage area in the real-time depth information obtained at present is located and the spraying path is planned.

[0127] Whether the current ice and snow coverage area is completely removed is determined by the real-time depth information, and if it is completely removed, the spraying angle and the flow of the spraying device are driven to move the drop point to the next deicing area in the planned path.

[0128] Embodiment 5, as shown in Figure 5 , the present application provides a kind of aircraft deicing liquid spraying trajectory planning and control system, comprising:

[0129] Target contour extraction module 1 is used to obtain the depth image of the initial alignment area of the spraying device by using dTOF distance sensor, and sends to host computer, and the host computer uses gradient calculation method and double threshold detection method to extract the target contour in the depth image;

[0130] Preselected deicing area position coordinate acquisition module 2 is used to construct three-dimensional space coordinate system, utilize the convolutional neural network of mask area to identify the depth feature of preselected deicing area in the target contour, and obtain the position coordinates of the preselected deicing area based on the depth feature;

[0131] The deicing liquid spraying angle and spraying flow adjustment module 3 is used for driving the spraying device to move to a preselected deicing area initial center position based on the obtained deicing area position coordinates, monitoring the environmental wind speed, the deicing liquid spraying drop point and the spraying flow based on the multi-element sensor data, calculating the spraying device spraying head angle and spraying flow in real time, and feeding back the above information to the upper computer, and the upper computer further adjusts the deicing liquid spraying angle and spraying flow of the spraying head through closed loop control.

[0132] The real-time path planning module 4 is used for the spraying device to remove the ice and snow in the preselected deicing area by using the initial path planning, to perform real-time path planning by using the depth image obtained by the dTOF sensor, to determine the position coordinates of the next preselected deicing area, and to determine the position coordinates of the next preselected deicing area until the deicing is completed.

[0133] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in a certain embodiment can be referred to the related description of other embodiments.

[0134] The information interaction and execution process between the above devices / units are based on the same concept as the method embodiments of the application, and the specific functions and brought technical effects can be referred to the method embodiments part, which will not be repeated here.

[0135] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is taken as an example, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or software. In addition, the specific name of each functional unit and module is only for easy distinction, and does not limit the protection scope of the application. The specific working process of the unit and module in the system can be referred to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0136] The embodiment of the application further provides a computer device, which comprises at least one processor, a memory and a computer program stored in the memory and executable on the at least one processor, and the processor implements the steps in any one of the above aircraft deicing liquid spraying trajectory planning and control system method embodiments when executing the computer program.

[0137] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program can realize the steps in each of the aircraft deicing liquid spraying trajectory planning and control system method embodiments above when executed by a processor.

[0138] The embodiment of the present application also provides an information data processing terminal, which is used to provide a user input interface to implement the steps in each of the aircraft deicing liquid spraying trajectory planning and control system method embodiments above when executed on an electronic device, and the information data processing terminal is not limited to a mobile phone, a computer or a switch.

[0139] The embodiment of the present application also provides a server, which is used to provide a user input interface to implement the steps in each of the aircraft deicing liquid spraying trajectory planning and control system method embodiments above when executed on an electronic device.

[0140] The embodiment of the present application provides a computer program product, which makes an electronic device execute the steps in each of the aircraft deicing liquid spraying trajectory planning and control system method embodiments above when the computer program product is run on the electronic device.

[0141] The integrated unit, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the present application can realize all or part of the processes in the above-mentioned embodiments by a computer program to instruct related hardware, and the computer program can be stored in a computer readable storage medium. The computer program can realize the steps in each of the method embodiments above when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium at least includes any entity or device capable of carrying the computer program code to a photographing device / terminal equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal and a software distribution medium. For example, a U disk, a mobile hard disk, a magnetic disk or an optical disk.

[0142] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any modification, equivalent replacement and improvement within the technical range disclosed by the present application and within the spirit and principle of the present application should be covered within the protection scope of the present application.

Claims

1. A method for planning and controlling the spray trajectory of aircraft de-icing fluid, characterized in that, The method The method comprises the following steps: S1, acquiring a depth image of an initial alignment area of the spraying device by using a dTOF distance sensor and sending the depth image to a host computer, and the host computer extracting a target object contour in the depth image by using a gradient calculation method and a double-threshold detection method; S2, constructing a three-dimensional space coordinate system, recognizing a depth feature of a preselected deicing area in the target object contour by using a convolutional neural network of a mask area, and acquiring a position coordinate of the preselected deicing area based on the depth feature; S3, driving the spraying device to move to an initial center position of the preselected deicing area based on the acquired position coordinate of the deicing area, monitoring an environmental wind speed, a deicing liquid spraying drop point, and a spraying flow rate by using a multi-element sensor, real-time calculating a spraying head angle and a spraying flow rate of the spraying device, and feeding back the information to the host computer, and the host computer further adjusting the deicing liquid spraying angle and the spraying flow rate of the spraying head by using a closed-loop control; S4, removing ice and snow in the preselected deicing area by using an initial path planning by the spraying device, acquiring a depth image by using the dTOF sensor, and real-time path planning, and determining a position coordinate of a next preselected deicing area until deicing is completed; In step S4, the position coordinate of the next preselected deicing area is determined, comprising the following steps: Step 1, acquire the speed of the spray drop movement the length L of the path currently being moved i , the values of the given parameters a, p, take Step 2, initialize road weight Solve the shortest spraying path moving to the coordinate of the last ice and snow coverage in the current depth image and start driving along the path; Step 3, after time interval T, average spray drop movement speed is taken The path weight is updated as follows; wherein, is a threshold speed, a k is a weight factor, positive; from the above equation, is positive, the weight of the spraying path only increases and does not decrease; when the average speed of the spraying point is less than the threshold speed, the weight is updated, otherwise it is not updated, denotes the weight of the k+1 steering node in the i section, denotes the angular velocity of the k steering node in the i section, denotes the weight of the k steering node in the i section, denotes the weight of the initial steering node in the i section; L i the length of the path where the current movement is located; Step 4, solving a spraying shortest path according to the updated road weight value and new position information of the spraying drop point until ice and snow in a current target area are removed.

2. The method of claim 1, wherein, In step S1, the gradient calculation method comprises: using four gradient operators to calculate the gradients in horizontal and vertical directions, respectively calculating the differences d in horizontal and vertical directions x and d y , determining the gradient G' and the direction θ of the pixel point; and the expression of the four gradient operators is: d x = f(x, y) x Sobel x (x, y) d y = f(x, y) x Sobel y (x, y) Gradient G ′ The calculation formula is: A direction θ calculation formula is: where f(x, y) is the partial derivative of a certain pixel point in the horizontal direction and the vertical direction, Sobel X is the gradient of the depth image in the horizontal direction, Sobel Y is the gradient of the depth image in the vertical direction, Sobel Y (x, y) is the gradient of a certain pixel point in the horizontal direction, Sobel X (x, y) is the gradient of a certain pixel point in the vertical direction, and M(x, y) is the gradient value of a certain pixel point.

3. The method of claim 1, wherein, In step S1, the double-threshold detection method comprises the following steps: using a high threshold with a size ratio of 2:1 and a low threshold which is 1 / 3-1 / 2 times of the high threshold to screen edge pixel point gradient values of an image screened by using the gradient calculation method; if the edge pixel point gradient value is greater than the high threshold, the image edge point is reserved, the points greater than the low threshold and less than the high threshold are fuzzy edge pixel points, and if there are the points, the points are reserved, and if there are not the points, the points are filtered out; the edge pixel points with the gradient values less than the low threshold are filtered out; and the reserved pixel points are connected to obtain the target object contour in the depth image, wherein the target object contour is an ice and snow covered area contour.

4. The method of claim 1, wherein, In step S2, the three-dimensional space coordinate system is a world coordinate system, and the world coordinate system comprises three-dimensional information converted from an image coordinate system, a pixel coordinate system, and a camera coordinate system in the depth image.

5. The method of claim 1, wherein, In step S3, the multi-element sensor comprises an anemometer for measuring a wind speed and a wind direction, a flowmeter for measuring a deicing liquid spraying flow rate, and an infrared temperature sensor for measuring a deicing liquid spraying drop point; and the closed-loop control uses a difference between the preselected deicing area and the deicing liquid drop point coordinate to perform PID algorithm on the spraying flow rate and the deicing liquid spraying angle.

6. The method of claim 1, wherein, In step S4, the spraying device removes ice and snow in the preselected deicing area by using the initial path planning, and specifically comprises the following steps: The coordinates of the preselected deicing areas are connected to each other in a straight line, an intersection between the connecting lines is defined as a node of the spraying path, a spraying path between the nodes is defined as an edge, and each edge is described by a weight value to describe a complexity of the spraying path, and an expression is: G=(V,E,W,P) V = {(V i |i = 1, 2...n} E = { < V i , V i > | V i , v i ∈ V} W = { W ij | < V i , B j > ∈ E} P = {P ijk | <V i , V j}, <v i , v k > E} Wherein, G represents the spraying path network; V represents the set of spraying path connection intersection nodes, V i represents the set of i spraying path connection intersection nodes, v i is the node with i quantity; V j is the node with j quantity, E represents the edge in the spraying path near the node, the edge in the actual spraying path network is the attribute with two directions back and forth, and is represented as <V i ,V j >; W represents the weight of the road section, including the spraying path length and the spraying point moving time; W ij represents the weight of the road section from i to j nodes, P represents the path node turning restriction; P ijk represents the speed limit value of the spraying path from i to j nodes in the turning node k, and k represents a certain value in the turning node; v k represents the speed of k turning nodes.

7. The method of claim 6, wherein, In step S4, the depth image obtained by the dTOF sensor is used for real-time path planning, including: when the ice and snow coverage position of the newly preselected deicing area is detected in the depth image detected by the dTOF sensor, the deicing path of the newly preselected deicing area is re-planned using the initial path planning.

8. An aircraft de-icing fluid spray trajectory planning and control system, characterized in that, The aircraft deicing liquid spraying trajectory planning and control method of any one of claims 1-7, the system comprises: A target object contour extraction module (1) obtains a depth image of an initial alignment area of the spraying device using a dTOF distance sensor and sends the depth image to a host computer, and the host computer extracts a target object contour in the depth image using a gradient calculation method and a double threshold detection method; A preselected deicing area position coordinate acquisition module (2) constructs a three-dimensional space coordinate system, identifies a depth feature of a preselected deicing area in the target object contour using a convolutional neural network of a mask area, and acquires a position coordinate of the preselected deicing area based on the depth feature; A deicing liquid spraying angle and spraying flow adjustment module (3) drives the spraying device to move to an initial center position of the preselected deicing area based on the acquired position coordinate of the deicing area, monitors an environmental wind speed, a deicing liquid spraying drop point, and a spraying flow based on a multi-element sensor, real-time calculates a spraying head angle and a spraying flow of the spraying device, and feeds back the information to the host computer, and the host computer further adjusts the deicing liquid spraying angle and the spraying flow of the spraying head through closed-loop control; A real-time path planning module (4) removes ice and snow of the preselected deicing area using the initial path planning, and uses the depth image obtained by the dTOF sensor for real-time path planning to determine a position coordinate of a next preselected deicing area until deicing is completed.

9. A computer device, comprising: The computer device comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the aircraft deicing liquid spraying trajectory planning and control method of any one of claims 1-7.

Images