Remote flight control method and system for aircraft

Abstract

The present invention relates to a remote flight control method and system for an aircraft, the method comprising: using a first image sensor and a second image sensor disposed on the aircraft to respectively capture in real time a video consisting of a video frame stream during flight; dividing the video frames corresponding to the same time captured by different image sensors into three different regions; and making control decisions based on the area ratio of each region. The present invention constructs a first sequence of selected feature points or a second sequence of selected feature points for alternating region construction using the video frame streams captured by different image sensors, thereby completing region division and image positioning with a high degree of redundancy, while fully utilizing the timing information provided by the video stream information to make real-time and rapid decisions, thereby improving flight control redundancy and robustness and reducing flight control computational overhead.

Description

Technical field

[0001] The present invention belongs to the technical field of flight control, and in particular relates to a remote flight control method and system for an aircraft. [Background Technology]

[0002] The flight control system, abbreviated as FC, can be considered the brain of an aircraft. A multirotor's flight, hovering, and attitude changes are controlled by various sensors that transmit attitude data back to the FC. The FC then calculates and determines commands, and the actuators carry out the maneuvers and flight attitude adjustments. This system involves sensor technology, communications technology, information processing technology, intelligent control technology, and aerospace propulsion technology, representing a high-tech product of the information age. Its applications are rapidly expanding, including aerial photography, package delivery, agriculture, policing and surveillance, infrastructure inspection, and scientific research, by maximizing production, reducing costs and risks, and ensuring on-site safety, confidentiality, and regulatory compliance. Aerial vehicles eliminate the threat to human life while retaining many of the advantages of manned aircraft. In particular, small sensor components can gather the necessary information for both reconnaissance and surveillance, as well as communications. Aerial vehicles often incorporate popular sensor components, including optoelectronic components, forward-looking infrared (FLIR), synthetic aperture radar (SAR), and moving target indicators (MTI). More importantly, control vehicles can also be equipped with image sensors to obtain high-resolution battlefield reconnaissance and surveillance imagery. Due to the low cost and widespread availability of image sensors, this significantly reduces the cost of aerial vehicles. Furthermore, the built-in sensors in the cameras provide excellent wide-area and long-range day and night surveillance capabilities under the control of ground operators. SAR provides all-weather surveillance that is difficult to achieve with optoelectronic or FLIR technologies. While aerial vehicles are often equipped with global navigation and positioning satellite systems (GNSS) and inertial navigation systems (INS) to transmit accurate positioning information and calculate the position of fixed and moving targets within the sensor's field of view, and may also be equipped with C-band variable frequency data links and ultra-high frequency (UHF) Ku-band satellite links for controlling drones and payloads, GNSS and INS systems cannot provide environmental information about the aerial vehicle, particularly in real time.

[0003] Using image information for flight control can form redundancy with other sensor information, increasing the robustness of flight control. When using image information for flight control, since the environment targets during the flight of the aircraft are relatively complex and the target features are often few or even no obvious target objects exist, the efficiency of region division or image positioning comparison by determining the monitoring target boundary or locating feature points is low. In addition, the computing power of the aircraft itself is limited and cannot support complex image analysis. In addition, many analysis results generated during the image analysis process are useless during the flight control process. How to use image information for flight control on the aircraft itself and also use this information for real-time flight control based on the server is a technical problem to be solved. In order to solve the problems in the prior art, the present invention proposes a remote flight control method and system for aircraft. By using video frame streams collected by different image sensors, a first sequence of candidate feature points or a second sequence of candidate feature points is constructed for alternating region construction. The method can complete region division and image positioning under a large redundancy, and can fully utilize the time sequence information brought by the video stream information to make real-time and rapid decisions, thereby improving the redundancy and robustness of flight control and greatly reducing the flight control calculation overhead. [Summary of the invention]

[0004] In order to solve the above problems in the prior art, the present invention proposes a remote flight control method and system for an aircraft, the method comprising:

[0005] Step S1: a first image sensor and a second image sensor provided on the aircraft respectively capture in real time a first video consisting of a first video frame stream and a second video consisting of a second video frame stream during flight;

[0006] Step S2: Divide the video frames corresponding to the same time acquired by different image sensors into three different regions. Specifically, a first region, a second region, and a third region are respectively determined for the first video frame and the second video frame acquired by the first image sensor and the second image sensor. The first region is a fixed region, and the image region within the fixed region is the image region of the fixed target. The second region is an overlapping region, and the overlapping region is the image region of the same target acquired by different image sensors from different perspectives. The third region is other regions.

[0007] The determining of the first area specifically includes: when the image area in the video frame belongs to a fixed target, determining the image area as the first area;

[0008] The determining of the third area specifically comprises: taking a portion other than the first area and the second area as the third area;

[0009] The determining of the second area specifically includes: determining the second area by locating a combination of feature areas; specifically including the following steps:

[0010] Step S2A1: Arrange the pixels in the first video frame and the second video frame in descending order of pixel value to form a first sequence of candidate feature points and a second sequence of candidate feature points, respectively;

[0011] Step S2A2: Determine the current candidate feature point sequence; specifically, alternately use the first candidate feature point sequence or the second candidate feature point sequence as the current candidate feature point sequence, and the sequence other than the current candidate feature point sequence is called another candidate feature point sequence; select the head pixel in the current candidate feature point sequence as the positioning pixel;

[0012] Step S2A3: The pixel value of the positioning pixel is 1 c ; Place the positioning pixel in the feature area; Extend the area with the positioning pixel as the center in the video frame where the positioning pixel is located. If there is another pixel I c1 If the following condition (1) is satisfied between the locating pixel and the other pixel, and the locating pixel and the other pixel are within the 4-neighborhood range, then the other pixel is placed in the feature area; repeat this step until there is no other pixel that can be placed in the feature area; where: Tr is a preset value;

[0013]

[0014] Step S2A4: If another feature region composed of a pixel point in another candidate feature point sequence also satisfies condition (1) and the pixels in the region are all within the 4-neighborhood range, then the pixel value similarity and area similarity between the feature region and the other feature region are calculated; if the pixel value similarity and area similarity of the two are both less than the similarity threshold, then the feature region located in the first video frame is called the first feature region, and the feature region located in the second video frame is called the second region; the first feature region and the second feature region constitute feature region combination a;

[0015] Step S2A5: deleting pixels involved in the feature region or another feature region from the first sequence of candidate feature points or the second sequence of candidate feature points;

[0016] Step S2A6: If there are still unprocessed pixels in the first candidate feature point sequence or the second candidate feature point sequence, return to step S2A2;

[0017] Step S2A7: taking the smallest region containing all feature regions in each video frame as the second region;

[0018] Step S3: For each video frame, for each first image sensor and second image sensor, obtain a first proportion of the second area in the first video frame and a second proportion of the second area in the second video frame; and make a control decision based on the first proportion and the second proportion.

[0019] Furthermore, the first image sensor and the second image sensor are arranged at different positions of the aircraft.

[0020] Furthermore, only pixels with pixel values ​​greater than the average pixel value are intercepted and included in the sequence of feature points to be selected.

[0021] Furthermore, the pixel value is a grayscale value.

[0022] Furthermore, the pixel values ​​are preprocessed after obtaining the video frame stream.

[0023] Furthermore, Tr=0.1.

[0024] Furthermore, it is characterized in that the minimum area is a minimum rectangular area, an elliptical area or a circular area.

[0025] Furthermore, the pixel values ​​are normalized after acquiring the video frame stream.

[0026] A remote flight control system for an aircraft, wherein the remote flight control system is used to implement the above-mentioned remote flight control method for the aircraft.

[0027] A remote flight control server for an aircraft, characterized in that the remote flight control server for the aircraft is used to implement the above-mentioned remote flight control method for the aircraft.

[0028] The beneficial effects of the present invention include:

[0029] (1) By using the video frame streams collected by different image sensors, the first candidate feature point sequence or the second candidate feature point sequence is constructed for alternating region construction, which can complete region division and image positioning under a large redundancy, and can fully utilize the timing information brought by the video stream information, thereby making simple real-time and rapid decisions;

[0030] (2) When the control decision based on area ratio cannot meet the control decision needs in terms of dimension and accuracy, the dimension of the control information can be further expanded based on the prediction model. The coverage area sequence can effectively eliminate the influence of the wrong feature area combination. The influence of environmental factors on the acquisition of different sensors is introduced by the pixel value mean, which facilitates the rapid decision-making based on environmental factors with a certain robustness, without the need for spatial projection and positional relationship transformation between different image sensors, avoiding the introduction of erroneous image analysis information to bring more decision-making influences;

[0031] (3) By achieving a balance in computing power between the aircraft and the server through local and remote control, the aircraft can make quick decisions locally without relying on information from other sensors without increasing excessive hardware computing costs; it can also make artificial intelligence decisions with the help of the server's computing power.

Brief Description of the Drawings

[0032] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application, but do not constitute an improper limitation of the present invention. In the drawings:

[0033] Figure 1 Schematic diagram of the remote flight control method for an aircraft provided by the present invention. [Specific implementation method]

[0034] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The exemplary embodiments and descriptions are only used to explain the present invention but are not intended to limit the present invention.

[0035] The present invention proposes a remote flight control method and system for an aircraft, as shown in the attached Figure 1 As shown, the method includes the following steps:

[0036] Step S1: a first image sensor and a second image sensor disposed on the aircraft respectively capture, in real time, a first video consisting of a first video frame stream and a second video consisting of a second video frame stream during flight; the first image sensor and the second image sensor are disposed at different positions on the aircraft;

[0037] Preferably: the fixed target is an aircraft;

[0038] Preferably, the fixed target is an additional device located on the aircraft;

[0039] Preferably, the same target is a monitoring target;

[0040] Preferably, the image sensors are two or more; when there are two image sensors, the two image sensors are binocular cameras; the first image sensor and the second image sensor have intersecting and opposite viewing angles; of course, when the image sensor is expanded to multiple, the control method is similar and will not be described in detail;

[0041] Step S2: Divide the video frames corresponding to the same time acquired by different image sensors into three different regions, and construct a second region based on the combination of feature regions; specifically, determine a first region, a second region, and a third region for the first video frame and the second video frame acquired by the first image sensor and the second image sensor, respectively; the first region is a fixed region, and the image region within the fixed region is the image region of the fixed target; the second region is an overlapping region, and the overlapping region is the image region of the same target (or target range) acquired by different image sensors from different perspectives; and the third region is other regions (not the first region and the second region);

[0042] The determining of the first region specifically includes: performing image segmentation on the video frame to determine the first region; when the image region in the video frame belongs to a fixed target, determining the image region as the first region; for example, this can be performed by template comparison;

[0043] The determining of the third area specifically includes: referring to a portion other than the first area and the second area as the third area;

[0044] The determining of the second area specifically includes: determining the second area by locating a combination of feature areas; specifically including the following steps:

[0045] Step S2A1: Arrange the pixels in the first video frame and the second video frame in descending order of pixel value to form a first sequence of candidate feature points and a second sequence of candidate feature points, respectively;

[0046] Preferably: only pixels with pixel values ​​greater than the average pixel value are intercepted and included in the sequence of candidate feature points;

[0047] Preferably: only pixels whose pixel values ​​are within a specific pixel value range are intercepted and entered into the sequence of feature points to be selected;

[0048] Preferably: the pixel value is a grayscale value;

[0049] Preferably: normalize the pixel values ​​and perform other preprocessing;

[0050] Preferably: if there are multiple pixels with the same pixel value, these pixels are randomly arranged; or pixels closer to the center point of the area formed by these pixels are arranged at the front;

[0051] Step S2A2: Determine the current candidate feature point sequence; specifically, alternately use the first candidate feature point sequence or the second candidate feature point sequence as the current candidate feature point sequence, and the sequence other than the current candidate feature point sequence is called another candidate feature point sequence; select the head pixel in the current candidate feature point sequence as the positioning pixel;

[0052] Step S2A3: The pixel value of the positioning pixel is 1 c ; Place the positioning pixel in the feature area; Extend the area with the positioning pixel as the center in the video frame where the positioning pixel is located. If there is another pixel I c1 If the following condition (1) is satisfied between the locating pixel and the other pixel, and the locating pixel and the other pixel are within the range of 4 neighborhoods (8 neighborhoods), then the other pixel is placed in the feature area; repeat this step until there is no other pixel that can be placed in the feature area; where: Tr is a preset value;

[0053]

[0054] Preferred: Tr = 0.1;

[0055] Step S2A4: If another feature region composed of a pixel point in another candidate feature point sequence also satisfies condition (1) and all pixels in the region are within the 4-neighborhood (8-neighborhood) range, then the pixel value similarity and area similarity between the feature region and the other feature region are calculated; if both the pixel value similarity and area similarity of the two are less than the similarity threshold, then the feature region (referred to as the first feature region) and the other feature region (referred to as the second feature region) constitute feature region combination a;

[0056] Preferably: pixel value similarity is described by Euclidean distance; area similarity is described by the number of pixels (for example, the number of pixels is close or equal);

[0057] Preferred: Select the top A feature region combinations with the smallest pixel value similarity and area similarity coupling (coupling method is sum, product, etc.) values ​​for feature region combination screening, which can reduce the difficulty of comparison using samples for subsequent decision-making;

[0058] Preferably: the similarity threshold is a preset value;

[0059] Step S2A5: deleting pixels involved in the feature region and another feature region from the first sequence of candidate feature points or the second sequence of candidate feature points;

[0060] Step S2A6: If there are still unprocessed pixels in the first candidate feature point sequence or the second candidate feature point sequence, return to step S2A2;

[0061] Step S2A7: The smallest region containing the feature region is taken as the second region; the feature region located in the first video frame is referred to as the first feature region, and the feature region located in the second video frame is referred to as the second region; the first feature region and the second feature region constitute a feature region combination;

[0062] Preferably: the minimum area is a closed area;

[0063] Preferably, the minimum area is a minimum rectangular area, an elliptical area, a circular area, etc. Of course, the characteristic area can also be directly used as the second area; the method is similar and will not be described in detail here;

[0064] Since the environmental targets of an aircraft during flight are relatively complex, and the target features are often few or even non-existent, the efficiency of region division or image positioning comparison by determining the monitoring target boundary or locating feature points is low. Region construction by alternating the first candidate feature point sequence or the second candidate feature point sequence can complete region division and image positioning with greater redundancy.

[0065] Step S3: For each video frame, for each of the first image sensor and the second image sensor, obtaining a first proportion of the second area (or non-first area) in the first video frame and a second proportion of the second area (or non-first area) in the second video frame; making a control decision based on the first proportion and the second proportion; specifically comprising the following steps:

[0066] Step S31: Obtain the first proportion and the second proportion of the second area in the first video frame and the second video frame respectively, and form a first proportion sequence (pa1 t ) and the second proportion sequence (pa2 t ); where: t is the video frame number in the video frame stream;

[0067] Step S32: Compare the first proportion sequence with the first proportion sequence sample to obtain a first decision corresponding to the first proportion sample; compare the second proportion sequence with the second proportion sequence sample to obtain a second decision corresponding to the first proportion sample; and determine a control decision based on the first decision and the second decision;

[0068] The first proportion sequence and the first proportion sequence sample are compared to obtain the first decision corresponding to the first proportion sequence sample, specifically: the first sample similarity dec1 is calculated using the following formula (2), and the first decision corresponding to the first proportion sequence sample with the smallest first sample similarity is taken as the determined first decision; wherein: Smpa1 t is the tth first proportion sequence sample in the first proportion sequence sample column;

[0069]

[0070] The second sample similarity dec1 is calculated using the following formula (3), and the second decision corresponding to the second proportion sequence sample with the smallest second sample similarity is taken as the determined second decision; where: Smpa2 t is the tth second proportion sequence sample in the second proportion sequence sample;

[0071]

[0072] Preferably: pre-store the corresponding relationship between the first proportion sample and the first decision; and the corresponding relationship between the second proportion sample and the second decision;

[0073] Preferably, the first decision and the second decision are one or more of the angle, viewing angle, field of view, attitude, direction, etc. of the aircraft or the monitored target;

[0074] The control decision is determined based on the first decision and the second decision; specifically, when the first decision and the second decision are the same, the control decision is determined as the first decision; otherwise, the decision information corresponding to the smaller of the first sample similarity and the second sample similarity is selected as the control decision;

[0075] Alternatively, the control decision is determined based on the first decision and the second decision; specifically, when the first decision and the second decision are the same, the control decision is determined to be the first decision; otherwise, for the first image sensor and the second image sensor, a third proportion of the first area in the first video frame and a fourth proportion of the first area in the second video frame are calculated, respectively, to form a third proportion sequence (pa3) for the video frame stream. t ) and the fourth proportion sequence (pa4 t ); calculating a third proportion mean of the third proportion sequence and a fourth proportion mean of the fourth proportion sequence; and using the decision information of the video frame stream corresponding to the smaller of the third proportion mean and the fourth proportion mean as the control decision;

[0076] Preferably, the third mean of proportions and the fourth mean of proportions are weighted means, and when the t value is larger, the weight is larger, and vice versa, the weight is smaller;

[0077] Preferably, the first proportion and the second proportion are area proportions;

[0078] Alternatively: the first proportion and the second proportion are proportions of specific pixels in the video frame;

[0079] Alternatively, step S3 specifically includes: obtaining, for each of the first image sensor and the second image sensor, a fifth proportion of each first feature region in the second region (or non-first region) in each video frame thereof in the first video frame, and a sixth proportion of each second feature region in the second region (or non-first region) in the second video frame; and making a control decision based on the fifth proportion and the sixth proportion;

[0080] The specific steps include:

[0081] Step S3 E1: For the first image sensor and the second image sensor, respectively, obtain the fifth proportion of each first feature area in the second area of ​​the video frame in the first video frame, and the sixth proportion of each second feature area in the second area in the second video frame; for the video frame stream, respectively, a fifth proportion matrix [pa5 t，a ] and the sixth proportion matrix [pa6 t，b ]; where: t is the video frame number in the video frame stream; a is the feature region number in the feature region combination;

[0082] Step S3E2: Compare the fifth proportion matrix with the fifth proportion sample to obtain a first decision corresponding to the fifth proportion sample; compare the sixth proportion matrix with the sixth proportion sample to obtain a second decision corresponding to the sixth proportion sample; determine a control decision based on the first decision and the second decision; the method for determining the control decision based on the first decision and the second decision is the same as above;

[0083] The fifth proportion matrix and the fifth proportion sample are compared to obtain the first decision corresponding to the fifth proportion sample, specifically: the fifth sample similarity dec5 is calculated using the following formula (4), and the first decision corresponding to the fifth proportion sample with the smallest fifth sample similarity is used as the determined first decision; wherein: Smpa5 t，a is the t, ath fifth-proportion sample element in the fifth-proportion sample;

[0084]

[0085] The sixth proportion matrix and the sixth proportion sample are compared to obtain the second decision corresponding to the sixth proportion sample, specifically: the sixth sample similarity dec6 is calculated using the following formula (5), and the second decision corresponding to the sixth proportion sample with the smallest sixth sample similarity is used as the determined second decision; wherein: Smpa6 t，a It is the t, ath sixth-proportion sample element in the sixth-proportion sample;

[0086]

[0087] Preferably: the corresponding relationship between the fifth proportion sample and the first decision is pre-stored; and the corresponding relationship between the sixth proportion sample and the second decision is pre-stored;

[0088] Preferably, the first decision and the second decision are one or more of the angle, viewing angle, field of view, attitude, direction, etc. of the aircraft or the monitored target;

[0089] The method further comprises step S4, determining a control decision based on position information of the feature regions in the video frame and a relationship between pixel values ​​of the feature regions;

[0090] Preferably, step S3 is performed locally on the aircraft, and step S4 is performed on the server side;

[0091] The step S4 specifically includes the following steps:

[0092] Step S41: splicing the first video frame and the second video frame into a third video frame; the splicing method is not limited, as long as it is a fixed splicing method; setting a reference origin in the third video frame, and the origin can be set at the splicing position;

[0093] Step S42: determining the position of each feature region in the third video frame; since it is direct splicing, coordinate conversion and pixel projection are not required;

[0094] Step S43: Calculate the coverage area between the origin and the feature region combination a; specifically, use the following formula (6) to calculate the coverage area S a ; Among them: (x a1 ,y a1 ) is the position of the feature point in the first feature area of ​​the feature area combination b, (x a2 ,y a2 ) is the position of the feature point in the second feature area of ​​the feature area combination a, (x0, y0) is the position of the reference origin; a = 1 ~ A;

[0095]

[0096] Preferably, the feature point is a specific point in the feature area, for example, the point farthest / closest to the origin in the Y-axis direction, the geometric center point in the feature area, the pixel value centroid point, etc.

[0097] Step S44: Obtain the mean pixel value of each feature region in the feature region combination; perform gradient difference between feature regions on the mean pixel value; specifically, use the following formula (7) to perform gradient difference between feature regions to obtain the differential pixel value p of feature region combination a a ; Among them: p a1 , p a2 are the mean pixel values ​​of the first feature area and the second feature area respectively;

[0098]

[0099] Step S45: normalize the mean pixel values ​​of the different feature region combinations a; normalize the coverage area S of the different feature region combinations a. a Perform normalization processing; specifically: set set up

[0100] Step S46: Use the neural network model as the prediction model to predict the control decision and set the parameters of the prediction model; specifically: use the BP neural network model as the prediction model; set the number of hidden layer nodes to 2A+n, and set the objective function to Where: 2A is the number of nodes in the input layer neural network; n is the number of nodes in the output layer neural network; dec i is the expected output of the i-th node of the neural network; decEXP i is the predicted output of the i-th node;

[0101] Step S47: Based on the first video frame and the second video frame acquired in real time, a coverage area sequence and a pixel value mean sequence are obtained to construct an input vector of the prediction model. a＝1～A , p a＝1～A >, input the input vector into the prediction model to obtain the control decision dec; n is the dimension of the control decision; that is, the control decision can contain n-dimensional control information;

[0102] When the control decision based on area ratio cannot meet the control decision needs in terms of dimension and accuracy, the control information dimension can be further expanded based on the prediction model. The coverage area sequence can effectively eliminate the impact of the wrong feature area combination. The influence of environmental factors on the acquisition of different sensors is introduced by the pixel value mean, which facilitates the rapid decision-making based on environmental factors with a certain degree of robustness. There is no need to perform spatial projection and positional relationship transformation between different image sensors, avoiding the introduction of erroneous image analysis information to bring more decision-making impacts.

[0103] Based on the same inventive concept, the present invention further provides a remote flight control system for an aircraft, the system being used to implement the aforementioned remote flight control method for an aircraft; the system comprising an aircraft and a server; the aircraft being provided with a first image sensor and a second image sensor, and a local decision module being provided on the aircraft for transmitting the video frames captured by the first image sensor and the second image sensor to the server, and the server transmitting the decision control information to the aircraft for remote control based on the remote decision control information obtained in step S4 of the aforementioned method;

[0104] By balancing the computational load between the aircraft and the server through local and remote control, the aircraft can make rapid decisions locally without relying on information from other sensors, without increasing excessive hardware computing costs. It can also leverage the computing power of the server to make AI-powered decisions.

[0105] ​Based on the same inventive concept, the present invention further provides a remote flight control server for an aircraft, the server being used to implement the aforementioned remote flight control method for an aircraft. A first image sensor and a second image sensor are provided on the aircraft, and the server is used to receive, in real time, video frame streams transmitted by the first and second image sensors. Based on the decision control information obtained in step S3 and / or step S4 of the aforementioned method, the server transmits the decision control information to the aircraft for remote control. In other words, rapid decision-making and AI-based decision-making can be deployed locally on the aircraft and remotely on the server as needed.

[0106] A computer program (also referred to as a program, software, software application, script, or code) can be written in any form of programming language, including assembly or interpreted languages, declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple collaborative files (e.g., files storing one or more modules, subroutines, or code portions). A computer program can be deployed to execute on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[0107] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0108] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0109] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0110] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A remote flight control method for an aircraft, characterized in that: The method comprises: Step S1: A first image sensor and a second image sensor provided on the aircraft respectively capture in real time a first video consisting of a first video frame stream and a second video consisting of a second video frame stream during flight; Step S2: Dividing the video frames corresponding to the same time acquired by different image sensors into three different regions, namely, a first region, a second region, and a third region. Specifically, the first region, the second region, and the third region are determined for the first video frame and the second video frame acquired by the first image sensor and the second image sensor, respectively. The first region is a fixed region, and the image region within the fixed region is an image region of a fixed target. The second area is the overlapping area, which is the image area of ​​different image sensors with different viewing angles for the same target; The third area is other areas; The determining of the first area specifically includes: when the image area in the video frame belongs to a fixed target, determining the image area as the first area; The determining of the third area specifically comprises: taking a portion other than the first area and the second area as the third area; The determining of the second area specifically comprises: determining the second area by locating a combination of feature areas; Step S3: Obtaining, for each of the first image sensor and the second image sensor, a first proportion of the second area in the first video frame and a second proportion of the second area in the second video frame; and making a control decision based on the first proportion and the second proportion; Step S31: obtaining a first proportion and a second proportion of the second region in the first video frame and the second video frame, respectively, and forming a first proportion sequence and a second proportion sequence for the video frame stream; Step S32: comparing the first proportion sequence with the first proportion sequence sample to obtain a first decision corresponding to the first proportion sample; Comparing the second proportion sequence with the second proportion sequence sample to obtain a second decision corresponding to the first proportion sample; A control decision is determined based on the first decision and the second decision.

2. The remote flight control method for an aircraft according to claim 1, characterized in that: The first image sensor and the second image sensor are arranged at different positions of the aircraft.

3. The remote flight control method for an aircraft according to claim 2, characterized in that: After obtaining the video frame stream, the pixel values ​​are preprocessed.

4. The remote flight control method for an aircraft according to claim 3, characterized in that: After acquiring the video frame stream, the pixel values ​​are normalized.

5. A remote flight control system for an aircraft, characterized in that: The remote flight control system of the aircraft is used to implement the remote flight control method of the aircraft according to any one of claims 1 to 4.

6. A remote flight control server for an aircraft, characterized in that: The remote flight control server of the aircraft is used to implement the remote flight control method of the aircraft according to any one of claims 1 to 4.

Images