Remote flight control method and system of aircraft

Abstract

The invention relates to a remote flight control method and system for an aircraft. The method comprises the steps that videos formed by video frame streams in the flight process are collected in real time through a first image sensor and a second image sensor which are arranged on the aircraft; performing region division on video frames which are acquired by different image sensors and correspond to the same time to form three different regions respectively; and making a control decision based on the area proportion of the region. According to the method, the first to-be-selected feature point sequence or the second to-be-selected feature point sequence is constructed through the video frame streams collected by different image sensors for alternate region construction, region division and image positioning can be completed under the condition of large redundancy, meanwhile, time sequence information brought by video stream information can be fully utilized, and the image positioning accuracy is improved. Therefore, real-time rapid decision making is carried out, the flight control redundancy and robustness are improved, and the flight control calculation overhead is reduced.

Description

[Technical Field]

[0001] This invention belongs to the field of flight control technology, and in particular relates to a remote flight control method and system for an aircraft. [Background Technology]

[0002] The flight control system, or flight controller for short, can be seen as the brain of an aircraft. The flight, hovering, and attitude changes of a multi-rotor aircraft are all achieved by various sensors transmitting attitude data from the aircraft back to the flight controller. The flight controller then calculates and interprets the data before issuing commands, which are then executed by the actuators to complete the actions and adjust the flight attitude. It involves sensor technology, communication technology, information processing technology, intelligent control technology, and aerospace propulsion technology, and is a high-tech product of the information age. It can maximize productivity, reduce costs and risks, ensure on-site safety, confidentiality, and compliance with regulations. Its applications are rapidly expanding, including aerial photography, package delivery, agriculture, law enforcement and surveillance, infrastructure inspection, and scientific research. Aircraft can eliminate the threat to human life while retaining many advantages of manned aircraft, especially with small sensor components that can collect necessary information for both reconnaissance and surveillance, as well as communication. Aircraft often employ commercially available sensor components, including electro-optical devices, forward-looking infrared (FLIR) systems, synthetic aperture radar (SAR), and moving target designators (MTDs). More importantly, the control aircraft can also be equipped with image sensors to acquire high-resolution battlefield reconnaissance and surveillance images. Because image sensors are inexpensive and widely used, this significantly reduces the cost of aircraft. Furthermore, cameras with built-in sensors provide excellent wide-area and long-range day and night surveillance capabilities under ground operator control. SAR offers all-weather surveillance capabilities that are difficult to achieve with electro-optical or infrared technologies. While aircraft are often equipped with GPS and inertial navigation systems to transmit accurate positioning information and calculate the positions of fixed and moving targets within the sensor's field of view, and may also be equipped with C-band variable-frequency data links and UHF Ku-band satellite links for controlling UAVs and payloads, GPS and inertial navigation systems cannot acquire environmental information, especially real-time environmental information, from aircraft.

[0003] Utilizing image information for flight control can create redundancy with other sensor information, increasing the robustness of flight control. However, when using image information for flight control, the environment during flight is often complex, with few or no obvious target features. Methods like defining target boundaries or using feature points for region segmentation or image localization comparison are inefficient. Furthermore, the aircraft's computing power is limited and cannot support complex image analysis. Many analysis results are also useless in flight control. The challenge lies in how to utilize image information for flight control while simultaneously using it for server-based real-time flight control. To address these issues, this invention proposes a remote flight control method and system for aircraft. By constructing alternating regions using video frame streams acquired from different image sensors, a first or second sequence of candidate feature points is built. This allows for region segmentation and image localization with significant redundancy, while fully utilizing the temporal information from the video stream for real-time, rapid decision-making. This improves flight control redundancy and robustness, and significantly reduces computational overhead. [Summary of the Invention]

[0004] To address the aforementioned problems in the prior art, this invention proposes a remote flight control method and system for an aircraft, the method comprising:

[0005] Step S1: The first image sensor and the second image sensor installed on the aircraft respectively acquire the first video consisting of the first video frame stream and the second video consisting of the second video frame stream during the flight process in real time.

[0006] Step S2: Divide the video frames acquired by different image sensors at the same time into three different regions; specifically: for the first video frame acquired by the first image sensor and the second video frame acquired by the second image sensor, determine the first region, the second region, and the third region respectively; the first region is a fixed region, and the image region within the fixed region is the image region for a fixed target; the second region is an overlapping region, and the overlapping region is the image region of different image sensors for the same target from different perspectives; the third region is other regions;

[0007] The determination of the first region specifically involves: when an image region in a video frame belongs to a fixed target, that image region is determined as the first region;

[0008] The determination of the third region specifically involves: taking the portion that is not in the first region and the second region as the third region;

[0009] Determining the second region specifically involves: determining the second region by combining location feature regions; specifically including the following steps:

[0010] Step S2A1: Arrange the pixels in the first video frame and the second video frame in descending order of pixel value to form the first candidate feature point sequence and the second candidate feature point sequence, respectively.

[0011] Step S2A2: Determine the current candidate feature point sequence; specifically: alternately use the first candidate feature point sequence or the second candidate feature point sequence as the current candidate feature point sequence, and the non-current candidate feature point sequence is called another candidate feature point sequence; select the head pixel in the current candidate feature point sequence as the positioning pixel;

[0012] Step S2A3: The pixel value of the located pixel is I c Place the located pixel into the feature region; extend the region in the video frame containing the located pixel, centered on the located pixel, and if another pixel I exists... c1 If the positioning pixel and the other pixel satisfy the following condition (1), and the positioning pixel and the other pixel are within a 4-neighborhood, then the other pixel is placed in the feature region; repeat this step until there is no other pixel that can be placed in the feature region; where: Tr is a preset value;

[0013]

[0014] Step S2A4: If another candidate feature point sequence has a pixel that forms another feature region that also satisfies condition (1) and all pixels in the region are within the 4-neighborhood, then calculate the pixel value similarity and area similarity between the feature region and the other feature region; if the pixel value similarity and area similarity of the two are both less than the similarity threshold, then the feature region located in the first video frame is called the first feature region, and the feature region located in the second video frame is called the second region; the first feature region and the second feature region constitute feature region combination a;

[0015] Step S2A5: Delete the pixels involved in the feature region or another feature region from the first candidate feature point sequence or the second candidate feature point sequence;

[0016] Step S2A6: If there are still unprocessed pixels in either the first candidate feature point sequence or the second candidate feature point sequence, return to step S2A2;

[0017] Step S2A7: Take the smallest region containing all feature regions in each video frame as the second region;

[0018] Step S3: For the first image sensor and the second image sensor respectively, obtain the first proportion of the second region in the first video frame and the second proportion of the second region in the second video frame; make control decisions based on the first proportion and the second proportion.

[0019] Furthermore, the first image sensor and the second image sensor are positioned at different locations on the aircraft.

[0020] Furthermore, only pixels with pixel values ​​greater than the average pixel value are selected for inclusion in the candidate feature point sequence.

[0021] Furthermore, the pixel value is a grayscale value.

[0022] Furthermore, the pixel values ​​are preprocessed after the video frame stream is acquired.

[0023] Furthermore, Tr = 0.1.

[0024] Furthermore, the minimum region is a minimum rectangular region, an elliptical region, or a circular region.

[0025] Furthermore, after acquiring the video frame stream, the pixel values ​​are normalized.

[0026] A remote flight control system for an aircraft, wherein the remote flight control system is used to implement the aforementioned remote flight control method for the aircraft.

[0027] A remote flight control server for an aircraft, characterized in that the remote flight control server is used to implement the aforementioned remote flight control method for the aircraft.

[0028] The beneficial effects of this invention include:

[0029] (1) By constructing a first candidate feature point sequence or a second candidate feature point sequence through video frame streams acquired by different image sensors, the region can be constructed alternately. This can complete the region division and image localization under a large degree of redundancy, and at the same time, it can make full use of the temporal information brought by the video stream information, and then make simple, real-time and fast decisions.

[0030] (2) When the control decision based on area ratio cannot meet the control decision requirements in terms of dimension and accuracy, the control information dimension can be further expanded based on the prediction model. The coverage area sequence can effectively eliminate the influence of the combination of erroneous feature regions. By introducing the influence of environmental factors on the acquisition of different sensors through the average pixel value, it is convenient to make a robust and fast decision based on environmental factors without the need for spatial projection and positional relationship transformation between different image sensors, thus avoiding the introduction of erroneous image analysis information and bringing more decision influence.

[0031] (3) By achieving a balance of computational load between the aircraft and the server through local control and remote control, the aircraft can make rapid decisions locally without relying on information from other sensors without increasing the hardware computing cost; it can also make artificial intelligence decisions with the help of the server's computing power. [Attached Image Description]

[0032] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to unduly limit the invention. In the drawings:

[0033] Figure 1 A schematic diagram of the remote flight control method for the aircraft provided by the present invention.

Detailed Implementation Methods

[0034] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. The illustrative embodiments and descriptions are only used to explain the present invention and are not intended to limit the present invention.

[0035] This invention proposes a remote flight control method and system for an aircraft, as shown in the appendix. Figure 1 As shown, the method includes the following steps:

[0036] Step S1: The first image sensor and the second image sensor installed on the aircraft respectively acquire, in real time, a first video consisting of a first video frame stream and a second video consisting of a second video frame stream during the flight process; the first image sensor and the second image sensor are installed at different positions on the aircraft;

[0037] Preferably, the fixed target is an aircraft;

[0038] Preferably, the fixed target is an auxiliary device located on the aircraft;

[0039] Preferably, the same target is the monitoring target;

[0040] Preferably, there are two or more image sensors; when there are two image sensors, the two image sensors are binocular cameras; the first image sensor and the second image sensor have intersecting and opposite viewing angles; of course, when the number of image sensors is expanded to multiple, the control method is similar and will not be described in detail here.

[0041] Step S2: Divide the video frames acquired by different image sensors at the same time into three different regions, and construct a second region based on the combination of feature regions; specifically: determine the first region, the second region, and the third region for the first video frame acquired by the first image sensor and the second video frame acquired by the second image sensor respectively; the first region is a fixed region, and the image region within the fixed region is the image region for a fixed target; the second region is an overlapping region, and the overlapping region is the image region from different perspectives of different image sensors for the same target (or target range); the third region is other regions (not the first region and the second region);

[0042] The determination of the first region specifically involves: performing image segmentation on the video frame to determine the first region; when an image region in the video frame belongs to a fixed target, that image region is determined as the first region; for example, it can be done by template comparison.

[0043] The determination of the third region specifically refers to: referring to the portion that is not in the first region and the second region as the third region;

[0044] Determining the second region specifically involves: determining the second region by combining location feature regions; specifically including the following steps:

[0045] Step S2A1: Arrange the pixels in the first video frame and the second video frame in descending order of pixel value to form the first candidate feature point sequence and the second candidate feature point sequence, respectively.

[0046] Preferred: Only pixels with pixel values ​​greater than the average pixel value are selected for inclusion in the candidate feature point sequence;

[0047] Preferred: Only pixels whose pixel values ​​fall within a specific range are selected for inclusion in the candidate feature point sequence;

[0048] Preferably, the pixel value is a grayscale value;

[0049] Preferred method: Perform preprocessing such as normalization on pixel values;

[0050] Preferred method: If multiple pixels have the same pixel value, these pixels are randomly arranged; or the pixels that are closer to the center point of the area formed by these pixels are arranged at the front.

[0051] Step S2A2: Determine the current candidate feature point sequence; specifically: alternately use the first candidate feature point sequence or the second candidate feature point sequence as the current candidate feature point sequence, and the non-current candidate feature point sequence is called another candidate feature point sequence; select the head pixel in the current candidate feature point sequence as the positioning pixel;

[0052] Step S2A3: The pixel value of the located pixel is I c Place the located pixel into the feature region; extend the region in the video frame containing the located pixel, centered on the located pixel, and if another pixel I exists... c1 If the positioning pixel satisfies the following condition (1), and the positioning pixel and the other pixel are within a 4-neighborhood (8-neighborhood), then the other pixel is placed in the feature region; repeat this step until there is no other pixel that can be placed in the feature region; where: Tr is a preset value;

[0053]

[0054] Preferred setting: Tr = 0.1;

[0055] Step S2A4: If another candidate feature point sequence has a pixel that forms another feature region that also satisfies condition (1) and all pixels in the region are within the range of 4 neighborhoods (8 neighborhoods), then calculate the pixel value similarity and area similarity between the feature region and the other feature region; if the pixel value similarity and area similarity of the two are both less than the similarity threshold, then the feature region (called the first feature region) and the other feature region (called the second feature region) constitute feature region combination a;

[0056] Preferably, pixel value similarity is described using Euclidean distance; area similarity is described using the number of pixels (e.g., the number of pixels is close or equal).

[0057] Preferred approach: Select the top A feature regions with the smallest pixel value similarity and area similarity coupling (coupling method is addition, product, etc.) to perform feature region combination screening. This can reduce the difficulty of using samples for comparison in subsequent decision-making.

[0058] Preferably, the similarity threshold is a preset value;

[0059] Step S2A5: Delete the pixels involved in the feature region and another feature region from the first candidate feature point sequence or the second candidate feature point sequence;

[0060] Step S2A6: If there are still unprocessed pixels in either the first candidate feature point sequence or the second candidate feature point sequence, return to step S2A2;

[0061] Step S2A7: The smallest region containing the feature region is designated as the second region; the feature region located in the first video frame is designated as the first feature region, and the feature region located in the second video frame is designated as the second region; the first feature region and the second feature region constitute a feature region combination;

[0062] Preferably, the minimum region is a closed region;

[0063] Preferably, the minimum region is a minimum rectangular region, an elliptical region, a circular region, etc.; of course, the feature region can also be directly used as the second region; the method is similar and will not be described in detail here.

[0064] Because the environment targets are complex during the flight of the aircraft, and the target features are often few or even non-existent, the efficiency of region division or image localization comparison by determining the boundary of the monitoring target or by locating through feature points is low. Region division and image localization can be completed with a large degree of redundancy by constructing regions by alternating the first candidate feature point sequence or the second candidate feature point sequence.

[0065] Step S3: For both the first and second image sensors, obtain the first proportion of the second region (or non-first region) in the first video frame and the second proportion of the second region (or non-first region) in the second video frame; make control decisions based on the first and second proportions; specifically including the following steps:

[0066] Step S31: Obtain the first and second proportions of the second region in the first and second video frames respectively, which constitute the first proportion sequence (pa1) for the video frame stream. t ) and the second proportion sequence (pa2) t ); where: t is the video frame number in the video frame stream;

[0067] Step S32: Compare the first proportion sequence with the first proportion sequence sample to obtain the first decision corresponding to the first proportion sample; compare the second proportion sequence with the second proportion sequence sample to obtain the second decision corresponding to the first proportion sample; determine the control decision based on the first decision and the second decision;

[0068] The step of comparing the first proportion sequence with the first proportion sequence sample to obtain the first decision corresponding to the first proportion sequence sample is as follows: the first sample similarity dec1 is calculated using the following formula (2), and the first decision corresponding to the first proportion sequence sample with the smallest first sample similarity is taken as the determined first decision; where: Smpa1 t It is the t-th first proportion sequence sample in the first proportion sequence sample column;

[0069]

[0070] The second sample similarity dec1 is calculated using the following formula (3), and the second decision corresponding to the second proportion sequence sample with the smallest second sample similarity is taken as the determined second decision; where: Smpa2 t It is the t-th second proportion sequence sample in the second proportion sequence sample;

[0071]

[0072] Preferably, the correspondence between the first proportion sample and the first decision is stored in advance; and the correspondence between the second proportion sample and the second decision is stored in advance.

[0073] Preferably, the first decision and the second decision are one or more of the angle, viewpoint, field of view, attitude, direction, etc. of the aircraft or the monitoring target;

[0074] The control decision is determined based on the first decision and the second decision. Specifically, when the first decision and the second decision are the same, the control decision is determined as the first decision; otherwise, the decision information corresponding to the smaller of the first sample similarity and the second sample similarity is selected as the control decision.

[0075] Alternatively: The control decision is determined based on the first decision and the second decision; specifically: when the first decision and the second decision are the same, the control decision is determined as the first decision; otherwise, for the first image sensor and the second image sensor respectively, the third proportion of the first region in the first video frame and the fourth proportion of the first region in the second video frame are calculated, thus forming the third proportion sequence (pa3) for the video frame stream. t ) and the fourth proportion sequence (pa4) t ); Calculate the mean of the third proportion of the third proportion sequence and the mean of the fourth proportion sequence; use the decision information of the video frame stream corresponding to the smaller of the mean of the third proportion and the mean of the fourth proportion as the control decision;

[0076] Preferably, the third and fourth average proportions are weighted averages. When the t value is larger, the weight is larger, and vice versa.

[0077] Preferably, the first and second percentages are area percentages;

[0078] Alternatively: The first percentage and the second percentage are the percentages of a specific pixel in a video frame;

[0079] Alternatively: Step S3 specifically involves: for the first image sensor and the second image sensor respectively, obtaining the fifth proportion of each first feature region in the second region (or non-first region) in the first video frame, and the sixth proportion of each second feature region in the second region (or non-first region) in the second video frame; and making control decisions based on the fifth and sixth proportions.

[0080] Specifically, the steps include the following:

[0081] Step S3 E1: For the first image sensor and the second image sensor respectively, obtain the fifth proportion of each first feature region in the second region of the video frame in the first video frame, and the sixth proportion of each second feature region in the second region of the second video frame; for the video frame stream, this constitutes the fifth proportion matrix [pa5]. t，a ] and the sixth proportion matrix [pa6 t，b Where: t is the video frame number in the video frame stream; a is the feature region number in the feature region combination;

[0082] Step S3E2: Compare the fifth proportion matrix with the fifth proportion sample to obtain the first decision corresponding to the fifth proportion sample; compare the sixth proportion matrix with the sixth proportion sample to obtain the second decision corresponding to the sixth proportion sample; determine the control decision based on the first and second decisions; the method of determining the control decision based on the first and second decisions is the same as above;

[0083] The step of comparing the fifth proportion matrix with the fifth proportion sample to obtain the first decision corresponding to the fifth proportion sample is as follows: the similarity dec5 of the fifth sample is calculated using the following formula (4), and the first decision corresponding to the fifth proportion sample with the smallest similarity to the fifth sample is taken as the determined first decision; where: Smpa5 t，a It is the t,a-th element of the fifth proportion sample;

[0084]

[0085] The step of comparing the sixth proportion matrix with the sixth proportion sample to obtain the second decision corresponding to the sixth proportion sample is as follows: the similarity dec6 of the sixth sample is calculated using the following formula (5), and the second decision corresponding to the sixth proportion sample with the smallest similarity to the sixth sample is taken as the determined second decision; where: Smpa6 t，a It is the t, a-th element of the sixth proportion sample in the sixth proportion sample;

[0086]

[0087] Preferred method: Pre-store the correspondence between the fifth proportion sample and the first decision; and the correspondence between the sixth proportion sample and the second decision;

[0088] Preferably, the first decision and the second decision are one or more of the angle, viewpoint, field of view, attitude, and direction of the aircraft or the monitoring target;

[0089] The method further includes step S4, which determines a control decision based on the location information of the feature region in the video frame and the pixel value relationship between the feature regions.

[0090] Preferably, step S3 is executed locally on the aircraft, while step S4 is executed on the server.

[0091] Step S4 specifically includes the following steps:

[0092] Step S41: Stitch the first video frame and the second video frame together to form the third video frame; the stitching method is not limited, as long as it is a fixed stitching method; set the reference origin in the third video frame, which can be set at the stitching point;

[0093] Step S42: Determine the position of each feature region in the third video frame; since it is a direct stitching, no coordinate transformation or pixel projection is required;

[0094] Step S43: Calculate the coverage area formed by the combination of the origin and the feature region a; specifically, calculate the coverage area S using the following formula (6). a ;wherein: (x a1 y a1 ) is the position of the feature point in the first feature region of the feature region combination b, (x a2 y a2 (x0, y0) represents the position of a feature point in the second feature region of feature region combination a, and (x0, y0) represents the position of the reference origin; a = 1 ~ A;

[0095]

[0096] Preferred features are specific points within a feature region, such as the point furthest / nearest from the origin along the Y-axis, the geometric center point of the feature region, or the centroid of pixel values.

[0097] Step S44: Obtain the mean pixel value of each feature region in the feature region combination; perform gradient difference between feature regions on the mean pixel value; specifically: use the following formula (7) to perform gradient difference between feature regions to obtain the difference pixel value p of feature region combination a. a Where: p a1 p a2 These are the average pixel values ​​of the first and second feature regions, respectively.

[0098]

[0099] Step S45: Normalize the average pixel values ​​of different feature region combinations a; normalize the coverage area S of different feature region combinations a. a Perform normalization processing; specifically: set set up

[0100] Step S46: Use a neural network model as the prediction model for control decision prediction, and set the parameters of the prediction model; specifically: use a BP neural network model as the prediction model; set the number of hidden layer nodes to 2A+n, and set the objective function to... Where: 2A is the number of nodes in the input layer neural network; n is the number of nodes in the output layer neural network; dec i Let decEXP be the expected output of the i-th node in the neural network. i This is the predicted output for the i-th node;

[0101] Step S47: Based on the real-time acquired first and second video frames, obtain the coverage area sequence and the mean pixel value sequence to construct the input vector of the prediction model. a＝1～A p a＝1～A The input vector is fed into the prediction model to obtain the control decision dec; n is the dimension of the control decision; that is, the control decision can contain control information in n dimensions.

[0102] When control decisions based on area proportion cannot meet the needs of control decisions in terms of dimensionality and accuracy, the controllable information dimension can be further expanded based on the prediction model. The coverage area sequence can effectively eliminate the influence of erroneous feature region combinations. By introducing the influence of environmental factors on the acquisition of different sensors through the pixel value mean, it is convenient to make rapid decisions based on environmental factors with a certain degree of robustness, without the need for spatial projection and positional relationship transformation between different image sensors, thus avoiding the introduction of erroneous image analysis information that brings more decision-making impact.

[0103] Based on the same inventive concept, the present invention also provides a remote flight control system for an aircraft, the system being used to implement the aforementioned remote flight control method for an aircraft; the system includes an aircraft and a server; the aircraft is equipped with a first image sensor and a second image sensor, and a local decision module is provided on the aircraft for using local decision control information obtained in step S3 of the aforementioned method; and sending video frames collected by the first image sensor and the second image sensor to the server, the server sending decision control information to the aircraft for remote control based on the remote decision control information obtained in step S4 of the aforementioned method;

[0104] By achieving a balance of computational load between the aircraft and the server through local and remote control, the aircraft can make rapid decisions locally without relying on information from other sensors, without increasing excessive hardware computing costs; it can also make artificial intelligence decisions by leveraging the computing power of the server.

[0105] ​Based on the same inventive concept, the present invention also provides a remote flight control server for an aircraft. The server is used to implement the aforementioned remote flight control method for an aircraft. A first image sensor and a second image sensor are installed on the aircraft. The server is used to receive video frame streams sent by the first and second image sensors in real time. Based on the decision control information obtained in steps S3 and / or S4 of the above method, the server sends the decision control information to the aircraft for remote control. In other words, rapid decision-making and AI-based decision-making can be deployed locally on the aircraft and remotely on the server as needed.

[0106] A computer program (also referred to as a program, software, software application, script, or code) can be written in any form of programming language, including assembly or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may, but does not necessarily, correspond to a file in a file system. A program can be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to said program, or in multiple co-located files (e.g., a file storing one or more modules, subroutines, or code portions). A computer program can be deployed to execute on a single computer or on multiple computers located at a single site or distributed across multiple sites and interconnected by a communications network.

[0107] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0108] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0109] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0110] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A remote flight control method for an aircraft, characterized in that: The method comprises: Step S1: a first image sensor and a second image sensor disposed on the aircraft respectively capture in real time a first video consisting of a first video frame stream and a second video consisting of a second video frame stream during flight; Step S2: dividing the video frames corresponding to the same time acquired by different image sensors into three different areas, namely, a first area, a second area, and a third area; specifically, determining the first area, the second area, and the third area for the first video frame and the second video frame acquired by the first image sensor and the second image sensor, respectively; the first area is a fixed area, and the image area within the fixed area is an image area for a fixed target; The second area is an overlapping area, and the overlapping area is an image area with different viewing angles for the same target by different image sensors; The third area is other areas; The determining of the first area specifically includes: when the image area in the video frame belongs to a fixed target, determining the image area as the first area; The determining of the third area specifically includes: taking a part other than the first area and the second area as the third area; The determining of the second area specifically includes: determining the second area by locating a combination of feature areas; specifically including the following steps: Step S2A1: Arrange the pixels in the first video frame and the second video frame in descending order of pixel value to form a first sequence of candidate feature points and a second sequence of candidate feature points respectively; Step S2A2: determining the current candidate feature point sequence; specifically: alternately taking the first candidate feature point sequence or the second candidate feature point sequence as the current candidate feature point sequence, and the sequence other than the current candidate feature point sequence is called another candidate feature point sequence; selecting the head pixel in the current candidate feature point sequence as the positioning pixel; Step S2A3: The pixel value of the positioning pixel is 1 c ;Put the positioning pixel into the feature area; In the video frame where the positioning pixel is located, the region is extended with the positioning pixel as the center. If there is another pixel I c1 If the following condition (1) is satisfied between the positioning pixel and the other pixel, and the positioning pixel and the other pixel are within the 4-neighborhood range, then the other pixel is placed in the feature area; This step is repeated until there is no more pixel that can be placed in the feature area; where: Tr is a preset value; Step S2A4: If another feature region composed of a pixel point in another candidate feature point sequence also satisfies condition (1) and the pixels in the region are all within the 4-neighborhood range, then the pixel value similarity and area similarity between the feature region and the other feature region are calculated; if the pixel value similarity and area similarity of the two are both less than the similarity threshold, then the feature region located in the first video frame is called the first feature region, and the feature region located in the second video frame is called the second region; the first feature region and the second feature region constitute a feature region combination a; Step S2A5: deleting pixels involved in the feature region or another feature region from the first sequence of feature points to be selected or the second sequence of feature points to be selected; Step S2A6: if there are still unprocessed pixels in the first candidate feature point sequence or the second candidate feature point sequence, return to step S2A2; Step S2A7: taking the smallest region including all feature regions in each video frame as the second region; Step S3: respectively obtaining, for the first image sensor and the second image sensor, a first proportion of the second area in the first video frame and a second proportion of the second area in the second video frame in each video frame; and making a control decision based on the first proportion and the second proportion.

2. The remote flight control method of an aircraft according to claim 1, characterized in that: The first image sensor and the second image sensor are arranged at different positions of the aircraft.

3. The remote flight control method of an aircraft according to claim 2, characterized in that: Only pixels whose pixel values ​​are greater than the average pixel value are intercepted and enter the sequence of candidate feature points.

4. The remote flight control method of an aircraft according to claim 3, characterized in that: The pixel value is a grayscale value.

5. The remote flight control method of an aircraft according to claim 4, characterized in that: After acquiring the video frame stream, the pixel values ​​are preprocessed.

6. The remote flight control method of an aircraft according to claim 5, characterized in that: Tr＝0.1。 7. The remote flight control method of an aircraft according to claim 5, characterized in that: It is characterized in that The minimum area is a minimum rectangular area, an elliptical area or a circular area.

8. The remote flight control method of an aircraft according to claim 5, characterized in that: After acquiring the video frame stream, the pixel values ​​are normalized.

9. A remote flight control system for an aircraft, characterized in that: The remote flight control system of the aircraft is used to implement the remote flight control method of the aircraft described in any one of claims 1 to 8.

10. A remote flight control server for an aircraft, characterized in that: The remote flight control server of the aircraft is used to implement the remote flight control method of the aircraft described in any one of claims 1-8.

Images