Flight target detection method, device and system based on ADS-B data and remote sensing image, and medium

Abstract

The invention provides a flying target detection method, device and system based on ADS-B data and a remote sensing image, and a medium, and relates to the technical field of remote sensing image application and target detection. The target detection method comprises the steps of obtaining a remote sensing image of a target area, imaging time and an imaging range of the remote sensing image and a first movement speed of a remote sensing satellite; obtaining target ADS-B data from pre-stored ADS-B data based on the imaging time and the imaging range; determining a candidate area in the remote sensing image based on the first motion speed and the target ADS-B data, wherein the candidate area comprises a flight target; and carrying out contour extraction on the candidate region, and determining the position of the flying target in the remote sensing image.

Description

Technical Field

[0001] The present invention relates to the technical fields of remote sensing image applications and target detection, and particularly to a method, device, system and medium for detecting flying targets based on ADS-B data and remote sensing images. Background Art

[0002] With the vigorous development of the aviation industry, aircraft have been more and more widely used in human life. How to effectively and accurately obtain the motion parameters of flying targets in the air is of crucial significance in the fields of aviation and the like.

[0003] Regarding the estimation of the motion parameters of flying targets, there are already methods to achieve it. For example, a method for calculating the altitude and speed of moving targets in the air based on field-of-view spectroscopic hyperspectral imaging detection, and an analysis method for detecting moving targets in the air and sea environment using hyperspectral technology. Although the above methods estimate the motion parameters of flying targets in the air through hyperspectral satellite images, the estimation results are not compared and verified with the real motion parameters. Summary of the Invention

[0004] In view of this, the present invention provides a method, device, system and medium for detecting flying targets based on ADS-B data and remote sensing images, which are used to at least partially solve the above technical problems.

[0005] The first aspect of the embodiments of the present invention provides a method for detecting flying targets based on ADS-B data and remote sensing images, including: obtaining a remote sensing image of a target area, the imaging time and imaging range of the remote sensing image, and the first motion speed of the remote sensing satellite; obtaining target ADS-B data from the pre-stored ADS-B data based on the imaging time and imaging range; determining a candidate area in the remote sensing image based on the first motion speed and the target ADS-B data, the candidate area containing a flying target; extracting the contour of the candidate area to determine the position of the flying target in the remote sensing image.

[0006] According to the embodiments of the present invention, obtaining target ADS-B data from the pre-stored ADS-B data based on the imaging time and imaging range includes: obtaining initial ADS-B data associated with the flying target in the remote sensing image from the pre-stored ADS-B data based on the imaging time and imaging range; determining the initial ADS-B data whose time difference between the sending time of the ADS-B data and the recording time of the last recorded ADS-B data of the flying target meets a preset condition as the target ADS-B data.

[0007] According to an embodiment of the present invention, determining a candidate region in a remote sensing image based on a first motion speed and target ADS-B data includes: obtaining the actual motion speed and actual position of a flight target from the target ADS-B data; determining a second motion speed of the flight target in the remote sensing image based on the first motion speed and the actual motion speed; determining an offset distance of the flight target based on the second motion speed and the time difference between the transmission time of the ADS-B data and the recording time of the last recorded ADS-B data of the flight target; and determining an implementation candidate region based on the offset distance and the actual position.

[0008] According to an embodiment of the present invention, the second motion speed of the flight target in the remote sensing image is equal to the sum of the first motion speed and the actual motion speed.

[0009] According to an embodiment of the present invention, performing contour extraction on the candidate region to determine the position of the flight target in the remote sensing image includes: processing the pixel values of each pixel point in the candidate region based on a preset pixel threshold to obtain a first image; and performing region division on the pixels in the first image to obtain a second image, where the second image only contains the pixels of the flight target.

[0010] According to an embodiment of the present invention, processing the pixel values of each pixel point in the candidate region based on a preset pixel threshold to obtain a first image includes: determining a preset pixel threshold based on the pixel values of each pixel in the candidate region; and setting the pixel values less than the preset pixel threshold in the candidate region to zero to obtain the first image.

[0011] According to an embodiment of the present invention, performing region division on the pixels in the first image to obtain a second image includes: determining the pixel point with the largest pixel value in the first image as a reference pixel point; starting from the reference pixel point, merging the pixel points with non-zero pixel values around the reference pixel point into a target region, and the target region is the position of the flight target in the remote sensing image.

[0012] A second aspect of the present invention provides a flight target detection device based on ADS-B data and a remote sensing image, including: a first acquisition module for acquiring a remote sensing image of a target region, the imaging time and imaging range of the remote sensing image, and the first motion speed of a remote sensing satellite; a second acquisition module for acquiring target ADS-B data from pre-stored ADS-B data based on the imaging time and imaging range; a candidate region determination module for determining a candidate region in the remote sensing image based on the first motion speed and the target ADS-B data, where the candidate region contains a flight target; and a position determination module for performing contour extraction on the candidate region to determine the position of the flight target in the remote sensing image.

[0013] The third aspect of the present invention provides a space-ground collaborative aviation monitoring system, comprising: a remote sensing satellite constellation configured to acquire remote sensing images of a target area, the imaging time and imaging range of the remote sensing images, and the first motion speed of the remote sensing satellites; a dynamic airspace situation visualization platform configured to acquire target ADS-B data from pre-stored ADS-B data based on the imaging time and imaging range; determine a candidate area in the remote sensing image based on the first motion speed and the target ADS-B data, the candidate area containing a flight target; perform contour extraction on the candidate area to determine the position of the flight target in the remote sensing image, and convert the position of the flight target into a track heat map containing an ADS-B data verification identifier.

[0014] The fourth aspect of the present invention provides a computer-readable storage medium disposed on board the satellite edge, having executable instructions stored thereon, which when executed by a processor cause the processor to implement the following method: acquire remote sensing images of a target area, the imaging time and imaging range of the remote sensing images, and the first motion speed of the remote sensing satellites; acquire target ADS-B data from pre-stored ADS-B data based on the imaging time and imaging range; determine a candidate area in the remote sensing image based on the first motion speed and the target ADS-B data, the candidate area containing a flight target; perform contour extraction on the candidate area to determine the position of the flight target in the remote sensing image.

[0015] The fifth aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the above method.

[0016] The flight target detection method, device, system and medium based on ADS-B data and remote sensing images according to the embodiments of the present invention have at least the following beneficial effects:

[0017] By using information such as the shooting time and coverage area of the remote sensing image to match the Automatic Dependent Surveillance-Broadcast (ADS-B) data, rapid screening of the remote sensing image containing the flight target and the corresponding ADS-B data of the flight target is achieved. Then, the position information in the corresponding ADS-B data is used to locate the flight target in the remote sensing image, which can reduce the time and workload of searching for air flight targets in the remote sensing image and improve the efficiency of target detection.

[0018] Calculate the speed of the flight target in the image based on the speed in the ADS-B data and the speed of the satellite, calculate the offset pixel range according to the image resolution, and then determine the search range, which can solve the problem of inaccurate positioning caused by the propagation delay of ADS-B data, improve the accuracy of target detection, and provide an accurate value for verifying the estimation results of motion parameters.

[0019] By screening and matching ADS-B data using the imaging time and imaging area in remote sensing images, not only the efficiency of data processing is improved, but also in areas lacking ground base station support, the blank of flight monitoring can be supplemented by satellite images, which is of great significance for enhancing the coverage ability and response efficiency of the global aviation monitoring system. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Through the following description of the embodiments of the present invention with reference to the drawings, the above and other objects, features and advantages of the present invention will become clearer. In the drawings:

[0021] Figure 1 Schematically shows a flowchart of a flight target detection method based on ADS-B data and remote sensing images according to an embodiment of the present invention;

[0022] Figure 2 Schematically shows a vector diagram of the satellite speed, the moving speed of the flight target in the image, and the true speed of the flight target according to an embodiment of the present invention;

[0023] Figure 3 Schematically shows the comparison results of the images before and after processing by the adaptive threshold method according to an embodiment of the present invention;

[0024] Figure 4 Schematically shows the comparison results of the images before and after processing by the region growing algorithm according to an embodiment of the present invention;

[0025] Figure 5 Schematically shows a structural block diagram of a flight target detection device based on ADS-B data and remote sensing images according to an embodiment of the present invention;

[0026] Figure 6 Schematically shows a structural block diagram of an airspace supervision system according to an embodiment of the present invention;

[0027] Figure 7 Schematically shows a block diagram of an electronic device suitable for implementing the flight target detection method according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. In the following detailed description, for the sake of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present invention. However, it is obvious that one or more embodiments can be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts of the present invention.

[0029] The terms used herein are merely for describing specific embodiments and are not intended to limit the present invention. The terms "including", "comprising", etc. used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0030] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0031] In the case of using expressions such as "at least one of A, B, and C, etc.", generally, it should be interpreted according to the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include, but not be limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0032] The ADS - B system is a new air traffic control surveillance technology based on the global positioning system, which uses air - to - ground and air - to - air data links to achieve traffic monitoring and information transmission. The system does not require manual operation or interrogation and can automatically (once per second) obtain parameters from relevant on - board equipment, and broadcast information such as the position, altitude, speed, heading, identification number, etc. of the flight target to other flight targets (aircraft) or ground stations for air traffic controllers to monitor the status of the flight target. Using ADS - B data, the flight parameters of the flight target can be obtained relatively accurately, but the remote sensing image only contains the appearance information of the flight target and does not contain the ADS - B data of the flight target.

[0033] To solve this problem, a method of combining ADS-B data with remote sensing images has emerged. Remote sensing images can provide extensive visual information of the flight area. By combining ADS-B data with remote sensing images, the position of flight targets can be accurately identified and tracked in cases where ADS-B signals are insufficient or visual confirmation is required. In addition, this method can enhance the reliability and effectiveness of flight monitoring in specific application scenarios, such as border monitoring, environmental supervision, or search and rescue operations. Some methods of combining ADS-B data with remote sensing images are mainly used for quickly retrieving and positioning abnormal aviation targets in a large area using low-earth orbit satellites; however, this method relies on spaceborne ADS-B data and the application scenarios are severely limited. Other methods project the sampled ADS-B data into image information and achieve deception detection of ADS-B route information by detecting the structural similarity between images; however, this method only discriminates based on ADS-B data and does not fully utilize the image information provided by remote sensing images.

[0034] In view of this, embodiments of the present invention provide a flight target detection method based on ADS-B data and remote sensing images, which fuses remote sensing images and ADS-B data of flight targets, matches a large amount of ADS-B data using the imaging information of remote sensing images, and avoids manual screening of remote sensing images and ADS-B data; by analyzing the relationship between satellite motion and the flight of flight targets, the problem of inaccurate positioning caused by the propagation delay of ADS-B data is solved, and the search time is greatly shortened.

[0035] Figure 1 The flowchart of the flight target detection method based on ADS-B data and remote sensing images according to an embodiment of the present invention is schematically shown.

[0036] As Figure 1 shown, the flight target detection method based on ADS-B data and remote sensing images of this embodiment may include operation S110 to operation S140.

[0037] In operation S110, obtain the remote sensing image of the target area, the imaging time and imaging range of the remote sensing image, and the first motion speed of the remote sensing satellite.

[0038] In operation S120, obtain the target ADS-B data from the pre-stored ADS-B data based on the imaging time and imaging range.

[0039] In operation S130, determine the candidate area in the remote sensing image based on the first motion speed and the target ADS-B data, and the candidate area contains the flight target.

[0040] In operation S140, perform contour extraction on the candidate area to determine the position of the flight target in the remote sensing image.

[0041] According to an embodiment of the present invention, the target area may be some remote areas, blank areas between ground base stations, or areas of interest. For example, eligible remote sensing images can be collected according to the area of interest and time range, and information such as the scene ID (sceneid), task ID (jobtaskid), satellite ID (satelliteid), imaging start time (scenestarttime), imaging end time (sceneendtime), cloud cover (cloudcover), and imaging range (spatialdata) of each remote sensing image can be extracted.

[0042] According to an embodiment of the present invention, the acquired imaging time and imaging range of each remote sensing image can be used to screen the pre-stored ADS-B data. The eligible ADS-B data returns information such as the identification code, longitude and latitude, flight speed, azimuth angle, altitude, time of sending ADS-B, and time of the last recorded ADS-B of the flight target. The application programming interface (API) of OpenSky can be called to screen the ADS-B data in the database. OpenSky is an open aviation data platform operated by OpenSky Network. OpenSky has collected more than 30 trillion messages, and these data come from more than 6,000 sensors around the world. This makes OpenSky the largest aviation traffic surveillance data repository of its kind, and researchers and developers can access these data in various ways, including API and Impala database.

[0043] In some embodiments, obtaining the target ADS-B data from the pre-stored ADS-B data based on the imaging time and imaging range may include:

[0044] Obtaining the initial ADS-B data associated with the flight target in the remote sensing image from the pre-stored ADS-B data based on the imaging time and imaging range.

[0045] Determining the initial ADS-B data whose time difference between the sending time of the ADS-B data and the recording time of the last recorded ADS-B data of the flight target meets a preset condition in the initial ADS-B data as the target ADS-B data.

[0046] According to an embodiment of the present invention, due to the delay in the transmission of ADS-B data, the matched data needs to be screened again. For example, the data with a time difference of one second between the time of sending ADS-B and the time of the last recorded ADS-B of the flight target is retained. By this method, the reliability of the matched data can be maximized.

[0047] It should be noted that during the imaging process of a remote sensing satellite, the probability of capturing a flying target during flight is very low. Therefore, it is extremely difficult to find a flying target in a remote sensing image. On the one hand, the amount of remote sensing image data is huge, and it is impossible to determine one by one whether a flying target is contained in a scene image. On the other hand, the number of pixels in a remote sensing image is generally in the tens of millions, but the flying target has only dozens of pixels in the image. Therefore, it is very difficult to determine whether a flying target exists in the image. Matching the ADS-B data of the flying target with the remote sensing image well solves this problem. Using information such as the shooting time and coverage area of the remote sensing image to match the ADS-B data, the image with a successful match contains the flying target. Then, use the position information in the corresponding ADS-B data to locate the image. However, there is a slight delay in the ADS-B data, and the specific position of the flying target often deviates from the position provided by the ADS-B. Calculate the speed of the flying target in the image based on the speed in the ADS-B data and the speed of the satellite, and calculate the offset pixel range according to the image resolution. Then, use image processing methods to determine the specific position of the flying target.

[0048] In some embodiments, determining a candidate region in a remote sensing image based on a first motion speed and target ADS-B data includes:

[0049] Obtain the actual motion speed and actual position of the flying target recorded from the target ADS-B data. Determine the second motion speed of the flying target in the remote sensing image based on the first motion speed and the actual motion speed. Determine the offset distance of the flying target based on the second motion speed and the time difference between the sending time of the ADS-B data and the recording time of the last recorded ADS-B data of the flying target. Determine the implementation candidate region based on the offset distance and the actual position.

[0050] In some embodiments, the second motion speed of the flying target in the remote sensing image is equal to the sum speed of the first motion speed and the actual motion speed

[0051] Figure 2 Schematically shows a vector diagram of the satellite speed, the moving speed of the flying target in the image, and the true speed of the flying target according to an embodiment of the present invention.

[0052] According to an embodiment of the present invention, due to the existence of the delay, the flying target also continues to move during the transmission of the ADS-B data, resulting in a deviation between the actual position of the flying target and the position recorded in the ADS-B data. Therefore, locating according to the position recorded in the ADS-B data in the remote sensing image cannot find the flying target. In addition, during the satellite imaging process, the satellite also continues to move, which leads to a deviation between the moving speed of the flying target in the image and the actual flying speed.

[0053] As shown Figure 2 in the figure, the moving speed of the flying object in the image is the resultant speed of the actual moving speed of the flying object and the moving speed of the satellite. Among them, represents the moving speed of the flying object in the image, represents the actual speed of the flying object, represents the speed of the satellite. According to the velocity geometric relationship, it can be obtained that:

[0054]

[0055] Among them, the ADS-B data contains the actual flying speed and direction of the flying object, the moving speed of the satellite is known, and the moving direction can be calculated. Due to the influence of the earth's curvature, the angle of the satellite's motion is related to the latitude of the location, and the specific expression can be as follows:

[0056]

[0057] Among them, represents the inclination angle of the satellite orbit, represents the latitude.

[0058] Substitute the actual flying speed of the flying object in the ADS-B data and the moving speed of the satellite into the velocity geometric relationship formula, and the speed and direction of the flying object moving in the image can be obtained. Since the ADS-B data is screened once, only the data with a delay within one second is retained. Therefore, the offset distance can be calculated according to the speed and direction of the flying object moving in the image, and the range of the search area can be determined according to the offset distance.

[0059] In some embodiments, contour extraction is performed on the candidate region to determine the position of the flying object in the remote sensing image, including:

[0060] Based on a preset pixel threshold, the pixel values of each pixel point in the candidate region are processed to obtain a first image.

[0061] The pixels in the first image are divided into regions to obtain a second image, where the second image only contains the pixels of the flying object.

[0062] According to the embodiments of the present invention, by analyzing the characteristics of the flying object in the image, it is found that: the flying object in the air has a relatively large pixel intensity in each image. Therefore, the present invention designs a target extraction method based on contour extraction. This method includes two parts: adaptive threshold segmentation and region growing algorithm. By performing adaptive threshold segmentation on the image of the candidate region to obtain a first image, and then performing region growing based on the first image, a second image containing only the pixels of the flying object can be obtained.

[0063] In some embodiments, processing the pixel values of each pixel point in the candidate region based on a preset pixel threshold to obtain a first image includes:

[0064] Determining a preset pixel threshold based on the pixel values of each pixel in the candidate region.

[0065] Setting the pixel values less than the preset pixel threshold in the candidate region to zero to obtain the first image.

[0066] According to the embodiments of the present invention, the flying target has an obvious brightness advantage relative to the background. Therefore, the target can be effectively extracted by the threshold method. For the same multi-spectral image, the reflection characteristics of different bands are different from each other. For different multi-spectral images, the imaging conditions are different, and the imaging results will also be different. Therefore, there is no fixed threshold applicable to all bands of all images. Based on this, for example, the pixel intensity at the 99th percentile of each image can be used as the threshold. Pixels greater than this threshold are retained, and pixels less than this threshold are set to 0. After processing by this method, most of the background interference can be removed.

[0067] Figure 3 Schematically shows the comparison results of the images before and after processing by the adaptive threshold method according to the embodiments of the present invention.

[0068] As Figure 3 shown, Figure 3 in, a and c are the original images corresponding to the candidate region, and b and d are the images after processing by the adaptive threshold method. For some pixels with relatively strong reflection (such as the wake of an airplane), the adaptive threshold segmentation method cannot remove them. Therefore, further processing is required to remove the remaining interfering pixels.

[0069] In some embodiments, partitioning the pixels in the first image to obtain a second image includes:

[0070] Determining the pixel point with the largest pixel value in the first image as the reference pixel point.

[0071] Starting from the reference pixel point, merging the pixel points with non-zero pixel values around the reference pixel point into a target region, and the target region is the position of the flying target in the remote sensing image.

[0072] According to an embodiment of the present invention, the region-growing based segmentation method is an image segmentation technique whose objective is to divide pixels into regions or objects with similar properties. This method starts from seed points selected in the image and gradually merges adjacent pixels into a region until certain predetermined stopping criteria are met. Through analysis, it can be found that the pixels with the maximum pixel intensity in the slice often belong to the flying target. Therefore, during specific segmentation, the pixel with the maximum pixel intensity is used as the seed point, and the surrounding pixels are added to the queue. If the surrounding pixels are not zero, they are regarded as target pixels, and the pixels around these pixels are added to the queue until no more pixels can be added.

[0073] Figure 4 Schematically shows the comparison results of the images before and after processing by the region-growing algorithm according to an embodiment of the present invention.

[0074] As Figure 4 shown, Figure 4 in which a and c are the images processed by the adaptive threshold method, and b and d are the images processed by the region-growing algorithm. It can be seen from Figure 4 that the final result obtained from the image processed by the region-growing algorithm only contains the pixels of the airplane.

[0075] Based on the above-mentioned flight target detection method using ADS-B data and remote sensing images, an embodiment of the present invention further provides a flight target detection device based on ADS-B data and remote sensing images.

[0076] Figure 5 Schematically shows the structural block diagram of the flight target detection device based on ADS-B data and remote sensing images according to an embodiment of the present invention.

[0077] As Figure 5 shown, the flight target detection device 500 for ADS-B data and remote sensing images in this embodiment may include a first acquisition module 510, a second acquisition module 520, a candidate region determination module 530, and a position determination module 540.

[0078] The first acquisition module 510 is configured to acquire the remote sensing image of the target area, the imaging time and imaging range of the remote sensing image, and the first movement speed of the remote sensing satellite.

[0079] The second acquisition module 520 is configured to acquire target ADS-B data from the pre-stored ADS-B data based on the imaging time and imaging range.

[0080] The candidate region determination module 530 is configured to determine a candidate region in the remote sensing image based on the first movement speed and the target ADS-B data, and the candidate region contains the flying target.

[0081] A position determination module 540, configured to extract the contour of a candidate region and determine the position of the flying target in the remote sensing image.

[0082] According to embodiments of the present invention, any plurality of modules, sub-modules, units, and sub-units, or at least part of the functions of any of them can be implemented in one module. Any one or more of the modules, sub-modules, units, and sub-units according to embodiments of the present invention can be split into multiple modules for implementation. Any one or more of the modules, sub-modules, units, and sub-units according to embodiments of the present invention can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by any other reasonable way of integrating or packaging circuits, or implemented in any one of the three implementation manners of software, hardware, and firmware, or in an appropriate combination of any several of them. Alternatively, one or more of the modules, sub-modules, units, and sub-units according to embodiments of the present invention can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding functions can be executed.

[0083] For example, any plurality of the first acquisition module 510, the second acquisition module 520, the candidate region determination module 530, and the position determination module 540 can be combined and implemented in one module / unit / sub-unit, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to embodiments of the present invention, at least one of the first acquisition module 510, the second acquisition module 520, the candidate region determination module 530, and the position determination module 540 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by any other reasonable way of integrating or packaging circuits, or implemented in any one of the three implementation manners of software, hardware, and firmware, or in an appropriate combination of any several of them. Alternatively, at least one of the first acquisition module 510, the second acquisition module 520, the candidate region determination module 530, and the position determination module 540 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding functions can be executed.

[0084] It should be noted that the device part in the embodiments of the present invention corresponds to the method part in the embodiments of the present invention, and their specific implementation details are the same, so they will not be elaborated here.

[0085] Figure 6 Schematically shows a structural block diagram of an airspace supervision system according to an embodiment of the present invention.

[0086] As Figure 6 shown, the satellite-ground collaborative aviation monitoring system 600 of this embodiment may include a remote sensing satellite constellation 610 and a dynamic airspace situation visualization platform 620.

[0087] The remote sensing satellite constellation 610 is configured to acquire remote sensing images of a target area, the imaging time and imaging range of the remote sensing images, and the first movement speed of the remote sensing satellites.

[0088] The dynamic airspace situation visualization platform 620 is configured to obtain target ADS-B data from pre-stored ADS-B data based on the imaging time and imaging range; determine candidate areas in the remote sensing images based on the first movement speed and the target ADS-B data, where the candidate areas contain flight targets; extract the contours of the candidate areas, determine the positions of the flight targets in the remote sensing images, and convert the positions of the flight targets into a track heat map including ADS-B data verification identifiers.

[0089] It should be noted that the system part in the embodiments of the present invention corresponds to the method part in the embodiments of the present invention, and their specific implementation details are the same, so they will not be elaborated here.

[0090] Figure 7 Schematically shows a block diagram of an electronic device suitable for implementing a flight target detection method according to an embodiment of the present invention.

[0091] As Figure 7 shown, the electronic device 700 according to an embodiment of the present invention includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general microprocessor (such as a CPU), an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (such as an application specific integrated circuit (ASIC)), etc. The processor 701 may also include on-board memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0092] In the RAM 703, various programs and data required for the operation of the electronic device 700 are stored. The processor 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. The processor 701 performs various operations of the method flow according to the embodiments of the present invention by executing the programs in the ROM 702 and / or the RAM 703. It should be noted that the programs may also be stored in one or more memories other than the ROM 702 and the RAM 703. The processor 701 may also perform various operations of the method flow according to the embodiments of the present invention by executing the programs stored in the one or more memories.

[0093] According to an embodiment of the present invention, the electronic device 700 may further include an input / output (I / O) interface 705, and the input / output (I / O) interface 705 is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the I / O interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed so that a computer program read from it can be installed into the storage portion 708 as needed.

[0094] The present invention also provides a computer-readable storage medium, which may be included in the device / device / system described in the above embodiments; or may exist separately without being assembled into the device / device / system.

[0095] In one embodiment, the computer-readable storage medium is disposed at the on-orbit edge, and executable instructions are stored thereon. When the instructions are executed by a processor, the processor is caused to implement the following method: obtaining a remote sensing image of a target area, the imaging time and imaging range of the remote sensing image, and the first movement speed of the remote sensing satellite; obtaining target ADS-B data from pre-stored ADS-B data based on the imaging time and imaging range; determining a candidate area in the remote sensing image based on the first movement speed and the target ADS-B data, the candidate area including a flight target; and performing contour extraction on the candidate area to determine the position of the flight target in the remote sensing image. For specific details, please refer to the foregoing method section and will not be elaborated here.

[0096] The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiments of the present invention is implemented.

[0097] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: portable computer disks, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present invention, the computer-readable storage medium may be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or apparatus. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the above-described ROM 702 and / or RAM 703 and / or one or more memories other than ROM702 and RAM 703.

[0098] An embodiment of the present invention further includes a computer program product, which includes a computer program that contains program code for executing the method shown in the flowchart. When the computer program product runs in a computer system, the program code is used to enable the computer system to implement the processing method provided by the embodiment of the present invention.

[0099] When the computer program is executed by the processor 701, it executes the above-described functions defined in the system / apparatus of the embodiment of the present invention. According to an embodiment of the present invention, the above-described systems, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0100] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and be downloaded and installed through the communication part 709, and / or be installed from the removable medium 711. The program code included in the computer program can be transmitted by any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0101] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or be installed from the removable medium 711. When the computer program is executed by the processor 701, it executes the above-described functions defined in the system of the embodiment of the present invention. According to an embodiment of the present invention, the above-described systems, devices, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0102] According to embodiments of the present invention, program code for executing the computer programs provided by the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The programming languages include, but are not limited to, such as Java, C++, Python, the "C" language, or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (for example, by using an Internet service provider to connect through the Internet).

[0103] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0104] Those skilled in the art can understand that the features recited in the various embodiments and / or claims of the present invention can be combined or combined in various ways, even if such combinations or combinations are not explicitly recited in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features recited in the various embodiments and / or claims of the present invention can be combined and combined in various ways. All such combinations and / or combinations fall within the scope of the present invention.

[0105] The embodiments of the present invention have been described above. However, these embodiments are merely for illustrative purposes and not for limiting the scope of the present invention. Although the embodiments have been described separately above, this does not mean that the measures in each embodiment cannot be used advantageously in combination. The scope of the present invention is defined by the appended claims and their equivalents. Without departing from the scope of the present invention, those skilled in the art can make various substitutions and modifications, and these substitutions and modifications should all fall within the scope of the present invention.

Claims

1. A flight target detection method based on ADS-B data and remote sensing images, characterized in that including: obtaining a remote sensing image of a target area, the imaging time and imaging range of the remote sensing image, and a first motion speed of a remote sensing satellite; obtaining target ADS-B data from pre-stored ADS-B data based on the imaging time and the imaging range; determining a candidate area in the remote sensing image based on the first motion speed and the target ADS-B data, the candidate area including a flight target; performing contour extraction on the candidate area to determine the position of the flight target in the remote sensing image.

2. The method according to claim 1, wherein The obtaining target ADS-B data from pre-stored ADS-B data based on the imaging time and the imaging range includes: obtaining initial ADS-B data associated with a flight target in the remote sensing image from pre-stored ADS-B data based on the imaging time and the imaging range; determining, as the target ADS-B data, the initial ADS-B data in which the time difference between the transmission time of the ADS-B data and the recording time of the last recorded ADS-B data of the flight target meets a preset condition.

3. The method according to claim 1 or 2, characterized in that, The determining the candidate area in the remote sensing image based on the first motion speed and the target ADS-B data includes: obtaining the actual motion speed and actual position of the flight target recorded from the target ADS-B data; determining a second motion speed of the flight target in the remote sensing image based on the first motion speed and the actual motion speed; determining an offset distance of the flight target based on the second motion speed and the time difference between the transmission time of the ADS-B data and the recording time of the last recorded ADS-B data of the flight target; determining an implementation candidate area based on the offset distance and the actual position.

4. The method according to claim 3, characterized in that, The second motion speed of the flight target in the remote sensing image is equal to the sum speed of the first motion speed and the actual motion speed.

5. The method according to claim 1, characterized in that, The performing contour extraction on the candidate area to determine the position of the flight target in the remote sensing image includes: processing the pixel values of each pixel point in the candidate area based on a preset pixel threshold to obtain a first image; performing region division on the pixels in the first image to obtain a second image, where the second image only includes the pixels of the flight target.

6. The method according to claim 5, wherein The processing the pixel values of each pixel point in the candidate area based on a preset pixel threshold to obtain a first image includes: determining a preset pixel threshold based on the pixel values of each pixel in the candidate area; setting the pixel values less than the preset pixel threshold in the candidate area to zero to obtain the first image.

7. The method according to claim 5 or 6, characterized in that, The performing region division on the pixels in the first image to obtain a second image includes: determining a reference pixel point as the pixel point with the largest pixel value in the first image; starting from the reference pixel point, merging the pixel points with non-zero pixel values around the reference pixel point into a target area, and the target area is the position of the flight target in the remote sensing image.

8. An aircraft target detection device based on ADS-B data and remote sensing images, characterized in that, including: A first acquisition module, configured to acquire a remote sensing image of a target area, the imaging time and imaging range of the remote sensing image, and the first movement speed of a remote sensing satellite; A second acquisition module, configured to acquire target ADS-B data from pre-stored ADS-B data based on the imaging time and the imaging range; A candidate area determination module, configured to determine a candidate area in the remote sensing image based on the first movement speed and the target ADS-B data, the candidate area containing a flight target; A position determination module, configured to perform contour extraction on the candidate area to determine the position of the flight target in the remote sensing image.

9. A space-ground collaborative aviation monitoring system, characterized in that, Including: A remote sensing satellite constellation, configured to acquire a remote sensing image of a target area, the imaging time and imaging range of the remote sensing image, and the first movement speed of a remote sensing satellite; A dynamic airspace situation visualization platform, configured to acquire target ADS-B data from pre-stored ADS-B data based on the imaging time and the imaging range; determine a candidate area in the remote sensing image based on the first movement speed and the target ADS-B data, the candidate area containing a flight target; perform contour extraction on the candidate area to determine the position of the flight target in the remote sensing image, and convert the position of the flight target into a track heat map including an ADS-B data verification identifier.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is disposed on the on-board edge, and has executable instructions stored thereon, and when the instructions are executed by a processor, the processor implements the following method: Acquire a remote sensing image of a target area, the imaging time and imaging range of the remote sensing image, and the first movement speed of a remote sensing satellite; Acquire target ADS-B data from pre-stored ADS-B data based on the imaging time and the imaging range; Determine a candidate area in the remote sensing image based on the first movement speed and the target ADS-B data, the candidate area containing a flight target; Perform contour extraction on the candidate area to determine the position of the flight target in the remote sensing image.

Images