Flying target detection method, device, system and medium based on ADS-B data and remote sensing image

Abstract

The present invention provides a method, device, system, and medium for detecting flying targets based on ADS-B data and remote sensing images, relating to the fields of remote sensing image applications and target detection technology. The target detection method includes: acquiring a remote sensing image of a target area, the imaging time and imaging range of the remote sensing image, and a first motion velocity of the remote sensing satellite; obtaining target ADS-B data from pre-stored ADS-B data based on the imaging time and imaging range; determining a candidate region in the remote sensing image based on the first motion velocity and the target ADS-B data, wherein the candidate region contains the flying target; and performing contour extraction on the candidate region to determine the flying target's position in the remote sensing image.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing image application and target detection technology, and in particular to a flying target detection method, device, system and medium based on ADS-B data and remote sensing images. Background Art

[0002] With the vigorous development of the aviation industry, aircraft are increasingly used in human life. How to effectively and accurately obtain the motion parameters of aerial targets is of vital importance in fields such as aviation.

[0003] Methods currently exist for estimating the motion parameters of flying targets. These include methods for calculating the altitude and velocity of moving aerial targets using field-of-view spectroscopic hyperspectral imaging, and methods for analyzing moving aerial targets in sea and air environments using hyperspectral technology. While these methods estimate the motion parameters of flying targets using hyperspectral satellite imagery, they fail to verify the estimated results against the actual motion parameters. Summary of the Invention

[0004] In view of this, the present invention provides a flying target detection method, device, system and medium based on ADS-B data and remote sensing images, which are used to at least partially solve the above technical problems.

[0005] A first aspect of an embodiment of the present invention provides a flying target detection method based on ADS-B data and remote sensing images, including: obtaining a remote sensing image of a target area, the imaging time and imaging range of the remote sensing image, and a first motion speed of the remote sensing satellite; obtaining target ADS-B data from pre-stored ADS-B data based on the imaging time and imaging range; determining a candidate area in the remote sensing image based on the first motion speed and the target ADS-B data, the candidate area containing the flying target; and performing contour extraction on the candidate area to determine the position of the flying target in the remote sensing image.

[0006] According to an embodiment of the present invention, target ADS-B data is obtained from pre-stored ADS-B data based on imaging time and imaging range, including: obtaining initial ADS-B data associated with a flight target in a remote sensing image from pre-stored ADS-B data based on imaging time and imaging range; and determining the initial ADS-B data, in which the time difference between the sending time of the ADS-B data and the recording time of the last recording of the flight target ADS-B data satisfies a preset condition, as target ADS-B data.

[0007] According to an embodiment of the present invention, determining a candidate area in a remote sensing image based on a first motion speed and target ADS-B data includes: obtaining the actual motion speed and actual position of the recorded flying target from the target ADS-B data; determining a second motion speed of the flying target in the remote sensing image based on the first motion speed and the actual motion speed; determining an offset distance of the flying target based on the second motion speed and the time difference between the sending time of the ADS-B data and the recording time of the last recording of the flying target ADS-B data; and determining an implementation candidate area based on the offset distance and the actual position.

[0008] According to an embodiment of the present invention, the second motion speed of the flying target in the remote sensing image is equal to the sum of the first motion speed and the actual motion speed.

[0009] According to an embodiment of the present invention, contour extraction is performed on a candidate area to determine the position of a flying target in a remote sensing image, including: processing the pixel value of each pixel point in the candidate area based on a preset pixel threshold to obtain a first image; and dividing the pixels in the first image into regions to obtain a second image, wherein the second image only contains pixels of the flying target.

[0010] According to an embodiment of the present invention, the pixel value of each pixel point in the candidate area is processed based on a preset pixel threshold to obtain a first image, including: determining the preset pixel threshold based on the pixel value of each pixel in the candidate area; setting the pixel value in the candidate area that is less than the preset pixel threshold to zero to obtain the first image.

[0011] According to an embodiment of the present invention, pixels in a first image are divided into regions to obtain a second image, including: determining the pixel point with the largest pixel value in the first image as a reference pixel point; taking the reference pixel point as the starting point, merging the pixel points with non-zero pixel values ​​around the reference pixel point into a target region, where the target region is the position of the flying target in the remote sensing image.

[0012] The second aspect of the present invention provides a flying target detection device based on ADS-B data and remote sensing images, including: a first acquisition module, used to acquire a remote sensing image of a target area, the imaging time and imaging range of the remote sensing image, and a first motion speed of the remote sensing satellite; a second acquisition module, used to acquire target ADS-B data from pre-stored ADS-B data based on the imaging time and imaging range; a candidate area determination module, used to determine a candidate area in the remote sensing image based on the first motion speed and the target ADS-B data, the candidate area containing the flying target; and a position determination module, used to perform contour extraction on the candidate area to determine the position of the flying target in the remote sensing image.

[0013] The third aspect of the present invention provides a satellite-ground collaborative aviation monitoring system, comprising: a remote sensing satellite constellation, configured to obtain remote sensing images of a target area, imaging time and imaging range of the remote sensing images, and a first motion speed of the remote sensing satellite; a dynamic airspace situation visualization platform, configured to obtain target ADS-B data from pre-stored ADS-B data based on the imaging time and imaging range; determining a candidate area in the remote sensing image based on the first motion speed and the target ADS-B data, the candidate area containing a flying target; performing contour extraction on the candidate area, determining the position of the flying target in the remote sensing image, and converting the position of the flying target into a track heat map containing an ADS-B data verification mark.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium, which is arranged at the edge of the satellite and stores executable instructions. When the instructions are executed by a processor, the processor implements the following method: obtaining a remote sensing image of a target area, the imaging time and imaging range of the remote sensing image, and a first motion speed of the remote sensing satellite; obtaining target ADS-B data from pre-stored ADS-B data based on the imaging time and imaging range; determining a candidate area in the remote sensing image based on the first motion speed and the target ADS-B data, the candidate area containing the flying target; performing contour extraction on the candidate area to determine the position of the flying target in the remote sensing image.

[0015] A fifth aspect of the present invention provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above method.

[0016] The flying target detection method, device, system, and medium based on ADS-B data and remote sensing images according to the embodiments of the present invention have at least the following beneficial effects:

[0017] By matching Automatic Dependent Surveillance-Broadcast (ADS-B) data using information such as the shooting time and coverage area of ​​remote sensing images, we can quickly screen remote sensing images containing flying targets and the corresponding ADS-B data of the flying targets. Then, we use the position information in the corresponding ADS-B data to locate the flying targets in the remote sensing images. This can reduce the time and workload of searching for aerial flying targets in remote sensing images and improve the efficiency of target detection.

[0018] The speed of the flying target in the image is calculated using the speed in the ADS-B data and the speed of the satellite, and the offset pixel range is calculated based on the image resolution to determine the search range. This can solve the problem of inaccurate positioning caused by the propagation delay of ADS-B data, improve the accuracy of target detection, and provide accurate values ​​for verifying the estimated results of motion parameters.

[0019] By using the imaging time and imaging area in remote sensing images to filter and match ADS-B data, not only is the efficiency of data processing improved, but also satellite images can be used to fill in the gaps in flight monitoring in areas lacking ground base station support. This is of great significance for improving the coverage capability and response efficiency of the global aviation monitoring system. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0021] Figure 1 The flowchart of the flying target detection method based on ADS-B data and remote sensing images according to an embodiment of the present invention is schematically shown;

[0022] Figure 2 A vector diagram schematically illustrates the base satellite velocity, the moving velocity of the flying target in the image, and the true velocity of the flying target according to an embodiment of the present invention;

[0023] Figure 3 Schematically showing the image comparison results before and after processing using the adaptive threshold method according to an embodiment of the present invention;

[0024] Figure 4 Schematically showing image comparison results before and after processing by a region growing algorithm according to an embodiment of the present invention;

[0025] Figure 5 Schematically shows a structural block diagram of a flying target detection device based on ADS-B data and remote sensing images according to an embodiment of the present invention;

[0026] Figure 6 The following schematically shows a structural block diagram of an airspace monitoring system according to an embodiment of the present invention;

[0027] Figure 7 The following schematically shows a block diagram of an electronic device suitable for implementing a flying target detection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concept of the present invention.

[0029] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the presence of the features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0030] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0031] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0032] The ADS-B system is a new air traffic control surveillance technology based on the Global Positioning System (GPS) that utilizes air-to-ground and air-to-air data links for traffic monitoring and information transmission. Requiring no manual operation or query, the system automatically (once per second) acquires parameters from relevant airborne equipment and broadcasts the target's position, altitude, speed, heading, identification number, and other information to other aircraft or ground stations, allowing air traffic controllers to monitor the target's status. ADS-B data can accurately capture the target's flight parameters, but remote sensing images only contain information about the target's appearance and do not include the target's ADS-B data.

[0033] To address this issue, methods combining ADS-B data with remote sensing imagery have emerged. Remote sensing imagery can provide extensive visual information about the flight area. By combining ADS-B data with remote sensing imagery, the position of flying targets can be accurately identified and tracked when ADS-B signals are insufficient or visual confirmation is required. Furthermore, this method can enhance the reliability and effectiveness of flight monitoring in specific application scenarios, such as border monitoring, environmental monitoring, or search and rescue operations. Some methods combining ADS-B data with remote sensing imagery primarily utilize low-orbit satellites to rapidly retrieve, locate, and identify anomalous aerial targets within a large area. However, these methods rely on satellite-borne ADS-B data, severely limiting their application scenarios. Other methods project sampled ADS-B data into image information and detect ADS-B route information spoofing by detecting structural similarities between images. However, these methods rely solely on ADS-B data and fail to fully utilize the image information provided by remote sensing imagery.

[0034] In view of this, an embodiment of the present invention provides a flying target detection method based on ADS-B data and remote sensing images, which fuses the remote sensing images and the ADS-B data of the flying targets, and uses the imaging information of the remote sensing images to match massive amounts of ADS-B data, thereby avoiding manual screening of remote sensing images and ADS-B data; by analyzing the relationship between satellite motion and flying target flight, the problem of inaccurate positioning caused by ADS-B data propagation delay is solved, and the search time is greatly shortened.

[0035] Figure 1 The flowchart of the flying target detection method based on ADS-B data and remote sensing images according to an embodiment of the present invention is schematically shown.

[0036] like Figure 1 As shown, the flying target detection method based on ADS-B data and remote sensing images of this embodiment may include operations S110 to S140.

[0037] In operation S110 , a remote sensing image of a target area, an imaging time and an imaging range of the remote sensing image, and a first moving speed of a remote sensing satellite are acquired.

[0038] In operation S120 , target ADS-B data is acquired from pre-stored ADS-B data based on an imaging time and an imaging range.

[0039] In operation S130 , a candidate region in the remote sensing image is determined based on the first motion speed and the target ADS-B data, where the candidate region includes the flying target.

[0040] In operation S140 , contour extraction is performed on the candidate region to determine the position of the flying target in the remote sensing image.

[0041] According to an embodiment of the present invention, the target area can be some remote areas, blank areas between ground base stations, or areas of interest. For example, qualified remote sensing images can be collected based on the area of ​​interest and time range, and the scene number (sceneID), task number (jobTaskid), satellite number (satelliteID), imaging start time (sceneStartTime), imaging end time (sceneEndTime), cloud cover (cloudCover), and imaging range (spatialData) information of each remote sensing image can be extracted.

[0042] According to an embodiment of the present invention, the imaging time and imaging range of each remote sensing image can be used to filter pre-stored ADS-B data. ADS-B data that meets the requirements returns information such as the flight target's identification code, longitude and latitude, flight speed, azimuth, altitude, time the ADS-B was sent, and the time the last ADS-B of the flight target was recorded. The OpenSky Application Programming Interface (API) can be called to filter the ADS-B data in the database. OpenSky is an open aviation data platform operated by the OpenSky Network. OpenSky has collected more than 30 trillion messages from more than 6,000 sensors worldwide. This makes OpenSky the largest aviation traffic surveillance data repository of its kind. Researchers and developers can access this data through various means, including APIs and the Impala database.

[0043] In some embodiments, acquiring target ADS-B data from pre-stored ADS-B data based on imaging time and imaging range may include:

[0044] Initial ADS-B data associated with a flight target in a remote sensing image is acquired from pre-stored ADS-B data based on imaging time and imaging range.

[0045] The initial ADS-B data whose time difference between the sending time of the ADS-B data and the recording time of the last recorded flight target ADS-B data meets the preset conditions is determined as the target ADS-B data.

[0046] According to an embodiment of the present invention, due to the transmission delay of ADS-B data, it is necessary to re-screen the matched data. For example, data with a one-second difference between the time the ADS-B was sent and the time the last ADS-B of the flight target was recorded is retained. This method can maximize the reliability of the matched data.

[0047] It should be noted that the probability of capturing a flying target during remote sensing satellite imaging is extremely small, making it extremely difficult to locate a flying target within remote sensing images. Firstly, the sheer volume of remote sensing imagery makes it impossible to individually determine whether a flying target is present in a scene. Secondly, remote sensing images typically contain tens of millions of pixels, while flying targets occupy only a few dozen pixels within the image, making it difficult to determine their presence. Matching the flying target's ADS-B data with remote sensing images effectively addresses this issue. This approach leverages information such as the acquisition time and coverage area of ​​the remote sensing image to match the ADS-B data. Images with successful matches contain the flying target. The image is then located using the corresponding location information in the ADS-B data. However, ADS-B data has a slight delay, so the target's exact location often deviates from the location provided by the ADS-B data. The target's velocity in the image is calculated using the velocity in the ADS-B data and the satellite's velocity. The offset pixel range is then calculated based on the image resolution. Image processing methods are then used to determine the target's exact location.

[0048] In some embodiments, determining a candidate area in a remote sensing image based on the first motion speed and the target ADS-B data includes:

[0049] The actual speed and actual position of the target are obtained from the target's ADS-B data. A second speed of the target in the remote sensing image is determined based on the first speed and the actual speed. An offset distance of the target is determined based on the second speed and the time difference between the time the ADS-B data was sent and the time the last ADS-B data of the target was recorded. A candidate implementation area is determined based on the offset distance and the actual position.

[0050] In some embodiments, the second motion speed of the flying target in the remote sensing image is equal to the sum of the first motion speed and the actual motion speed.

[0051] Figure 2 A vector diagram schematically illustrates the base satellite velocity, the moving velocity of the flying target in the image, and the true velocity of the flying target according to an embodiment of the present invention.

[0052] According to embodiments of the present invention, due to latency, the target continues to move during ADS-B data transmission, resulting in a discrepancy between the target's actual location and the location recorded in the ADS-B data. Consequently, positioning the target in the remote sensing image based on the location recorded in the ADS-B data cannot locate the target. Furthermore, because the satellite is in continuous motion during imaging, this can cause a discrepancy between the target's speed in the image and its actual flight speed.

[0053] like Figure 2 As shown in the figure, the moving speed of the flying target in the image is the combined speed of the flying target's actual moving speed and the satellite's moving speed. Indicates the moving speed of the flying target in the image, Indicates the actual speed of the flying target. represents the speed of the satellite. According to the geometric relationship of speed, we can get:

[0054]

[0055] Among them, ADS-B data contains the actual flight speed and direction of the flying target. The speed of the satellite is known, and the direction of movement can be calculated. Due to the influence of the curvature of the earth, the angle of the satellite movement It is related to the latitude of the location. The specific expression can be as follows:

[0056]

[0057] in, represents the inclination angle of the satellite orbit, Indicates latitude.

[0058] Substituting the actual flight speed of the target in the ADS-B data and the satellite's velocity into the velocity geometry equation, the target's speed and direction within the image can be determined. Because the ADS-B data is filtered to retain only data delayed within one second, the offset distance can be calculated based on the target's speed and direction within the image, and the search area can be determined based on this offset distance.

[0059] In some embodiments, performing contour extraction on the candidate area to determine the position of the flying target in the remote sensing image includes:

[0060] The pixel value of each pixel point in the candidate area is processed based on a preset pixel threshold to obtain a first image.

[0061] The pixels in the first image are divided into regions to obtain a second image, wherein the second image only contains pixels of the flying target.

[0062] According to an embodiment of the present invention, by analyzing the characteristics of flying targets in images, it was found that flying targets in the air have high pixel intensities in all images. Therefore, the present invention has designed a target extraction method based on contour extraction. This method includes two parts: adaptive threshold segmentation and region growing. The method performs adaptive threshold segmentation on the image of the candidate region to obtain a first image. Then, region growing is performed on this first image to obtain a second image containing only the pixels of the flying target.

[0063] In some embodiments, processing the pixel value of each pixel point in the candidate area based on a preset pixel threshold to obtain the first image includes:

[0064] A preset pixel threshold is determined based on the pixel value of each pixel in the candidate area.

[0065] The pixel values ​​in the candidate area that are less than a preset pixel threshold are set to zero to obtain a first image.

[0066] According to an embodiment of the present invention, the flying target has a significant brightness advantage over the background, so the target can be effectively extracted by the threshold method. For the same multispectral image, the reflectance characteristics of different bands are different. For different multispectral images, the imaging conditions are different, and the imaging results will also be different. Therefore, there is no fixed threshold that is applicable to all bands of all images. Based on this, for example, the 99th percentile pixel intensity of each image can be used as the threshold, and pixels greater than the threshold are retained, and pixels less than the threshold are set to 0. After processing by this method, most of the background interference can be removed.

[0067] Figure 3 The figure schematically shows the image comparison results before and after processing using the adaptive threshold method according to an embodiment of the present invention.

[0068] like Figure 3 As shown, Figure 3 Figures a and c are the original images corresponding to the candidate regions, while images b and d are processed using the adaptive thresholding method. The adaptive thresholding method cannot remove some highly reflective pixels (such as airplane contrails). Therefore, further processing is required to remove the remaining interfering pixels.

[0069] In some embodiments, dividing pixels in the first image into regions to obtain the second image includes:

[0070] The pixel with the maximum pixel value in the first image is determined as the reference pixel.

[0071] Taking the reference pixel point as the starting point, the pixels with non-zero pixel values ​​around the reference pixel point are merged into the target area. The target area is the position of the flying target in the remote sensing image.

[0072] According to an embodiment of the present invention, a segmentation method based on region growing is an image segmentation technique whose goal is to divide pixels into regions or objects with similar properties. The method starts with a seed point selected in the image and gradually merges adjacent pixels into a region until certain predetermined stopping criteria are met. After analysis, it can be found that the pixel with the highest pixel intensity in the slice often belongs to the flying target. Therefore, during the specific segmentation, the pixel with the highest pixel intensity is used as the seed point, and the surrounding pixels are added to the queue. If the surrounding pixels are not zero, it is regarded as the target pixel, and the pixels around the pixel are added to the queue until there are no more pixels to be added.

[0073] Figure 4 The diagram schematically illustrates image comparison results before and after processing using a region growing algorithm according to an embodiment of the present invention.

[0074] like Figure 4 As shown, Figure 4 In the figure, a and c are images processed by the adaptive threshold method, and b and d are images processed by the region growing algorithm. Figure 4 It can be seen that the final result of the image processed by the region growing algorithm only contains the pixels of the aircraft.

[0075] Based on the above-mentioned flying target detection method based on ADS-B data and remote sensing images, an embodiment of the present invention further provides a flying target detection device based on ADS-B data and remote sensing images.

[0076] Figure 5 The structure block diagram of a flying target detection device based on ADS-B data and remote sensing images according to an embodiment of the present invention is schematically shown.

[0077] like Figure 5 As shown, the flying target detection device 500 for ADS-B data and remote sensing images of this embodiment may include a first acquisition module 510 , a second acquisition module 520 , a candidate region determination module 530 and a position determination module 540 .

[0078] The first acquisition module 510 is configured to acquire a remote sensing image of a target area, an imaging time and an imaging range of the remote sensing image, and a first motion speed of the remote sensing satellite.

[0079] The second acquisition module 520 is configured to acquire target ADS-B data from pre-stored ADS-B data based on the imaging time and imaging range.

[0080] The candidate region determining module 530 is configured to determine a candidate region in the remote sensing image based on the first motion speed and the target ADS-B data, where the candidate region includes the flying target.

[0081] The position determination module 540 is used to extract the contour of the candidate area and determine the position of the flying target in the remote sensing image.

[0082] Any number of the modules, submodules, units, and subunits according to embodiments of the present invention, or at least part of the functionality of any number of these units, can be implemented in a single module. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present invention can be split into multiple modules for implementation. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present invention can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented in hardware or firmware using any other reasonable method of circuit integration or packaging, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or any appropriate combination of any of these. Alternatively, one or more of the modules, submodules, units, and subunits according to embodiments of the present invention can be at least partially implemented as a computer program module that, when executed, can perform the corresponding functionality.

[0083] For example, any number of the first acquisition module 510, the second acquisition module 520, the candidate region determination module 530, and the location determination module 540 may be combined into a single module / unit / sub-unit, or any one of these modules / units / sub-units may be split into multiple modules / units / sub-units. Alternatively, at least part of the functionality of one or more of these modules / units / sub-units may be combined with at least part of the functionality of other modules / units / sub-units and implemented in a single module / unit / sub-unit. According to an embodiment of the present invention, at least one of the first acquisition module 510, the second acquisition module 520, the candidate region determination module 530, and the location determination module 540 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or may be implemented in hardware or firmware through any other reasonable means of circuit integration or packaging, or may be implemented in any one of software, hardware, and firmware, or any suitable combination of any of these. Alternatively, at least one of the first acquisition module 510 , the second acquisition module 520 , the candidate region determination module 530 , and the position determination module 540 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0084] It should be noted that the device part in the embodiment of the present invention corresponds to the method part in the embodiment of the present invention, and their specific implementation details are also the same, which will not be repeated here.

[0085] Figure 6 The structural block diagram of the airspace monitoring system according to an embodiment of the present invention is schematically shown.

[0086] like Figure 6 As shown, the satellite-ground collaborative aviation monitoring system 600 of this embodiment may include a remote sensing satellite constellation 610 and a dynamic airspace situation visualization platform 620.

[0087] The remote sensing satellite constellation 610 is configured to acquire a remote sensing image of a target area, an imaging time and an imaging range of the remote sensing image, and a first motion speed of the remote sensing satellite.

[0088] The dynamic airspace situation visualization platform 620 is configured to obtain target ADS-B data from pre-stored ADS-B data based on imaging time and imaging range; determine a candidate area in the remote sensing image based on the first motion speed and the target ADS-B data, where the candidate area contains the flight target; perform contour extraction on the candidate area to determine the position of the flight target in the remote sensing image, and convert the position of the flight target into a track heat map containing an ADS-B data verification mark.

[0089] It should be noted that the system part in the embodiment of the present invention corresponds to the method part in the embodiment of the present invention, and their specific implementation details are also the same, which will not be repeated here.

[0090] Figure 7 The following schematically shows a block diagram of an electronic device suitable for implementing a flying target detection method according to an embodiment of the present invention.

[0091] like Figure 7 As shown, an electronic device 700 according to an embodiment of the present invention includes a processor 701, which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 702 or programs loaded from a storage unit 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or related chipsets and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0092] Various programs and data required for the operation of the electronic device 700 are stored in the RAM 703. The processor 701, ROM 702, and RAM 703 are connected to each other via a bus 704. The processor 701 performs various operations according to the method flow of the embodiment of the present invention by executing the programs in the ROM 702 and / or RAM 703. It should be noted that the programs may also be stored in one or more memories other than the ROM 702 and RAM 703. The processor 701 may also perform various operations according to the method flow of the embodiment of the present invention by executing the programs stored in the one or more memories.

[0093] According to an embodiment of the present invention, electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to bus 704. Electronic device 700 may also include one or more of the following components connected to I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 708 including a hard disk; and a communication section 709 including a network interface card such as a LAN card or modem. Communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to I / O interface 705 as needed. Removable media 711, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 710 as needed, so that computer programs read from the removable media can be installed into storage section 708 as needed.

[0094] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system.

[0095] In one embodiment, a computer-readable storage medium is disposed at the edge of a satellite and stores executable instructions. When executed by a processor, these instructions cause the processor to implement the following method: obtaining a remote sensing image of a target area, the imaging time and imaging range of the remote sensing image, and a first motion velocity of the remote sensing satellite; obtaining target ADS-B data from pre-stored ADS-B data based on the imaging time and imaging range; determining a candidate region in the remote sensing image based on the first motion velocity and the target ADS-B data, wherein the candidate region contains a flying target; and performing contour extraction on the candidate region to determine the flying target's position in the remote sensing image. Detailed details are provided in the aforementioned method section and are not further elaborated here.

[0096] The computer-readable storage medium carries one or more programs. When the one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0097] According to an embodiment of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium, and may include, for example, but not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present invention, a computer-readable storage medium may include the ROM 702 and / or RAM 703 described above, and / or one or more memories other than ROM 702 and RAM 703.

[0098] The embodiments of the present invention further include a computer program product, which includes a computer program containing program code for executing the method shown in the flowchart. When the computer program product is executed in a computer system, the program code is used to cause the computer system to implement the processing method provided by the embodiments of the present invention.

[0099] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when the computer program is executed by the processor 701. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0100] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 709, and / or installed from a removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0101] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709 and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above-described functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the systems, devices, means, modules, units, etc. described above can be implemented by computer program modules.

[0102] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0103] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0104] Those skilled in the art will appreciate that various combinations and / or combinations of features described in the various embodiments and / or claims of the present invention may be made, even if such combinations and / or combinations are not explicitly described in the present invention. In particular, various combinations and / or combinations of features described in the various embodiments and / or claims of the present invention may be made, without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

[0105] The above describes embodiments of the present invention. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. The scope of the present invention is defined by the appended claims and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which are intended to fall within the scope of the present invention.

Claims

1. A flying target detection method based on ADS-B data and remote sensing images, characterized in that: include: Acquire a remote sensing image of a target area, an imaging time and an imaging range of the remote sensing image, and a first motion speed of the remote sensing satellite; acquiring target ADS-B data from pre-stored ADS-B data based on the imaging time and the imaging range; Acquire and record the actual movement speed and actual position of the flying target from the target ADS-B data; determine a second movement speed of the flying target in the remote sensing image based on the first movement speed and the actual movement speed; determine an offset distance of the flying target based on the second movement speed and a time difference between a transmission time of the ADS-B data and a last recording time of the ADS-B data of the flying target; determine a candidate area based on the offset distance and the actual position; the candidate area includes the flying target; Perform contour extraction on the candidate area to determine the position of the flying target in the remote sensing image.

2. The method according to claim 1, characterized in that The acquiring target ADS-B data from pre-stored ADS-B data based on the imaging time and the imaging range includes: Acquiring initial ADS-B data associated with the flight target in the remote sensing image from pre-stored ADS-B data based on the imaging time and the imaging range; The initial ADS-B data, for which the time difference between the sending time of the ADS-B data and the recording time of the last recording of the flight target ADS-B data satisfies a preset condition, is determined as the target ADS-B data.

3. The method according to claim 1, characterized in that The second motion speed of the flying target in the remote sensing image is equal to the sum of the first motion speed and the actual motion speed.

4. The method according to claim 1, wherein The step of extracting the contour of the candidate area to determine the position of the flying target in the remote sensing image includes: Processing the pixel value of each pixel point in the candidate area based on a preset pixel threshold to obtain a first image; Performing region division on pixels in the first image to obtain a second image, wherein the second image only contains pixels of the flying target.

5. The method according to claim 4, characterized in that The step of processing the pixel value of each pixel point in the candidate area based on a preset pixel threshold to obtain a first image includes: Determining a preset pixel threshold based on the pixel value of each pixel in the candidate area; The pixel values ​​in the candidate area that are less than the preset pixel threshold are set to zero to obtain the first image.

6. The method according to claim 4 or 5, characterized in that Dividing the pixels in the first image into regions to obtain the second image includes: Determine the pixel with the largest pixel value in the first image as a reference pixel; Taking the reference pixel point as a starting point, the pixel points with non-zero pixel values ​​around the reference pixel point are merged into a target area, and the target area is the position of the flying target in the remote sensing image.

7. A flying target detection device based on ADS-B data and remote sensing images, characterized in that: include: A first acquisition module is used to acquire a remote sensing image of a target area, an imaging time and an imaging range of the remote sensing image, and a first motion speed of the remote sensing satellite; A second acquisition module is configured to acquire target ADS-B data from pre-stored ADS-B data based on the imaging time and the imaging range; a candidate area determination module, configured to obtain the actual motion speed and actual position of the flying target recorded from the target ADS-B data; determine a second motion speed of the flying target in the remote sensing image based on the first motion speed and the actual motion speed; determine an offset distance of the flying target based on the second motion speed and a time difference between a transmission time of the ADS-B data and a last recording time of the ADS-B data of the flying target; and determine a candidate area based on the offset distance and the actual position; wherein the candidate area includes the flying target; The position determination module is used to extract the outline of the candidate area and determine the position of the flying target in the remote sensing image.

8. A satellite-ground collaborative aviation monitoring system, characterized in that: include: a remote sensing satellite constellation configured to acquire a remote sensing image of a target area, an imaging time and an imaging range of the remote sensing image, and a first motion speed of the remote sensing satellite; A dynamic airspace situation visualization platform is configured to obtain target ADS-B data from pre-stored ADS-B data based on the imaging time and the imaging range; obtain the actual movement speed and actual position of the recorded flight target from the target ADS-B data; determine the second movement speed of the flight target in the remote sensing image based on the first movement speed and the actual movement speed; determine the offset distance of the flight target based on the second movement speed and the time difference between the sending time of the ADS-B data and the recording time of the last recording of the ADS-B data of the flight target; determine a candidate area based on the offset distance and the actual position; the candidate area contains the flight target; perform contour extraction on the candidate area, determine the position of the flight target in the remote sensing image, and convert the position of the flight target into a track heat map containing an ADS-B data verification mark.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores executable instructions, which, when executed by a processor, cause the processor to implement the following method: Acquire a remote sensing image of a target area, an imaging time and an imaging range of the remote sensing image, and a first motion speed of the remote sensing satellite; acquiring target ADS-B data from pre-stored ADS-B data based on the imaging time and the imaging range; Obtaining the actual motion speed and actual position of the flying target from the target ADS-B data; determining a second motion speed of the flying target in the remote sensing image based on the first motion speed and the actual motion speed; determining an offset distance of the flying target based on the second motion speed and a time difference between a transmission time of the ADS-B data and a last recording time of the ADS-B data of the flying target; determining a candidate area based on the offset distance and the actual position; wherein the candidate area includes the flying target; Perform contour extraction on the candidate area to determine the position of the flying target in the remote sensing image.

Images