On-satellite real-time target pre-detection and intelligent fine detection fused information processing method and system, and medium

Through the information processing method that integrates real-time target pre-detection and intelligent precision detection on the satellite, the accuracy and false alarm problems of real-time target detection on the satellite are solved at ultra-high code speed, real-time processing and high-precision trajectory generation are realized, and real-time processing of the entire process and high-precision trajectory generation are improved. Satellite data timeliness and application performance.

CN119063732AActive Publication Date: 2024-12-03SHANGHAI SATELLITE ENG INST
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202411175753.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-12-03
Estimated Expiration
2044-08-26

AI Technical Summary

Technical Problem

Under ultra-high code rate conditions, there are difficulties in real-time processing of real-time targets on the satellite, such as high-precision low-fidelity alarm detection, trajectory fusion correlation and management, line of sight determination and fusion positioning, and the existing technology has not been effectively solved.

Method used

Using the information processing method of combining real-time target pre-detection and intelligent precision detection on-satellite real-time imaging data, pre-processing and judging the target position by receiving real-time imaging data of the camera, target pre-detection or intelligent precision detection, and generate target trajectory in combination with satellite position velocity and attitude data. , and perform binary or multi-star fusion positioning to output target three-dimensional trajectory and parameter estimation.

Benefits of technology

Real-time high-precision detection of targets on satellites at ultra-high code rates is achieved, virtual alarm point filtering is reduced, target detection accuracy and data processing efficiency are improved, and it is suitable for space-based information processing and satellite applications.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119063732A_ABST
    Figure CN119063732A_ABST
Patent Text Reader

Abstract

The invention provides an on-satellite real-time target pre-detection and intelligent fine detection fused information processing method and system, and a medium. The method comprises the following steps: S1, data preprocessing; s2, pre-detecting a target; s3, track association and management; s4, target intelligent fine detection is carried out; s5, determining the sight line of the track target; s6, self-service parameter control is carried out; and S7, performing target fusion positioning. According to the invention, target pre-detection is adopted to realize target real-time processing of camera superspeed and mass input data such as large area array and high resolution on a satellite, and intelligent fine detection and a unified trajectory management method are fused to further improve the target detection precision and increase the target trajectory detection length. The method is suitable for on-satellite real-time high-precision information processing under the space-based ultra-large detection data rate, the satellite-ground data transmission bandwidth can be effectively reduced, and the space-based information availability is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of real-time processing of space-based information, and in particular to an information processing method, system and medium for integrating on-board real-time target pre-detection and intelligent precision detection. Background Art

[0002] Real-time information processing on board is the key to the efficient application of space-based information. With the increase in the scale and spatial resolution of satellite payload detectors, the real-time code rate of some payloads has exceeded 40Gbps, and the imaging time is long. The massive amount of data generated has brought huge challenges to satellite-to-ground transmission and real-time processing on board. How to achieve high-precision and low false alarm detection of on-board targets under ultra-high code rate conditions, trajectory fusion association and management, line of sight determination and fusion positioning and other full-process real-time processing is the focus and difficulty of current space-based information processing and satellite application efficiency improvement.

[0003] Patent document CN115294439 discloses a method, system, device and storage medium for detecting weak moving targets in the air. It mainly uses multi-spectral data with imaging parallax to detect displacement parallax in the target image. It is a traditional moving target detection method and system, but does not involve intelligent target detection methods and real-time processing methods.

[0004] Patent document CN115497003A discloses a method for satellite edge computing lightweight and fast arbitrary-direction target detection, which mainly implements the on-orbit reasoning operation of the lightweight target detection model obtained through ground training on the edge processor with limited satellite resources, and can realize on-orbit real-time detection operation. The present invention is oriented to ultra-high code rate satellite payload data, and adopts the fusion of pre-detection and intelligent precision detection to solve the problem of on-board real-time information processing, and at the same time completes the on-orbit real-time generation of target angular trajectory and three-dimensional trajectory based on the detection results.

[0005] Patent document CN117036985A discloses a small target detection method and device for video satellite images, which mainly uses the YoloV5 model to train a small target detection model, solving the problem of unclear boundaries of small objects. However, it does not involve the processing procedures such as detection of moving targets, line of sight determination and fusion positioning.

[0006] Patent document CN117173039A discloses a real-time moving target detection noise removal system on satellite, which mainly uses the target detection results to perform geographic location projection and elevation estimation, and removes the noise introduced by the fixed point on the surface through the target position estimation and maintenance in the earth's fixed coordinate system, so as to avoid the situation where the real-time position of the target is lost due to the temporary loss of the image target. The present invention is a full-process space-based real-time information processing system including target image detection and tracking, trajectory association and management, line of sight determination and fusion positioning, which realizes false alarm point filtering and improves the target detection accuracy by combining target pre-detection with target intelligent precision detection, trajectory fusion association and management.

[0007] Patent document CN110647977A discloses a Tiny-YOLO network optimization method for onboard ship target detection, and patent document CN111598004B discloses a target detection and tracking method suitable for high-resolution remote sensing satellite video data. Both are target detection and tracking based on the Yolo model, and do not involve the target pre-detection, line of sight determination, fusion positioning and other processing processes in the present invention.

[0008] In order to cope with the current situation of increasing satellite payload data code rates, with the help of space applications of high-performance processors such as CPU, FPGA, DSP, and AI, it is necessary to develop on-board real-time information processing methods at ultra-high code rates, generate target information that users are concerned about, significantly reduce the amount of data transmitted between the satellite and the ground, and improve the timeliness of satellite data and the efficiency of information application. Summary of the invention

[0009] In view of the defects in the prior art, the purpose of the present invention is to provide an information processing method, system and medium for integrating on-board real-time target pre-detection and intelligent precision detection.

[0010] The information processing method for integrating on-board real-time target pre-detection and intelligent precise detection provided by the present invention comprises the following steps:

[0011] Step S1: receiving real-time imaging data from the camera and completing preprocessing in real time, including data frame extraction, data unpacking, image data caching, grayscale transformation, image registration and image difference;

[0012] Step S2: determine whether the target to be pre-inspected is located in the preset key observation area. If so, directly perform intelligent precision detection. If not, perform target pre-detection and output suspected targets and slices.

[0013] Step S3: receiving the target pre-detection or intelligent precise detection result, performing target image trajectory association and management, and determining whether to perform target intelligent precise detection, if yes, executing step S4, otherwise executing step S5;

[0014] Step S4: read in the target slice sequence to be precisely detected, perform intelligent precise detection of the target, output precise detection results, and perform trajectory association and management;

[0015] Step S5: Determine the target sight line by combining the satellite position speed and attitude data, generate the target angle trajectory and output the target image trajectory and target dynamic slice data;

[0016] Step S6: combining the target image trajectory and the target dynamic slice data, calculating and generating satellite payload imaging parameters and preprocessing parameters in real time, and completing adaptive satellite payload imaging and preprocessing;

[0017] Step S7: Receive the target angular trajectory obtained by other satellite detection, combine it with the target angular trajectory generated by the local satellite, perform target dual-star or multi-star fusion positioning, output the target three-dimensional trajectory, and estimate the target parameters.

[0018] Preferably, the step S1 comprises:

[0019] Step S1.1: Receive camera imaging data in real time, find the frame header and frame count, and complete data frame extraction in the form of code stream;

[0020] Step S1.2: completing data unpacking in real time, and generating camera imaging auxiliary data and image data sequence with imaging time and frame count as identifiers;

[0021] Step S1.3: Cache multiple frames of image data with preset intervals between the current frame image, generate an interval cache image sequence, and use it as input for subsequent image registration and image difference; cache multiple consecutive frames of image data before the current frame, generate a continuous cache image sequence, and use it as input for generating the target slice sequence to be precisely detected and the target dynamic slice data; the cache image data is updated once in real time when each frame of data is input;

[0022] Step S1.4: After inter-frame registration of the current frame image and the interval buffer image sequence, pixel by pixel difference is performed with the interval buffer image sequence to generate a differential image sequence.

[0023] Preferably, non-uniformity correction and grayscale transformation are performed on the current frame image and the differential image sequence to be subjected to target pre-detection;

[0024] Suppose the original DN value of the pixel in the i-th row and j-th column of the image is x ij , and its corresponding non-uniform correction parameter is k ij 、f ij , the corrected pixel DN value is y ij =k ij ·x ij +f ij ;

[0025] Perform image grayscale transformation on the image after non-uniformity correction, using linear stretching. If the pixel value before transformation is x, the pixel value after transformation is y, the expression is:

[0026]

[0027] Among them, a and b are prefabricated parameters and adaptive parameters respectively. The prefabricated parameters are fixed values ​​set on the ground; DN max is the maximum value of pixels.

[0028] Preferably, the step S2 comprises:

[0029] Step S2.1: Pre-detect the current frame image and the differential image sequence, and output the target point set {(m,n,y m,n )}, where m and n are the row and column numbers of the target pixel points detected in the current frame image; y m,n is the pixel value of m rows and n columns in the target point set;

[0030] The current frame image is X 0 , X 0-i, X 0-j, ,X 0-k is the differential image sequence with the current frame interval i, j, k frames, for the weighted image X = w 1 X 0 +w 2 X 0-i +w 2 X 0-j +w 3 X 0-k Median filtering and threshold segmentation are performed, and image pixels greater than the threshold are judged as target points to form a target point set {(m,n,ym,n)}; where w 1 ,w 2 ,w 3 ,w 4 is the image weight value, and its sum is 1;

[0031] Step S2.2, for the target point set {(m,n,y m,n )} Perform connected domain clustering calculation, and calculate the centroid of the subset of the clustered target point set. The row and column numbers of the centroid are used as the position of the detected target on the current frame image; all targets detected in the current frame image are numbered, and the target imaging time, frame count and aligned auxiliary data are matched to generate the pre-detection result of the current frame image and perform trajectory fusion association and management.

[0032] Preferably, the step S3 comprises:

[0033] Step S3.1: Receive the target pre-detection result and the intelligent fine detection result of the current frame image at the same time. If the intelligent fine detection obtains a new result, replace or supplement the target pre-detection result of each frame image with the fine detection result, and re-associate the target trajectory of all the image frames of the replaced or supplemented target pre-detection results; if there is no intelligent fine detection result, associate the target trajectory of the pre-detection result of the current frame image;

[0034] Step S3.2: After the current frame image trajectory is associated, the start, merge, continuation and termination processing of the target image plane trajectory batch number is performed to generate and maintain the target image plane trajectory library; when multiple frames in a target image plane trajectory in the target trajectory library cannot be associated within a preset range, the target intelligent precision detection algorithm is called to output the target precision detection result; if the continuous frames of the target trajectory exceed the preset threshold and there is no new associated detection point, it means that the trajectory has been terminated;

[0035] Step S3.3: When it is determined that intelligent precision detection of the target is required, a target slice sequence to be precisely detected is generated according to the target trajectory and trajectory prediction results in combination with a continuous cache image sequence, and then intelligent precision detection of the target is performed.

[0036] Preferably, step S4 comprises:

[0037] Step S4.1: After receiving multiple target slice sequences to be precisely detected of the current frame image, batches are simultaneously input into a preset intelligent target precision detection algorithm based on a spatiotemporal fusion deep learning model, and the length S of the target slice sequence to be precisely detected is adjusted according to the timeliness of the target intelligent precision detection, where 5≤S≤the length of the continuous cache image sequence;

[0038] Step S4.2: After the detection points are output, they are clustered in the connected domain and the centroid is calculated to generate the S-frame target precision detection results, and then return to step S3 for trajectory fusion association and management.

[0039] Preferably, step S5 comprises:

[0040] Step S5.1: When it is determined that intelligent precision detection is not required, the target image plane trajectory is received to determine the target sight line, and the auxiliary data is aligned with the target image plane trajectory according to the imaging time and frame count;

[0041] Step S5.2: According to the position, velocity and attitude of the satellite during imaging, combined with the camera imaging model and parameters, the satellite-target pointing line of sight is determined to generate the satellite target angular trajectory.

[0042] Preferably, step S6 comprises:

[0043] Step S6.1: judging whether the target area response exceeds the camera detector response range according to the target image plane trajectory and the target dynamic slice data, and generating satellite payload detector imaging adaptive control parameters, including gain and integration time;

[0044] Step S6.2: Calculate and obtain the adaptive coefficient of grayscale correction of the current frame according to the target image plane trajectory and the target dynamic slice data, which serves as the basis for adjusting the adaptive parameter of grayscale correction of the next frame;

[0045] The adaptive coefficients a and b are calculated according to the following formula:

[0046]

[0047] Where m is the mean of all pixels in the dynamic slice; σ is the standard deviation of all pixels in the dynamic slice; K is the preset scale factor; DN max is the maximum value of pixels.

[0048] The information processing system for integrating on-board real-time target pre-detection and intelligent precise detection provided by the present invention adopts the information processing method for integrating on-board real-time target pre-detection and intelligent precise detection, and includes the following modules:

[0049] Module M1: Receives real-time imaging data from the camera and completes preprocessing in real time, including data frame extraction, data unpacking, image data caching, grayscale transformation, image registration and image difference;

[0050] Module M2: Determine whether the target to be pre-inspected is located in the preset key observation area. If so, directly perform intelligent precision detection. If not, perform target pre-detection and output suspected targets and slices.

[0051] Module M3: Receives target pre-detection or intelligent precision detection results, associates and manages target image trajectory, and determines whether to perform intelligent precision detection of the target. If yes, module M4 is triggered, otherwise module M5 is triggered;

[0052] Module M4: reads in the target slice sequence to be precisely inspected, performs intelligent precise inspection of the target, outputs precise inspection results, and performs trajectory association and management;

[0053] Module M5: Determine the target line of sight by combining the satellite position, velocity and attitude data, generate the target angular trajectory and output the target image trajectory and target dynamic slice data;

[0054] Module M6: Combines the target image trajectory and target dynamic slice data to calculate and generate satellite payload imaging parameters and preprocessing parameters in real time, completing adaptive satellite payload imaging and preprocessing;

[0055] Module M7: Receives the target angular trajectory obtained by other satellite detection, combines it with the target angular trajectory generated by the local satellite, performs target dual-star or multi-star fusion positioning, outputs the target three-dimensional trajectory, and estimates the target parameters.

[0056] According to the computer-readable storage medium storing a computer program provided by the present invention, when the computer program is executed by a processor, the steps of the information processing method for integrating on-board real-time target pre-detection and intelligent precise detection are implemented.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] (1) The present invention provides an information processing method and system for integrating on-board real-time target pre-detection and intelligent precision detection, which can complete target pre-detection in real time at an ultra-high bit rate on-board, and can further use intelligent target detection methods to reduce target missed detection and false alarms, thereby improving target detection accuracy;

[0059] (2) The present invention provides an information processing method and system for integrating on-board real-time target pre-detection and intelligent precision detection, which can realize the full-process space-based information processing such as on-board target pre-detection, intelligent precision detection, trajectory fusion association and management, line of sight determination, autonomous parameter control and fusion positioning under ultra-high bit rate and massive data;

[0060] (3) The present invention adopts a method and system that integrates real-time target pre-detection and intelligent precision detection, which solves the problems of high-code rate real-time processing, high-precision and low-false-alarm target trajectory detection and generation. It can be widely used in various on-board target real-time detection and information generation application scenarios, and can also be used for satellite ground information processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0062] Figure 1 It is a schematic diagram of the overall process of the method of the present invention. DETAILED DESCRIPTION

[0063] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0064] Example

[0065] The present invention provides an information processing method for integrating on-board real-time target pre-detection and intelligent precise detection, comprising:

[0066] Step S1: receiving real-time imaging data from the camera, and completing pre-processing processes such as data frame extraction, data unpacking, image data caching, grayscale adjustment, and image difference in real time;

[0067] Step S2: Determine whether the area where the target to be detected is located is a key observation area. If so, directly connect to intelligent precise detection. If not, perform target pre-detection and output suspected targets and slices.

[0068] Step S3: receiving target pre-detection and intelligent precision detection results, performing target image trajectory association and management, and determining whether to perform target intelligent precision detection;

[0069] Step S4: If target precision detection is required, read in the target slice sequence to be precisely detected to perform intelligent target precision detection, further improve the target detection accuracy and output the precision detection result, and perform trajectory association and management;

[0070] Step S5: If it is determined that the current target image plane trajectory no longer requires precise detection, the target line of sight is determined in combination with the satellite position speed and attitude data, the target angle trajectory is generated, and the target image plane trajectory and target dynamic slice data are output;

[0071] Step S6: combining the target image trajectory and the target dynamic slice data, calculating and generating satellite payload imaging parameters and preprocessing parameters in real time, and completing adaptive satellite payload imaging and preprocessing;

[0072] Step S7: Receive the target angular trajectory acquired by other satellite detection, combine it with the target angular trajectory generated by the local satellite, perform target dual-star or multi-star fusion positioning, output the target three-dimensional trajectory, and estimate the target parameters.

[0073] The present invention uses target pre-detection to achieve real-time target processing of large-format, high-resolution and other cameras at ultra-high speeds and massive input data on board, and integrates intelligent precision detection with a unified trajectory management method to further improve target detection accuracy and increase target trajectory detection length. It is suitable for real-time high-precision information processing on board at ultra-large space-based detection data rates, and can effectively reduce the bandwidth of satellite-to-ground data transmission and improve the availability of space-based information. The method flow of the present invention is as follows: Figure 1 shown.

[0074] The step S1 comprises:

[0075] Step S1.1, receiving camera imaging data in real time, searching for frame header and frame count, and completing data frame extraction in the form of code stream, and the frame extraction frequency can be updated;

[0076] Step S1.2, complete data unpacking in real time, and generate camera imaging auxiliary data (including at least satellite position speed, attitude, integration time, gain, etc. during imaging) and image data sequence with imaging time and frame count as identifiers;

[0077] Step S1.3, cache several frames of image data with a certain interval between the current frame image to generate an interval cache image sequence as input for subsequent image registration and image difference; cache several consecutive frames of image data before the current frame to generate a continuous cache image sequence as input for target precision detection sequence slices and target dynamic slice data generation; cache image data is updated once in real time when each frame of data is input; the total number of frames, interval frames and continuous cache frames of the interval cache image sequence can be updated according to the task scenario;

[0078] Step S1.4, after inter-frame registration of the current frame image and the interval cache image sequence, pixel by pixel difference is performed with the interval cache image to generate a differential image sequence;

[0079] Step S1.5, performing non-uniformity correction and grayscale transformation on the current frame image and the differential image sequence to be pre-detected for the target;

[0080] If the original DN value of the pixel in the i-th row and j-th column in the image is x ij , and its corresponding non-uniform correction parameter is k ij 、f ij , the corrected pixel DN value is y ij =k ij ·x ij +f ij The non-uniformity correction parameters are set pixel by pixel and can be updated online.

[0081] Further image grayscale transformation is performed on the image after non-uniformity correction. Using linear stretching, if the pixel value before transformation is x, and the pixel value after transformation is y, the expression is:

[0082]

[0083] Parameters a and b can be set as pre-set parameters and adaptive parameters. Pre-set parameters are fixed values ​​set on the ground and can be updated online. Adaptive parameters are calculated and updated in real time according to step S6.

[0084] The step S2 comprises:

[0085] Step S2.1, pre-detect the current frame image and the differential sequence image, and output the target point set {(m,n,y m,n )}; where m and n are the row and column numbers of the target pixel points detected in the current frame.

[0086] The target pre-detection algorithm depends on the on-board computing resources and detection accuracy requirements. It can use the traditional differential weighted threshold method or intelligent algorithms such as deep learning.

[0087] The current frame image is X 0 , X 0-i, X 0-j, ,X 0-k is a differential image sequence with the current frame interval i, j, k frames, then for the weighted image X = w 1 X 0 +w 2 X 0-i +w 2 X 0-j +w 3 X 0-k Median filtering and threshold segmentation are performed, and image pixels greater than the threshold are judged as target points to form a target point set {(m,n,ym,n)}; where w 1 ,w 2 ,w 3 ,w 4 is the image weight value, which sums to 1 and can be updated.

[0088] Step S2.2, for the target point set {(m,n,y m,n )} Perform connected domain clustering calculation, and calculate the centroid of the subset of the clustered target point set. The row and column numbers of the centroid are used as the position of the detected target on the current frame image; all targets detected in the current frame are numbered, and the target imaging time, frame count and aligned auxiliary data are matched. The pre-detection result of the current frame image is generated and sent to the trajectory fusion association and management.

[0089] The step S3 comprises:

[0090] Step S3.1, simultaneously receive the pre-detection result and fine detection result of the target in the current frame. If the intelligent fine detection detects a new result, the target detection result of each frame is replaced or supplemented with the fine detection result, and all image frames of the replaced or supplemented target detection results are re-associated with the target trajectory; if there is no fine detection result, the target trajectory is associated according to the pre-detection result of the current frame image.

[0091] Step S3.2, after the current frame trajectory is associated, the start, merge, continuation, and termination of the target image plane trajectory batch number are processed to generate and maintain the target image plane trajectory library. When 3 out of 5 frames in a target image plane trajectory in the target trajectory library cannot be associated, the target intelligent precision detection algorithm is called, and the target precision detection result is output to step S3.1; if there is no new associated detection point for 20 consecutive frames of the target trajectory, the trajectory has been terminated. The trajectory termination condition can be updated according to the on-track situation.

[0092] Step S3.3: When it is determined that the target needs to be precisely detected, a target slice sequence to be precisely detected is generated based on the target trajectory and trajectory prediction results, combined with the cached continuous image frame sequence, and output to the target intelligent precise detection S4. The size of the target slice to be precisely detected is updated on-orbit according to the demand.

[0093] The step S4 comprises:

[0094] Step S4.1, after receiving the slice sequence of multi-track targets to be precisely detected in the current frame image, batch and simultaneously input the intelligent target precision detection algorithm based on the spatiotemporal fusion deep learning model. The length S of the slice sequence to be precisely detected is adjusted according to the timeliness of the intelligent precision detection of the target (5≤S≤the length of the continuous cache image sequence), and can be updated on the track.

[0095] Step S4.2: After the detection points are output, they are clustered in the communication domain and the centroid is calculated to generate S-frame target precision detection results, and then return to step S3 for trajectory fusion association and management. To ensure real-time performance, the intelligent precision detection is completed within 5 frame periods and the results are packaged and transmitted.

[0096] The step S5 comprises:

[0097] Step S5.1, when it is determined that it is not necessary to enter fine detection, the target image plane trajectory is received to determine the target line of sight. The auxiliary data is aligned with the target image plane trajectory according to the imaging time and frame count;

[0098] Step S5.2, according to the position, speed and attitude of the satellite during imaging, combined with the camera imaging model and parameters, the satellite-target pointing line of sight is determined, and the satellite target angular trajectory is generated and output to step S7.

[0099] The step S6 comprises:

[0100] Step S6.1, judging whether the target area response exceeds the camera detector response range according to the target image plane trajectory and dynamic slices, and generating imaging adaptive control parameters such as satellite payload detector gain and integration time;

[0101] Step S6.2, automatically calculating and obtaining the adaptive coefficient of grayscale correction of the current frame according to the target image plane trajectory and dynamic slicing, and outputting it to step S1 as a basis for adjusting the adaptive parameters of grayscale correction of the next frame;

[0102] The adaptive coefficients a and b are calculated according to the following formula:

[0103]

[0104] Among them, m is the mean of all pixels in the dynamic slice, σ is the standard deviation of all pixels in the dynamic slice, and K is a preset scale factor that can be updated on the track.

[0105] The present invention also provides an information processing system for integrating on-board real-time target pre-detection and intelligent precise detection, comprising:

[0106] Module M1: Receives real-time imaging data from satellite payloads and completes pre-processing processes such as data frame extraction, data unpacking, image data caching, grayscale conversion, image registration, and image difference in real time;

[0107] Module M2: Determine whether the area where the target to be detected is located is a key observation area. If so, directly connect to intelligent precision detection. If not, perform target pre-detection and output suspected targets and slices.

[0108] Module M3: Receives target pre-detection and intelligent precision detection results to associate and manage target image trajectory, and determines whether to perform intelligent precision detection of the target;

[0109] Module M4: If target precision detection is required, the target slice sequence to be precisely detected is read in for intelligent target precision detection, which further improves the target detection accuracy and outputs the precision detection results, and then performs trajectory association and management;

[0110] Module M5: If it is determined that the current target image plane trajectory no longer requires precise detection, the target line of sight is determined in combination with the satellite position, velocity and attitude data, the target angle trajectory is generated, and the target image plane trajectory and dynamic slice data are output;

[0111] Module M6: Combines the target image trajectory and target dynamic slice data to calculate and generate satellite payload imaging parameters and preprocessing parameters in real time, completing adaptive satellite payload imaging and preprocessing;

[0112] Module M7: Receives the target angular trajectory obtained by other satellite detection, combines it with the target angular trajectory generated by the local satellite, performs target dual-star or multi-star fusion positioning, outputs the target three-dimensional trajectory, and estimates the target parameters.

[0113] Those skilled in the art know that, in addition to implementing the system, device and its various modules provided by the present invention in a purely computer-readable program code, it is entirely possible to implement the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps. Therefore, the system, device and its various modules provided by the present invention can be considered as a hardware component, and the modules included therein for implementing various programs can also be considered as structures within the hardware component; the modules for implementing various functions can also be considered as both software programs for implementing the method and structures within the hardware component.

[0114] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. An information processing method for integrating on-board real-time target pre-detection and intelligent precision detection, characterized in that: The steps include: Step S1: receiving real-time imaging data from the camera and completing preprocessing in real time, including data frame extraction, data unpacking, image data caching, grayscale transformation, image registration and image difference; Step S2: determine whether the target to be pre-inspected is located in the preset key observation area. If so, directly perform intelligent precision detection. If not, perform target pre-detection and output suspected targets and slices. Step S3: receiving the target pre-detection or intelligent precise detection result, performing target image trajectory association and management, and determining whether to perform target intelligent precise detection, if yes, executing step S4, otherwise executing step S5; Step S4: read in the target slice sequence to be precisely detected, perform intelligent precise detection of the target, output precise detection results, and perform trajectory association and management; Step S5: Determine the target sight line by combining the satellite position speed and attitude data, generate the target angle trajectory and output the target image trajectory and target dynamic slice data; Step S6: combining the target image trajectory and the target dynamic slice data, calculating and generating satellite payload imaging parameters and preprocessing parameters in real time, and completing adaptive satellite payload imaging and preprocessing; Step S7: Receive the target angular trajectory obtained by detecting other satellites, combine it with the target angular trajectory generated by the local satellite, perform target dual-star or multi-star fusion positioning, output the target three-dimensional trajectory, and estimate the target parameters.

2. The information processing method for integrating on-board real-time target pre-detection and intelligent precise detection according to claim 1 is characterized in that: The step S1 comprises: Step S1.1: Receive camera imaging data in real time, find the frame header and frame count, and complete data frame extraction in the form of code stream; Step S1.2: completing data unpacking in real time, and generating camera imaging auxiliary data and image data sequence with imaging time and frame count as identifiers; Step S1.3: Cache multiple frames of image data with preset intervals between the current frame image, generate an interval cache image sequence, and use it as input for subsequent image registration and image difference; cache multiple consecutive frames of image data before the current frame, generate a continuous cache image sequence, and use it as input for generating the target slice sequence to be precisely detected and the target dynamic slice data; the cache image data is updated once in real time when each frame of data is input; Step S1.4: After inter-frame registration of the current frame image and the interval buffer image sequence, pixel by pixel difference is performed with the interval buffer image sequence to generate a differential image sequence.

3. The information processing method for integrating on-board real-time target pre-detection and intelligent precise detection according to claim 2 is characterized in that: Performing non-uniformity correction and grayscale transformation on the current frame image and differential image sequence for target pre-detection; Suppose the original DN value of the pixel in the i-th row and j-th column of the image is x ij , and its corresponding non-uniform correction parameter is k ij 、f ij , the corrected pixel DN value is y ij =k ij ·x ij +f ij ; Perform image grayscale transformation on the image after non-uniformity correction, using linear stretching. If the pixel value before transformation is x, and the pixel value after transformation is y, the expression is: Among them, a and b are prefabricated parameters and adaptive parameters respectively. The prefabricated parameters are fixed values ​​set on the ground; DN max is the maximum value of pixels.

4. The information processing method for integrating on-board real-time target pre-detection and intelligent precise detection according to claim 2 is characterized in that: The step S2 comprises: Step S2.1: Pre-detect the current frame image and the differential image sequence, and output the target point set {(m,n,y m,n )}, where m and n are the row and column numbers of the target pixel points detected in the current frame image; y m,n is the pixel value of m rows and n columns in the target point set; The current frame image is X0, X 0-i, X 0-j, ,X 0-k is the differential image sequence with the current frame interval i, j, k frames, for the weighted image X = w1X0 + w2X 0-i +w2X 0-j +w3X 0-k Perform median filtering and threshold segmentation, and judge the image pixels greater than the threshold as target points to form a target point set {(m,n,ym,n)}; where w1,w2,w3,w4 are image weight values, and their sum is 1; Step S2.2, for the target point set {(m,n,y m,n )} Perform connected domain clustering calculation, and calculate the centroid of the subset of the clustered target point set. The row and column numbers of the centroid are used as the position of the detected target on the current frame image; all targets detected in the current frame image are numbered, and the target imaging time, frame count and aligned auxiliary data are matched to generate the pre-detection result of the current frame image and perform trajectory fusion association and management.

5. The information processing method for integrating on-board real-time target pre-detection and intelligent precise detection according to claim 2 is characterized in that: The step S3 comprises: Step S3.1: Receive the target pre-detection result and the intelligent fine detection result of the current frame image at the same time. If the intelligent fine detection obtains a new result, replace or supplement the target pre-detection result of each frame image with the fine detection result, and re-associate the target trajectory of all the image frames of the replaced or supplemented target pre-detection results; if there is no intelligent fine detection result, associate the target trajectory of the pre-detection result of the current frame image; Step S3.2: After the current frame image trajectory is associated, the start, merge, continuation and termination processing of the target image plane trajectory batch number is performed to generate and maintain the target image plane trajectory library; when multiple frames in a target image plane trajectory in the target trajectory library cannot be associated within a preset range, the target intelligent precision detection algorithm is called to output the target precision detection result; if the continuous frames of the target trajectory exceed the preset threshold and there is no new associated detection point, it means that the trajectory has been terminated; Step S3.3: When it is determined that intelligent precision detection of the target is required, a target slice sequence to be precisely detected is generated according to the target trajectory and trajectory prediction results in combination with a continuous cache image sequence, and then intelligent precision detection of the target is performed.

6. The information processing method for integrating on-board real-time target pre-detection and intelligent precise detection according to claim 2 is characterized in that: The step S4 comprises: Step S4.1: After receiving multiple target slice sequences to be precisely detected of the current frame image, batches are simultaneously input into a preset intelligent target precision detection algorithm based on a spatiotemporal fusion deep learning model, and the length S of the target slice sequence to be precisely detected is adjusted according to the timeliness of the target intelligent precision detection, where 5≤S≤the length of the continuous cache image sequence; Step S4.2: After the detection points are output, they are clustered in the connected domain and the centroid is calculated to generate the S-frame target precision detection results, and then return to step S3 for trajectory fusion association and management.

7. The information processing method for integrating on-board real-time target pre-detection and intelligent precise detection according to claim 1 is characterized in that: The step S5 comprises: Step S5.1: When it is determined that intelligent precision detection is not required, the target image plane trajectory is received to determine the target sight line, and the auxiliary data is aligned with the target image plane trajectory according to the imaging time and frame count; Step S5.2: According to the position, velocity and attitude of the satellite during imaging, combined with the camera imaging model and parameters, the satellite-target pointing line of sight is determined to generate the satellite target angular trajectory.

8. The information processing method for integrating on-board real-time target pre-detection and intelligent precise detection according to claim 1 is characterized in that: The step S6 comprises: Step S6.1: judging whether the target area response exceeds the camera detector response range according to the target image plane trajectory and the target dynamic slice data, and generating satellite payload detector imaging adaptive control parameters, including gain and integration time; Step S6.2: Calculate and obtain the adaptive coefficient of grayscale correction of the current frame according to the target image plane trajectory and the target dynamic slice data, which serves as the basis for adjusting the adaptive parameter of grayscale correction of the next frame; The adaptive coefficients a and b are calculated according to the following formula: Where m is the mean of all pixels in the dynamic slice; σ is the standard deviation of all pixels in the dynamic slice; K is the preset scale factor; DN max is the maximum value of pixels.

9. An information processing system integrating on-board real-time target pre-detection and intelligent precision detection, characterized in that: The information processing method for integrating on-board real-time target pre-detection and intelligent precision detection according to any one of claims 1 to 8 comprises the following modules: Module M1: Receives real-time imaging data from the camera and completes preprocessing in real time, including data frame extraction, data unpacking, image data caching, grayscale transformation, image registration and image difference; Module M2: Determine whether the target to be pre-inspected is located in the preset key observation area. If so, directly perform intelligent precision detection. If not, perform target pre-detection and output suspected targets and slices. Module M3: Receives target pre-detection or intelligent precision detection results, associates and manages target image trajectory, and determines whether to perform intelligent precision detection of the target. If yes, module M4 is triggered, otherwise module M5 is triggered; Module M4: reads in the target slice sequence to be precisely inspected, performs intelligent precise inspection of the target, outputs precise inspection results, and performs trajectory association and management; Module M5: Determine the target line of sight by combining the satellite position, velocity and attitude data, generate the target angle trajectory and output the target image trajectory and target dynamic slice data; Module M6: Combines the target image trajectory and target dynamic slice data to calculate and generate satellite payload imaging parameters and preprocessing parameters in real time, completing adaptive satellite payload imaging and preprocessing; Module M7: Receives the target angular trajectory obtained by other satellite detection, combines it with the target angular trajectory generated by the local satellite, performs target dual-star or multi-star fusion positioning, outputs the target three-dimensional trajectory, and estimates the target parameters.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the information processing method for integrating on-board real-time target pre-detection and intelligent precise detection according to any one of claims 1 to 8 are implemented.

Citation Information

Patent Citations

  • Multi-satellite formation cooperative area imaging autonomous task planning method and system

    CN111950877A

  • Multi-satellite joint test method and system for multi-satellite distributed information fusion system

    CN114509063A

  • Space moving target detection tracking method based on space-based optical observation image

    CN115131392A

  • Space target motion trail optical measurement and correlation method

    CN115775258A

  • Optical remote sensing satellite target three-dimensional tracking method

    CN116879968A