Information processing methods, systems, and media for integrating real-time onboard target pre-detection and intelligent precision detection

Abstract

This invention provides an information processing method, system, and medium that integrates on-board real-time target pre-detection and intelligent precision detection, comprising: step S1: data preprocessing; step S2: target pre-detection; step S3: trajectory association and management; step S4: intelligent precision detection of targets; step S5: trajectory target line-of-sight determination; step S6: self-service parameter control; and step S7: target fusion positioning. This invention utilizes target pre-detection to achieve real-time processing of massive input data from large-area, high-resolution on-board cameras at ultra-high rates. It further enhances target detection accuracy and increases target trajectory detection length by integrating intelligent precision detection and a unified trajectory management method. This method is suitable for on-board real-time high-precision information processing under ultra-high data rates in space-based applications, effectively reducing the bandwidth of space-to-ground data transmission and improving the availability of space-based information.

Description

Technical Field

[0001] This invention relates to the field of space-based real-time information processing technology, specifically to an information processing method, system, and medium that integrates on-board real-time target pre-detection and intelligent precision detection. Background Technology

[0002] Real-time onboard information processing is crucial for the efficient application of space-based information today. With the increasing scale and spatial resolution of satellite payload detectors, some payloads now have real-time code rates exceeding 40 Gbps, and the long imaging times generate massive amounts of data, posing significant challenges to satellite-to-ground transmission and onboard real-time processing. Achieving high-precision, low-false-alarm detection of onboard targets, trajectory fusion and management, line-of-sight determination, and fusion positioning under ultra-high code rate conditions is a key focus and challenge for improving the efficiency of current space-based information processing and satellite applications.

[0003] Patent document CN115294439 discloses a method, system, device and storage medium for detecting weak moving targets in the air. It mainly uses multispectral data with imaging parallax to detect displacement parallax in target images. It is a traditional moving target detection method and system, but does not involve intelligent target detection methods and real-time processing methods.

[0004] Patent document CN115497003A discloses a lightweight and fast target detection method for arbitrary orientation based on satellite edge computing. This method primarily enables on-orbit inference and execution of a lightweight target detection model trained on the ground on a satellite's resource-constrained edge processor, achieving real-time on-orbit detection. In contrast, this invention addresses the challenge of real-time on-board information processing by fusing pre-detection and intelligent precision detection with ultra-high bitrate satellite payload data. Simultaneously, it generates target angular trajectories and 3D trajectories in real-time on-orbit based on the detection results.

[0005] Patent document CN117036985A discloses a method and device for small target detection in video satellite images. It mainly utilizes a YOLOv5 model to train a small target detection model, solving the problem of unclear boundaries of small objects. However, it does not involve processing procedures such as moving target detection, line-of-sight determination, and fusion localization.

[0006] Patent document CN117173039A discloses a satellite-based real-time moving target detection noise removal system. This system primarily utilizes target detection results for geographic location projection and elevation prediction. By estimating and maintaining the target position in a fixed Earth coordinate system, it removes noise introduced by stationary points on the Earth's surface, preventing the loss of the target's real-time position due to temporary image loss. In contrast, this invention involves a full-process space-based real-time information processing system encompassing target image plane detection and tracking, trajectory association and management, line-of-sight determination, and fusion positioning. It achieves false alarm point filtering and improves target detection accuracy through a combination of target pre-detection and intelligent precision target detection, as well as trajectory fusion association and management.

[0007] Patent document CN110647977A discloses a Tiny-YOLO network optimization method for onboard ship target detection, and patent document CN111598004B discloses a target detection and tracking method applicable to high-resolution remote sensing satellite video data. Both are target detection and tracking based on the YOLO model and do not involve the target pre-detection, line-of-sight determination, and fusion positioning processing procedures in this invention.

[0008] To address the current situation of continuously increasing satellite payload data code rates, and with the help of high-performance processors such as CPUs, FPGAs, DSPs, and AIs in space applications, it is necessary to develop on-board real-time information processing methods at ultra-high code rates to generate target information of interest to users, significantly reduce the amount of data transmitted between satellite and ground, and improve the timeliness of satellite data and the efficiency of information application. Summary of the Invention

[0009] To address the shortcomings of existing technologies, the purpose of this invention is to provide an information processing method, system, and medium that integrates on-board real-time target pre-detection and intelligent precision detection.

[0010] The information processing method for fusing real-time on-board target pre-detection and intelligent precision detection provided by the present invention includes the following steps:

[0011] Step S1: Receive real-time imaging data from the camera and complete preprocessing in real time, including data frame extraction, data unpacking, image data buffering, grayscale transformation, image registration, and image differencing;

[0012] Step S2: Determine whether the target to be pre-inspected is located in the preset key observation area. If so, perform intelligent fine detection directly. If not, perform target pre-inspection and output the suspected target and slice.

[0013] Step S3: Receive the target pre-detection or intelligent fine detection results, perform target image plane trajectory association and management, and determine whether to perform target intelligent fine detection. If yes, proceed to step S4; otherwise, proceed to step S5.

[0014] Step S4: Read in the target slice sequence to be precisely detected, perform intelligent precision detection of the target, output the precision detection results, and perform trajectory association and management;

[0015] Step S5: Combine satellite position, velocity and attitude data to determine the target line of sight, generate the target angular trajectory and output the target image plane trajectory and target dynamic slice data;

[0016] Step S6: Combining the target image plane trajectory and target dynamic slice data, calculate and generate satellite payload imaging parameters and preprocessing parameters in real time to complete adaptive satellite payload imaging and preprocessing;

[0017] Step S7: Receive the target angular trajectory obtained by other satellites, combine it with the target angular trajectory generated by the local satellite, perform target binary or multi-satellite fusion positioning, output the target three-dimensional trajectory, and perform target parameter estimation.

[0018] Preferably, step S1 includes:

[0019] Step S1.1: Receive camera imaging data in real time, find the frame header and frame count, and extract data frames in the form of a bitstream;

[0020] Step S1.2: Real-time data unpacking is completed, and camera imaging auxiliary data and image data sequences are generated using imaging time and frame count as identifiers;

[0021] Step S1.3: Cache image data from multiple frames with a preset interval between the current frame and the image, and generate an interval cached image sequence as input for subsequent image registration and image differencing; cache image data from multiple consecutive frames before the current frame, and generate a continuous cached image sequence as input for the target slice sequence to be finely detected and the generation of target dynamic slice data; the cached image data is updated in real time once when each frame of data is input;

[0022] Step S1.4: After performing inter-frame registration between the current frame image and the interval buffer image sequence, the difference between the current frame image and the interval buffer image sequence is calculated pixel by pixel to generate a difference image sequence.

[0023] Preferably, non-uniformity correction and grayscale transformation are performed on the current frame image and the difference image sequence for target pre-detection.

[0024] Let the original DN value of the pixel in the i-th row and j-th column of the image be x. ij The corresponding non-uniformity correction parameter is k. ij f ij Its corrected pixel DN value is y ij =k ij ·x ij +f ij ;

[0025] To perform image grayscale transformation on the non-uniformity corrected image using linear stretching, if the pixel value before transformation is x and the pixel value after transformation is y, the expression is:

[0026]

[0027] Where a and b are the prefabricated parameter and the adaptive parameter, respectively, and the prefabricated parameter is a fixed value set on the ground; DN max This represents the maximum pixel value.

[0028] Preferably, step S2 includes:

[0029] Step S2.1: Perform pre-detection on the current frame image and the difference image sequence, and output the target point set {(m,n,y} in the image. m,n )}, where m and n are the row and column numbers of the target pixels detected in the current frame image; y m,n The pixel values ​​in row m and column n of the target point set;

[0030] The current frame image is X0, X 0-i, X 0-j, ,X 0-k For the difference image sequence between the current frame and the current frame at intervals of i, j, k, the weighted image X = w1X0 + w2X 0-i +w2X 0-j +w3X 0-k Median filtering and threshold segmentation are performed. Image pixels larger than the threshold are identified as target points, forming a target point set {(m,n,ym,n)}; where w1,w2,w3,w4 are image weights, and their sum is 1.

[0031] Step S2.2, for the target point set {(m,n,y) of the current frame image m,n Perform connected domain clustering calculations, and calculate the centroid of a subset of the clustered target point set. The row and column numbers of the centroids are used as the positions of the detected targets on the current frame image. Number all detected targets in the current frame image, match the target imaging time, frame count and alignment auxiliary data, generate the pre-detection results of the current frame image and perform trajectory fusion association and management.

[0032] Preferably, step S3 includes:

[0033] Step S3.1: Simultaneously receive the target pre-detection result and the intelligent fine detection result of the current frame image. If the intelligent fine detection yields a new result, replace or supplement the target pre-detection result of each frame image with the fine detection result, and re-associate the target trajectory of all replaced or supplemented target pre-detection result image frames. If there is no intelligent fine detection result, perform target trajectory association of the pre-detection result of the current frame image.

[0034] Step S3.2: After associating the current frame image trajectory, perform start, merge, continuation, and termination processing of the target image plane trajectory batch number to generate and maintain the target image plane trajectory library; when multiple frames of a target image plane trajectory in the target trajectory library cannot be associated within a preset range, the target intelligent fine detection algorithm is called to output the target fine detection result; if the consecutive frames of the target trajectory exceed the preset threshold and there are no new associated detection points, it indicates that the trajectory has been terminated.

[0035] Step S3.3: When it is determined that intelligent fine detection of the target is required, a target slice sequence to be finely detected is generated based on the target trajectory and trajectory prediction results, combined with the continuous cached image sequence, and then intelligent fine detection of the target is performed.

[0036] Preferably, step S4 includes:

[0037] Step S4.1: After receiving multiple target slice sequences to be finely detected from the current frame image, input the preset intelligent target fine detection algorithm based on the spatiotemporal fusion deep learning model in batches. The length S of the target fine detection slice sequence is adjusted according to the timeliness of intelligent target fine detection, where 5≤S≤length of continuous cached image sequence.

[0038] Step S4.2: After outputting the detection points, cluster them in the connected domain and find the centroid to generate S-frame target fine detection results. Then return to step S3 for trajectory fusion association and management.

[0039] Preferably, step S5 includes:

[0040] Step S5.1: When it is determined that intelligent fine detection is not required, the target image plane trajectory is received to determine the target line of sight, and the auxiliary data is aligned with the target image plane trajectory according to the imaging time and frame count;

[0041] Step S5.2: Based on the satellite's position, velocity, and attitude during imaging, and combined with the camera imaging model and parameters, determine the satellite-target pointing line of sight and generate the local satellite target angular trajectory.

[0042] Preferably, step S6 includes:

[0043] Step S6.1: Based on the target image trajectory and target dynamic slice data, determine whether the target region response exceeds the camera detector response range, and generate satellite payload detector imaging adaptive control parameters, including gain and integration time;

[0044] Step S6.2: Calculate and obtain the adaptive coefficient of grayscale correction for the current frame based on the target image trajectory and target dynamic slice data, and use it as the basis for adjusting the adaptive parameters of grayscale correction for the next frame;

[0045] The adaptive coefficients a and b are calculated according to the following formula:

[0046]

[0047] Where m is the mean of all pixels in the dynamic slice; σ is the standard deviation of all pixels in the dynamic slice; K is the preset scaling factor; DN max This represents the maximum pixel value.

[0048] The information processing system for fusing on-board real-time target pre-detection and intelligent precision detection provided by the present invention, employing the aforementioned information processing method for fusing on-board real-time target pre-detection and intelligent precision detection, includes the following modules:

[0049] Module M1: Receives real-time imaging data from the camera and performs preprocessing in real time, including data frame extraction, data unpacking, image data buffering, grayscale transformation, image registration, and image differencing.

[0050] Module M2: Determines whether the target to be pre-inspected is located in the preset key observation area. If so, it directly performs intelligent fine detection; otherwise, it performs target pre-detection and outputs the suspected target and slice.

[0051] Module M3: Receives the target pre-detection or intelligent fine detection results, performs target image plane trajectory association and management, and determines whether to perform target intelligent fine detection. If so, it triggers module M4; otherwise, it triggers module M5.

[0052] Module M4: Reads in the target slice sequence to be precisely detected, performs intelligent precision detection of the target, outputs the precision detection results, and performs trajectory association and management;

[0053] Module M5: Combines satellite position, velocity and attitude data to determine the target line of sight, generates the target angular trajectory and outputs the target image plane trajectory and target dynamic slice data;

[0054] Module M6: Combines target image plane trajectory and target dynamic slice data to calculate and generate satellite payload imaging parameters and preprocessing parameters in real time, and completes adaptive satellite payload imaging and preprocessing;

[0055] Module M7: Receives the target angular trajectory obtained by other satellites, combines it with the target angular trajectory generated by the local satellite, performs target dual-satellite or multi-satellite fusion positioning, outputs the target's three-dimensional trajectory, and performs target parameter estimation.

[0056] According to the computer-readable storage medium storing a computer program provided by the present invention, when the computer program is executed by a processor, it implements the steps of the information processing method for fusing on-board real-time target pre-detection and intelligent precision detection.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] (1) This invention provides an information processing method and system that integrates real-time target pre-detection and intelligent precision detection on satellites. It can complete target pre-detection in real time under ultra-high code rate conditions on satellites, and can further utilize intelligent target detection methods to reduce target missed detection and false alarms, and improve target detection accuracy.

[0059] (2) This invention provides an information processing method and system that integrates on-board real-time target pre-detection and intelligent precision detection, which can realize the entire process of space-based information processing such as on-board target pre-detection, intelligent precision detection, trajectory fusion association and management, line of sight determination, autonomous parameter control and fusion positioning under ultra-high bit rate and massive data.

[0060] (3) The present invention adopts a method and system that integrates real-time target pre-detection and intelligent precision detection, which solves problems such as high code rate real-time processing, high precision low false alarm target trajectory detection and generation. It can be widely used in various on-board target real-time detection and information generation application scenarios, and can also be used in satellite ground information processing. Attached Figure Description

[0061] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0062] Figure 1 This is a schematic diagram of the overall process of the method of the present invention. Detailed Implementation

[0063] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0064] Example

[0065] This invention provides an information processing method that integrates on-board real-time target pre-detection and intelligent precision detection, including:

[0066] Step S1: Receive real-time imaging data from the camera and complete preprocessing processes such as data frame extraction, data unpacking, image data caching, grayscale adjustment, and image difference in real time;

[0067] Step S2: Determine whether the area where the target to be detected is located is a key observation area. If so, directly connect to intelligent precision detection. If not, perform target pre-detection and output suspected targets and slices.

[0068] Step S3: Receive the target pre-detection and intelligent fine detection results, perform target image plane trajectory association and management, and determine whether to perform target intelligent fine detection;

[0069] Step S4: If fine target detection is required, read in the target slice sequence to be finely detected, perform intelligent fine target detection, further improve the target detection accuracy and output the fine detection results, and perform trajectory association and management;

[0070] Step S5: If it is determined that the current target image plane trajectory no longer needs fine detection, then the target line of sight is determined by combining the satellite position, velocity and attitude data, the target angular trajectory is generated and the target image plane trajectory and target dynamic slice data are output;

[0071] Step S6: Combining the target image plane trajectory and target dynamic slice data, calculate and generate satellite payload imaging parameters and preprocessing parameters in real time to complete adaptive satellite payload imaging and preprocessing;

[0072] Step S7: Receive the target angular trajectory obtained by other satellites, combine it with the target angular trajectory generated by this satellite, perform target binary or multi-satellite fusion positioning, output the target three-dimensional trajectory, and perform target parameter estimation.

[0073] This invention employs target pre-detection to achieve real-time processing of massive input data from large-area, high-resolution onboard cameras at ultra-high rates. It further integrates intelligent precision detection with a unified trajectory management method to improve target detection accuracy and increase the target trajectory detection length. This method is suitable for real-time, high-precision onboard information processing under ultra-high data rates in space-based systems, effectively reducing the bandwidth required for satellite-to-ground data transmission and improving the availability of space-based information. The method flow of this invention is as follows: Figure 1 As shown.

[0074] Step S1 includes:

[0075] Step S1.1: Receive camera imaging data in real time, find the frame header and frame count, and complete the data frame extraction in the form of a bitstream. The frame extraction frequency can be updated.

[0076] Step S1.2: Real-time data unpacking is completed, and camera imaging auxiliary data (including at least satellite position and velocity, attitude, integration time, gain, etc. during imaging) and image data sequences are generated using imaging time and frame count as identifiers.

[0077] Step S1.3: Cache image data from several frames with a certain interval between them and the current frame to generate an interval cached image sequence, which serves as the input for subsequent image registration and image differencing; cache image data from several consecutive frames before the current frame to generate a continuous cached image sequence, which serves as the input for generating target-to-be-detected sequence slices and target dynamic slice data; the cached image data is updated in real time each time a frame of data is input; the total number of frames, the interval number of frames, and the number of continuous cached frames in the interval cached image sequence can be updated according to the task scenario.

[0078] Step S1.4: After performing inter-frame registration between the current frame image and the interval buffer image sequence, the difference between each pixel and the interval buffer image is calculated to generate a difference image sequence.

[0079] Step S1.5: Perform non-uniformity correction and grayscale transformation on the current frame image and the differential image sequence to be pre-detected for the target;

[0080] If the original DN value of the pixel in the i-th row and j-th column of the image is x ij The corresponding non-uniformity correction parameter is k. ij f ij Its corrected pixel DN value is y ij =k ij ·x ij +f ij Non-uniformity correction parameters can be set pixel-by-pixel and updated via annotation.

[0081] Further image grayscale transformation is performed on the image after non-uniformity correction. Linear stretching is used; if the pixel value before transformation is x and the pixel value after transformation is y, the expression is:

[0082]

[0083] Parameters a and b can be set as pre-defined parameters and adaptive parameters, respectively. Pre-defined parameters are fixed values ​​set on the ground and can be updated via injection. Adaptive parameters are calculated and updated in real time according to step S6.

[0084] Step S2 includes:

[0085] Step S2.1: Perform pre-detection on the current frame image and the difference sequence image, and output the target point set {(m,n,y} in the image. m,n )}; where m and n are the row and column numbers of the target pixel detected in the current frame.

[0086] The target pre-detection algorithm depends on the on-board computing resources and detection accuracy requirements. It can adopt the traditional differential weighted threshold method or intelligent algorithms such as deep learning.

[0087] The current frame image is X0, X 0-i, X 0-j, ,X 0-k For a sequence of differential images with intervals of i, j, k frames from the current frame, then for the weighted image X = w1X0 + w2X 0-i +w2X 0-j +w3X 0-k Median filtering and threshold segmentation are performed. Image pixels larger than the threshold are identified as target points, forming a target point set {(m,n,ym,n)}. Among them, w1,w2,w3,w4 are image weights, which sum to 1 and can be updated by upsampling.

[0088] Step S2.2, for the target point set {(m,n,y) of the current frame m,n Perform connected domain clustering calculations, and calculate the centroid of a subset of the clustered target point set. The row and column numbers of the centroids are used as the positions of the detected targets on the current frame image. Number all targets detected in the current frame, match the target imaging time, frame count and alignment auxiliary data, generate the pre-detection results of the current frame image and send them to trajectory fusion association and management.

[0089] Step S3 includes:

[0090] Step S3.1: Simultaneously receive the target pre-detection result and fine detection result of the current frame. If the intelligent fine detection detects a new result, replace or supplement the target detection result of each frame with the fine detection result, and re-associate the target trajectory of all image frames with replaced or supplemented target detection results. If there is no fine detection result, associate the target trajectory according to the pre-detection result of the current frame image.

[0091] Step S3.2: After the current frame trajectory is associated, the starting, merging, continuation, and termination of the target image plane trajectory batch number are processed to generate and maintain the target image plane trajectory library. When 3 out of 5 frames in a target image plane trajectory in the target trajectory library cannot be associated, the target intelligent fine detection algorithm is called, and the target fine detection result is output to step S3.1; if no new associated detection points are found in 20 consecutive frames of the target trajectory, the trajectory is terminated. The trajectory termination condition can be updated according to the on-orbit status.

[0092] Step S3.3: When it is determined that target precision detection is required, a target precision detection slice sequence is generated based on the target trajectory and trajectory prediction results, combined with the cached continuous image frame sequence, and output to the target precision detection S4. The size of the target precision detection slice is updated on the track as needed.

[0093] Step S4 includes:

[0094] Step S4.1: After receiving the multi-track target fine detection slice sequence of the current frame image, the intelligent target fine detection algorithm based on the spatiotemporal fusion deep learning model is simultaneously input in batches. The length S of the target fine detection slice sequence is adjusted according to the timeliness of intelligent target fine detection (5≤S≤length of continuous cached image sequence) and can be updated on the track.

[0095] Step S4.2: After outputting the detection points, the results are clustered in the connected domain and the centroids are calculated to generate S-frame target precision detection results. The process then returns to step S3 for trajectory fusion, association, and management. To ensure real-time performance, intelligent precision detection is completed within 5 frame periods, and the results are packetized and transmitted.

[0096] Step S5 includes:

[0097] Step S5.1: When it is determined that fine detection is not required, the target image plane trajectory is received to determine the target line of sight. Auxiliary data is aligned with the target image plane trajectory according to the imaging time and frame count.

[0098] Step S5.2: Based on the satellite's position, velocity, and attitude during imaging, and combined with the camera imaging model and parameters, determine the satellite-target pointing line of sight, and generate the local satellite target angular trajectory output to step S7.

[0099] Step S6 includes:

[0100] Step S6.1: Based on the target image plane trajectory and dynamic slices, determine whether the response of the target area exceeds the response range of the camera detector, and generate imaging adaptive control parameters such as satellite payload detector gain and integration time.

[0101] Step S6.2: Based on the target image plane trajectory and dynamic slices, automatically calculate and obtain the adaptive coefficients for grayscale correction of the current frame, and output them to step S1 as the basis for adjusting the adaptive parameters for grayscale correction of the next frame.

[0102] The adaptive coefficients a and b are calculated according to the following formula:

[0103]

[0104] Where m is the mean of all pixels in the dynamic slice, σ is the standard deviation of all pixels in the dynamic slice, and K is a preset scaling factor that can be updated on track.

[0105] This invention also provides an information processing system that integrates on-board real-time target pre-detection and intelligent precision detection, comprising:

[0106] Module M1: Receives real-time imaging data from satellite payloads and performs preprocessing processes such as data frame extraction, data unpacking, image data caching, grayscale conversion, image registration, and image differentiation in real time.

[0107] Module M2: Determines whether the area where the target to be detected is located is a key observation area. If so, it directly connects to intelligent precision detection. If not, it performs target pre-detection and outputs suspected targets and slices.

[0108] Module M3: Receives target pre-detection and intelligent fine detection results, performs target image plane trajectory association and management, and determines whether to perform target intelligent fine detection;

[0109] Module M4: If fine target detection is required, the target slice sequence to be finely detected is read in for intelligent fine target detection, which further improves the target detection accuracy and outputs the fine detection results. Then, trajectory association and management are performed.

[0110] Module M5: If it is determined that the current target image plane trajectory no longer needs fine detection, the target line of sight is determined by combining satellite position, velocity and attitude data, the target angular trajectory is generated and the target image plane trajectory and dynamic slice data are output;

[0111] Module M6: Combines target image plane trajectory and target dynamic slice data to calculate and generate satellite payload imaging parameters and preprocessing parameters in real time, and completes adaptive satellite payload imaging and preprocessing;

[0112] Module M7: Receives the target angular trajectory obtained by other satellites, combines it with the target angular trajectory generated by the local satellite, performs target dual-satellite or multi-satellite fusion positioning, outputs the target's three-dimensional trajectory, and performs target parameter estimation.

[0113] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0114] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. An information processing method that integrates on-board real-time target pre-detection and intelligent precision detection, characterized in that, Includes the following steps: Step S1: Receive real-time imaging data from the camera and complete preprocessing in real time, including data frame extraction, data unpacking, image data buffering, grayscale transformation, image registration, and image differencing; Step S2: Determine whether the target to be pre-inspected is located in the preset key observation area. If so, perform intelligent fine detection directly. If not, perform target pre-inspection and output the suspected target and slice. Step S3: Receive the target pre-detection or intelligent fine detection results, perform target image plane trajectory association and management, and determine whether to perform target intelligent fine detection. If yes, proceed to step S4; otherwise, proceed to step S5. Step S4: Read in the target slice sequence to be precisely detected, perform intelligent precision detection of the target, output the precision detection results, and perform trajectory association and management; Step S5: Combine satellite position, velocity and attitude data to determine the target line of sight, generate the target angular trajectory and output the target image plane trajectory and target dynamic slice data; Step S6: Combining the target image plane trajectory and target dynamic slice data, calculate and generate satellite payload imaging parameters and preprocessing parameters in real time to complete adaptive satellite payload imaging and preprocessing; Step S7: Receive the target angular trajectory obtained by other satellites, combine it with the target angular trajectory generated by the local satellite, perform target binary or multi-satellite fusion positioning, output the target three-dimensional trajectory, and perform target parameter estimation; Step S1 includes: Step S1.1: Receive camera imaging data in real time, find the frame header and frame count, and extract data frames in the form of a bitstream; Step S1.2: Real-time data unpacking is completed, and camera imaging auxiliary data and image data sequences are generated using imaging time and frame count as identifiers; Step S1.3: Cache image data from multiple frames with a preset interval between the current frame and the image, and generate an interval cached image sequence as input for subsequent image registration and image differencing; cache image data from multiple consecutive frames before the current frame, and generate a continuous cached image sequence as input for the target slice sequence to be finely detected and the generation of target dynamic slice data; the cached image data is updated in real time once when each frame of data is input; Step S1.4: After performing inter-frame registration between the current frame image and the interval buffer image sequence, the difference between the current frame image and the interval buffer image sequence is calculated pixel by pixel to generate a difference image sequence; Step S4 includes: Step S4.1: After receiving multiple target slice sequences to be finely detected from the current frame image, input the preset intelligent target fine detection algorithm based on the spatiotemporal fusion deep learning model in batches. The length S of the target fine detection slice sequence is adjusted according to the timeliness of intelligent target fine detection, where 5≤S≤length of continuous cached image sequence. Step S4.2: After outputting the detection points, cluster them in the connected domain and find the centroids to generate S-frame target fine detection results. Then return to step S3 for trajectory fusion association and management. Step S5 includes: Step S5.1: When it is determined that intelligent fine detection is not required, the target image plane trajectory is received to determine the target line of sight, and the auxiliary data is aligned with the target image plane trajectory according to the imaging time and frame count; Step S5.2: Based on the satellite's position, velocity, and attitude during imaging, and combined with the camera imaging model and parameters, determine the satellite-target pointing line of sight and generate the local satellite target angular trajectory; Step S6 includes: Step S6.1: Based on the target image trajectory and target dynamic slice data, determine whether the target region response exceeds the camera detector response range, and generate satellite payload detector imaging adaptive control parameters, including gain and integration time; Step S6.2: Calculate and obtain the adaptive coefficient of grayscale correction for the current frame based on the target image trajectory and target dynamic slice data, and use it as the basis for adjusting the adaptive parameters of grayscale correction for the next frame; Adaptive coefficients , Calculate using the following formula: in, This is the average value of all pixels in the dynamic slice; The standard deviation of all pixels in the dynamic slice; This is a preset proportionality coefficient; This represents the maximum pixel value.

2. The information processing method for fusing on-board real-time target pre-detection and intelligent precision detection according to claim 1, characterized in that, Non-uniformity correction and grayscale transformation are performed on the current frame image and the difference image sequence for target pre-detection. Let the original DN value of the pixel in the i-th row and j-th column of the image be... The corresponding non-uniformity correction parameter is , Its corrected pixel DN value ; A grayscale transformation is performed on the image after non-uniformity correction, using linear stretching. If the pixel value before transformation is... The transformed pixel value The expression is: in, , These are pre-set parameters and adaptive parameters, respectively. The pre-set parameters are fixed values ​​set on the ground. This represents the maximum pixel value.

3. The information processing method for fusing on-board real-time target pre-detection and intelligent precision detection according to claim 1, characterized in that, Step S2 includes: Step S2.1: Perform pre-detection on the current frame image and the difference image sequence, and output the target point set in the image. Where m and n are the row and column numbers of the target pixels detected in the current frame image; The pixel values ​​in row m and column n of the target point set; The current frame image is , Interval with the current frame A sequence of differencing images of frames, weighted images Median filtering and thresholding are performed; image pixels larger than the threshold are identified as target points, forming a target point set. ;in, The image is weighted, and the sum of the weights is 1. Step S2.2, for the target point set of the current frame image Perform connected domain clustering calculations and calculate the centroid of a subset of the clustered target point set. The row and column numbers of the centroids are used as the positions of the detected targets on the current frame image. Number all detected targets in the current frame image, match the target imaging time, frame count and alignment auxiliary data, generate the pre-detection results of the current frame image and perform trajectory fusion association and management.

4. The information processing method for fusing on-board real-time target pre-detection and intelligent precision detection according to claim 1, characterized in that, Step S3 includes: Step S3.1: Simultaneously receive the target pre-detection result and the intelligent fine detection result of the current frame image. If the intelligent fine detection yields a new result, replace or supplement the target pre-detection result of each frame image with the fine detection result, and re-associate the target trajectory of all replaced or supplemented target pre-detection result image frames. If there is no intelligent fine detection result, perform target trajectory association of the pre-detection result of the current frame image. Step S3.2: After associating the current frame image trajectory, perform start, merge, continuation, and termination processing of the target image plane trajectory batch number to generate and maintain the target image plane trajectory library; when multiple frames of a target image plane trajectory in the target trajectory library cannot be associated within a preset range, the target intelligent fine detection algorithm is called to output the target fine detection result; if the consecutive frames of the target trajectory exceed the preset threshold and there are no new associated detection points, it indicates that the trajectory has been terminated. Step S3.3: When it is determined that intelligent fine detection of the target is required, a target slice sequence to be finely detected is generated based on the target trajectory and trajectory prediction results, combined with the continuous cached image sequence, and then intelligent fine detection of the target is performed.

5. An information processing system integrating on-board real-time target pre-detection and intelligent precision detection, characterized in that, The information processing method for fusing on-board real-time target pre-detection and intelligent precision detection as described in any one of claims 1 to 4 includes the following modules: Module M1: Receives real-time imaging data from the camera and performs preprocessing in real time, including data frame extraction, data unpacking, image data buffering, grayscale transformation, image registration, and image differencing. Module M2: Determines whether the target to be pre-inspected is located in the preset key observation area. If so, it directly performs intelligent fine detection; otherwise, it performs target pre-detection and outputs the suspected target and slice. Module M3: Receives the target pre-detection or intelligent fine detection results, performs target image plane trajectory association and management, and determines whether to perform target intelligent fine detection. If so, it triggers module M4; otherwise, it triggers module M5. Module M4: Reads in the target slice sequence to be precisely detected, performs intelligent precision detection of the target, outputs the precision detection results, and performs trajectory association and management; Module M5: Combines satellite position, velocity and attitude data to determine the target line of sight, generates the target angular trajectory and outputs the target image plane trajectory and target dynamic slice data; Module M6: Combines target image plane trajectory and target dynamic slice data to calculate and generate satellite payload imaging parameters and preprocessing parameters in real time, and completes adaptive satellite payload imaging and preprocessing; Module M7: Receives the target angular trajectory obtained by other satellites, combines it with the target angular trajectory generated by the local satellite, performs target dual-satellite or multi-satellite fusion positioning, outputs the target's three-dimensional trajectory, and performs target parameter estimation.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the information processing method for fusing real-time onboard target pre-detection and intelligent precision detection as described in any one of claims 1 to 4.

Images