Deep space exploration camera image autonomous windowing control system and method

Abstract

The application discloses a deep space exploration camera image autonomous windowing control system and method, and relates to the technical field of deep space exploration. A programmable logic gate array chip drives an image sensor to continuously output full-frame color image data at a fixed frame rate; an image buffer is connected with the programmable logic gate array chip; an image data receiving module receives the full-frame color image data, writes the full-frame color image data into the image buffer for caching, and outputs the full-frame color image data to a subsequent module for processing; an image preprocessing module converts the full-frame color image data into a grayscale image; a center position calculation module calculates the center position of a detection target according to the grayscale image; a windowing parameter calculation module calculates image autonomous windowing parameters according to the center position; and an image output module reads windowed color image data from the cached full-frame color image data according to the image autonomous windowing parameters and outputs the windowed color image data. The application realizes the functions of camera autonomous identification and calculation of the position of an asteroid target in an image, autonomous windowing output of the image, and improves the efficiency of the deep space exploration camera.

Description

Technical Field

[0001] This invention relates to the field of deep space exploration technology, and in particular to a control system and method for autonomous windowing of images from a deep space exploration camera. Background Technology

[0002] Deep space exploration refers to exploration activities on the Moon and other celestial bodies or space environments. Deep space exploration is of great significance for studying major scientific questions such as the formation and evolution of the solar system and the universe, and the origin and evolution of life, and is an important direction of human space activities.

[0003] Asteroids are a hot topic in deep space exploration. They preserve original information about the formation and evolution of the solar system, serving as "living fossils" for studying its origins and providing scientific clues. Asteroid impacts have historically caused numerous environmental catastrophes and mass extinctions on Earth, directly threatening human survival and development. Asteroid exploration can deepen our understanding of asteroid orbital evolution and provide scientific evidence for researching methods and approaches to responding to asteroid impacts.

[0004] Optical imaging cameras are one of the main scientific payloads in asteroid deep space exploration. The direct driving force behind asteroid deep space exploration is to gain a close and realistic understanding of the entirety of asteroids, including their size, shape, rotation period, and orbital characteristics. Traditional ground-based observations are limited by distance, resulting in very limited resolution of asteroid morphology. Therefore, close-range imaging is the primary means of comprehensively and deeply understanding the characteristics of asteroids.

[0005] A key characteristic of asteroids is their small size. At different detection distances, asteroids appear as images of varying sizes against a dark deep-space background. The asteroid's size within the image is inversely proportional to the distance between the camera and the asteroid target. At medium to long distances, the effective image data area containing the asteroid target occupies only a small portion of the entire image, with a large amount of data consisting of meaningless dark deep-space background data. Due to the limitations of deep-space probes' Earth-to-ground transmission distance, data transmission bandwidth is extremely limited. Therefore, windowing the images, retaining only the windowed image containing the target, and discarding meaningless dark deep-space background images can effectively reduce the amount of data transmitted and fully utilize the data transmission bandwidth.

[0006] Due to limitations in the platform's pointing accuracy, the position of an asteroid target in camera images is uncertain. Given the long distances involved in deep space exploration, a single telemetry and control interaction can be delayed by tens of minutes, making it difficult to manually set image windows. Summary of the Invention

[0007] The purpose of this invention is to provide an autonomous windowing control system and method for deep space exploration cameras, so as to realize the function of the camera autonomously identifying and calculating the position of asteroid targets in the image and autonomously opening windows to output the image, thereby improving the efficiency of deep space exploration cameras.

[0008] To achieve the above objectives, embodiments of the present invention provide the following solutions:

[0009] An autonomous windowing control system for images from a deep space exploration camera includes: an image sensor, a programmable gate array chip, and an image buffer;

[0010] The programmable gate array chip drives the image sensor to continuously output full-frame color image data at a fixed frame rate;

[0011] The programmable gate array chip includes: an image sensor driving module, an image data receiving module, an image preprocessing module, a center position calculation module, a windowing parameter calculation module, and an image output module;

[0012] The image sensor driving module is used to: send driving signals to drive the image sensor;

[0013] The image data receiving module is used to: receive the full-frame color image data, write it into the image buffer, and output it to subsequent modules for processing;

[0014] The image preprocessing module is used to: convert the full-frame color image data into a grayscale image;

[0015] The center position calculation module is used to: calculate the center position of the detected target based on the grayscale image;

[0016] The windowing parameter calculation module is used to: calculate the autonomous windowing parameters of the image based on the center position;

[0017] The image output module is used to: read windowed color image data from the full-frame color image data stored in the image buffer according to the image autonomous windowing parameters and output it;

[0018] The image buffer is connected to the programmable gate array chip, and the image buffer is used to store the full-frame color image data.

[0019] Optionally, the image preprocessing module includes:

[0020] The format conversion unit is used to convert full-frame color image data from the RGB color space to the YUV color space of a grayscale image;

[0021] The Y component of the YUV color space represents the grayscale information of the grayscale image; the formula for calculating the Y component is:

[0022] Y = 0.299R + 0.587G + 0.114B;

[0023] Wherein, Y is the Y component, R is the R component of the RGB color space, G is the G component of the RGB color space, and B is the B component of the RGB color space.

[0024] Optionally, the center position calculation module includes:

[0025] The background threshold calculation submodule is used to determine the background threshold; the background threshold is determined based on the image background acquisition value of the grayscale image and a threshold parameter; the specific calculation process is as follows:

[0026] Background threshold = Image background acquisition value + Threshold parameter;

[0027] The image background acquisition value is determined by acquiring grayscale information at the edges of the grayscale image; the threshold parameter is a preset value.

[0028] The target line segment extraction and aggregation submodule is used for:

[0029] The target line segment is extracted from any row of pixels in the grayscale image based on the background threshold; the target line segment is a continuous target pixel; the target pixel is a pixel whose grayscale value is greater than the background threshold.

[0030] Sort all target line segments in any row of pixels in the grayscale image in descending order of length to obtain the three longest target line segments; and record the starting pixel position, ending pixel position and length of any of the three longest target line segments.

[0031] Based on the positional relationship of the three longest target line segments in any two adjacent rows of pixels in the grayscale image, the connectivity of the target line segments is determined; if the starting pixel position and the ending pixel position of the target line segments between any two rows are adjacent or coincident, then the two target line segments are determined to be connected; otherwise, the two target line segments are determined to be disconnected; all connected target line segments are combined into a target region; and the starting pixel row number, starting pixel column number, ending pixel row number, ending pixel column number, and the size of the target region are recorded.

[0032] The target line segments are extracted row by row from all pixels in the grayscale image, and their connectivity is determined to obtain multiple target regions. The sizes of the multiple target regions are compared to obtain the starting pixel row number, starting pixel column number, ending pixel row number, ending pixel column number, and size of the largest target region in the grayscale image. Based on the starting pixel row number, starting pixel column number, ending pixel row number, and ending pixel column number of the largest target region, the center pixel row number and center pixel column number of the largest target region are calculated. The position of the intersection of the center pixel row number and center pixel column number is the initial position of the target.

[0033] The target center location calculation submodule is used for:

[0034] Using the initial target position of the previous grayscale image as the center position, a target range is defined in the current grayscale image; the target range is centered on the center position; the size of the target range is N times the largest target area of ​​the previous grayscale image;

[0035] Within the target range, threshold segmentation is performed using the background threshold of the previous frame grayscale image to obtain target pixels and background pixels;

[0036] Calculate the average row number and average column number of all target pixels within the target range; the average row number and average column number constitute the center position of the detected target.

[0037] Optionally, the windowing parameter calculation module specifically includes:

[0038] The window start row calculation unit is used to calculate the window start row m in the image autonomous windowing parameters based on the center position of the detected target. W The formula is as follows:

[0039]

[0040] The window start column calculation unit is used to calculate the window start column n in the image autonomous windowing parameters based on the center position of the detected target. W The formula is as follows:

[0041]

[0042] Where M is the total number of rows in the full-frame color image, and N is the total number of columns in the full-frame color image. W N is the number of rows in the window. W m is the number of columns in the window. t n is the row number of the center position. t The center position column number; the number of rows in the window M W And the number of columns in the window N W This is the default value.

[0043] Optionally, the image output module is used for:

[0044] According to the window start line m W The starting column n of the window W Number of rows in window M W And the number of columns in the window N W Read and output windowed color image data from cached full-frame color image data.

[0045] To achieve the above objectives, embodiments of the present invention also provide the following solutions:

[0046] A method for autonomous windowing control of images from a deep space exploration camera includes:

[0047] Send a drive signal to drive the image sensor to acquire full-frame color image data that is continuously output at a fixed frame rate;

[0048] The full-frame color image data is written into the image buffer cache;

[0049] Convert the full-frame color image data into a grayscale image;

[0050] Calculate the center position of the target based on the grayscale image;

[0051] Calculate the image autonomous windowing parameters based on the center position;

[0052] The windowed color image data is read from the cached full-frame color image data and output according to the image autonomous windowing parameters.

[0053] Optionally, converting the full-frame color image data into a grayscale image includes:

[0054] Convert full-frame color image data from RGB color space to YUV color space;

[0055] The Y component of the YUV color space represents the grayscale information of the grayscale image; the formula for calculating the Y component is:

[0056] Y = 0.299R + 0.587G + 0.114B

[0057] Wherein, Y is the Y component, R is the R component of the RGB color space, G is the G component of the RGB color space, and B is the B component of the RGB color space.

[0058] Optionally, calculating the center position of the target based on the grayscale image includes:

[0059] The target pixels and background pixels in each frame of the grayscale image are divided according to a background threshold; the background threshold is determined based on the image background acquisition value and threshold parameters of the grayscale image; the specific calculation process is as follows:

[0060] Background threshold = Image background acquisition value + Threshold parameter;

[0061] The image background acquisition value is determined by acquiring grayscale information at the edges of the grayscale image; the threshold parameter is a preset value.

[0062] Target line segments are extracted from any row of pixels in the grayscale image based on a background threshold; the target line segments are continuous target pixels; the target pixels are pixels with grayscale values ​​greater than the background threshold.

[0063] Sort all target line segments in any row of pixels in the grayscale image in descending order of length to obtain the three longest target line segments; and record the starting pixel position, ending pixel position and length of any of the three longest target line segments.

[0064] Based on the positional relationship of the three longest target line segments in any two adjacent rows of pixels in the grayscale image, the connectivity of the target line segments is determined; if the starting pixel position and the ending pixel position of the target line segments between any two rows are adjacent or coincident, then the two target line segments are determined to be connected; otherwise, the two target line segments are determined to be disconnected; all connected target line segments are combined into a target region; and the starting pixel row number, starting pixel column number, ending pixel row number, ending pixel column number, and the size of the target region are recorded.

[0065] The target line segments are extracted row by row from all pixel rows in the grayscale image, and their connectivity is determined to obtain multiple target regions. The sizes of the multiple target regions are compared to obtain the starting pixel row number, starting pixel column number, ending pixel row number, ending pixel column number, and region size of the largest target region in the grayscale image. Based on the starting pixel row number, starting pixel column number, ending pixel row number, and ending pixel column number of the largest target region, the center pixel row number and center pixel column number of the largest target region are calculated. The position of the intersection of the center pixel row number and center pixel column number is the initial position of the target.

[0066] Using the initial target position of the previous grayscale image as the center position, a target range is defined in the current grayscale image; the target range is centered on the center position; the size of the target range is N times the largest target area of ​​the previous grayscale image;

[0067] Within the target range, threshold segmentation is performed using the background threshold of the previous frame grayscale image to obtain target pixels and background pixels;

[0068] Calculate the average row number and average column number of all target pixels within the target range; the average row number and average column number constitute the center position of the detected target.

[0069] Optionally, calculating the image autonomous windowing parameters based on the center position specifically includes:

[0070] The window start row m in the image autonomous windowing parameters is calculated based on the center position of the detected target. W The formula is as follows:

[0071]

[0072] The window start column n in the image autonomous windowing parameters is calculated based on the center position of the detected target. W The formula is as follows:

[0073]

[0074] Where M is the total number of rows in the full-frame color image, and N is the total number of columns in the full-frame color image. W N is the number of rows in the window. W m is the number of columns in the window. t n is the row number of the center position. t The center position column number; the number of rows in the window M W And the number of columns in the window N W This is the default value.

[0075] Optionally, outputting windowed color image data based on the image autonomous windowing parameters specifically includes:

[0076] According to the window start line m W The starting column n of the window W Number of rows in window M W And the number of columns in the window N W Read and output windowed color image data from cached full-frame color image data.

[0077] In this embodiment of the invention, a programmable gate array (PGA) chip drives an image sensor to output full-frame color image data. The PGA chip includes an image sensor driving module, an image data receiving module, an image preprocessing module, a center position calculation module, a windowing parameter calculation module, and an image output module. The application of the PGA chip enables the camera to autonomously identify and calculate the position of asteroid targets in the image without human intervention, thus improving the efficiency of deep space exploration cameras. Furthermore, the parallel execution of each module and pipelined processing greatly simplify the control flow and improve processing speed and reliability. The PGA chip can realize all control, processing, and transmission functions of the camera, simplifying the camera's circuit and mechanical structure design, saving size, weight, and power consumption, improving reliability, enhancing the camera's adaptability to operating temperature, mechanical environment, and radiation environment, and extending the camera's lifespan, which is in line with the limited resources of deep space exploration.

[0078] The image sensor driving module sends driving signals to drive the image sensor; the image data receiving module receives full-frame color image data, and the image preprocessing module converts the full-frame color image data into a grayscale image; the center position calculation module calculates the center position of the detected target based on the grayscale image; the windowing parameter calculation module calculates the autonomous windowing parameters of the image based on the center position; the system autonomously calculates the target position and windowing parameters without manual intervention, overcoming the disadvantages of long-distance deep space exploration and large telemetry and control delays. The calculated target position results can be used not only for determining the autonomous windowing parameters but also as a reference for automatic target exposure control and for guiding the target's pointing position during multi-payload joint detection.

[0079] The windowing parameter calculation module outputs windowed color image data based on the autonomous windowing parameters of the image. Considering the relatively small size of the target in deep space asteroid exploration, the windowing parameter calculation module realizes autonomous image windowing output control, outputting a windowed image containing the target, discarding meaningless dark background image data in deep space, effectively reducing the amount of data transmitted, and making full use of the limited data bandwidth of deep space exploration. Attached Figure Description

[0080] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0081] Figure 1 This is a schematic diagram of the structure of the autonomous windowing control system for deep space exploration camera images provided in an embodiment of the present invention;

[0082] Figure 2 A detailed structural diagram of the smallest unit of Bayer format full-frame color image data provided in an embodiment of the present invention;

[0083] Figure 3 This is a schematic diagram of image background acquisition values ​​provided in an embodiment of the present invention;

[0084] Figure 4 This is a schematic diagram illustrating the target line segment extraction, aggregation, and preliminary target position calculation provided in an embodiment of the present invention.

[0085] Figure 5 This is a schematic diagram illustrating the calculation of the target center position in an embodiment of the present invention.

[0086] Figure 6 This is a schematic diagram of the target center position calculation process provided in an embodiment of the present invention;

[0087] Figure 7 This is a flowchart illustrating the autonomous windowing control method for deep space exploration camera images provided in an embodiment of the present invention.

[0088] Symbol explanation:

[0089] Image sensor-1, programmable gate array chip-2, image sensor driver module-21, image data receiving module-22, image preprocessing module-23, center position calculation module-24, windowing parameter calculation module-25, image output module-26, image buffer-3. Detailed Implementation

[0090] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0091] The purpose of this invention is to provide a method and system for autonomous windowing control of images from a deep space exploration camera, in order to solve the problems existing in current deep space exploration cameras: low autonomy, inability to autonomously complete asteroid target position calculation and image windowing output; large amount of invalid data output by the camera in full-image mode, resulting in low utilization of storage and transmission resources; and large delay and low efficiency in manual windowing mode.

[0092] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0093] Figure 1An exemplary structure of the aforementioned autonomous windowing control system for deep space exploration cameras is shown. It includes: an image sensor 1, a programmable gate array chip 2, and an image buffer 3. Each component is described in detail below.

[0094] The programmable gate array chip 2 drives the image sensor 1 to continuously output full-frame color image data at a fixed frame rate;

[0095] In one example, image sensor 1 uses a CMOS chip, model CMV20000, with a full-frame resolution of 5120×3840, Bayer format encoding, and outputs full-frame color image data. The CMV20000 controls the start and end of exposure (i.e., exposure time) via an external synchronization signal, and uses an SPI interface to read and write internal registers to control the operating mode. The CMV20000 outputs images through a multi-channel serial LVDS interface, including one LVDS clock channel, 16 LVDS data channels, and one control channel.

[0096] The programmable gate array chip 2 includes: an image sensor driving module 21, an image data receiving module 22, an image preprocessing module 23, a center position calculation module 24, a windowing parameter calculation module 25, and an image output module 26.

[0097] In one example, a programmable gate array (FPGA) chip 2 is used as the core unit for control and data processing of the deep space exploration camera. The FPGA chip 2 is model JFM4VSX55RT, with a 5 million gate scale, and features high resistance to total radiation dose and single-event lockout threshold. The FPGA handles CMOS driving and control, image receiver buffer control and image processing, automatic exposure control, target position calculation and autonomous windowing, control command reception and processing, status feedback, image output, and thermal control. Using a single FPGA to complete all control and processing functions simplifies the design of the mechanical structure and circuitry, improving system integration and reliability.

[0098] Image sensor driver module 21 is used to send drive signals to drive image sensor 1;

[0099] The image data receiving module 22 is used to receive full-frame color image data, write it into the image buffer 3, and output it to subsequent modules for processing.

[0100] Image preprocessing module 23 is used to convert full-frame color image data into grayscale images;

[0101] Image preprocessing module 23 includes: format conversion unit.

[0102] The format conversion unit is used to convert the RGB color space of full-frame color image data into the YUV color space of grayscale images;

[0103] The Y component of the YUV color space represents the grayscale information of a grayscale image; the formula for calculating the Y component is:

[0104] Y = 0.299R + 0.587G + 0.114B;

[0105] Where Y is the Y component, R is the R component of the RGB color space, G is the G component of the RGB color space, and B is the B component of the RGB color space.

[0106] In one example, a common full-frame color image data format is Bayer format. The conversion is performed on the smallest unit of 2×2 pixels, with pixels arranged as follows: Figure 2 As shown.

[0107] The coefficients in the formula are approximated, so that the calculation process can be completed by shifting and adding operations, which is suitable for rapid implementation on FPGA platform.

[0108] After the above processing, the original Bayer format RGB image is converted to a grayscale image. The number of rows and columns in the grayscale image is half that of the original RGB image. For example, if the original RGB image output by the CMOS chip CMV20000 is 5120 columns × 3840 rows, the converted grayscale image will be 2560 columns × 1920 rows. Subsequent explanations will refer to grayscale images.

[0109] The center position calculation module 24 is used to calculate the center position of the detected target based on the grayscale image;

[0110] The center position calculation module 24 includes: a background threshold calculation submodule, a target line segment extraction and aggregation processing submodule, and a target center position calculation submodule.

[0111] The background threshold calculation submodule is used to determine the background threshold; the background threshold is determined based on the image background acquisition value of the grayscale image and the threshold parameter; the specific calculation process is as follows:

[0112] Background threshold = Image background acquisition value + Threshold parameter;

[0113] The image background value is determined by acquiring grayscale information at the edges of the grayscale image; the threshold parameter is a preset value.

[0114] In one example, the target asteroid (the true target) is generally considered to be in the center of the grayscale image, and the image background is obtained by collecting grayscale information from the corners of the grayscale image. Specifically, this involves analyzing the grayscale values... Figure 4 Each corner is used to take 4×4 pixels, and the average value is calculated to obtain 4 background values, such as... Figure 3 As shown, the real target is the detection target.

[0115] The four background values ​​are compared, and the minimum value among them is taken as the image background acquisition value. The purpose is to exclude situations where the target is in a corner or there are interfering targets in the corner.

[0116] The threshold parameters are set through experimentation and experience.

[0117] The target line segment extraction and aggregation submodule is used for:

[0118] Extract target line segments from any row of pixels in the grayscale image based on the background threshold; the target line segments are continuous target pixels; the target pixels are pixels with grayscale values ​​greater than the background threshold.

[0119] Sort all target line segments in any row of pixels in the grayscale image in descending order of length to obtain the three longest target line segments; and record the starting pixel position, ending pixel position and length of any of the three longest target line segments.

[0120] like Figure 4 As shown, the connectivity of target line segments is determined based on the positional relationship of the three longest target line segments in any two adjacent rows of pixels in the grayscale image. If the starting and ending pixel positions of target line segments between any two rows are adjacent or coincident, the two target line segments are determined to be connected; otherwise, the two target line segments are determined to be disconnected. All connected target line segments are combined into a target region, and the starting pixel row number, starting pixel column number, ending pixel row number, ending pixel column number, and the size of the target region are recorded.

[0121] The target line segments are extracted row by row from all pixels in the grayscale image, and their connectivity is determined to obtain multiple target regions. The sizes of the multiple target regions are compared to obtain the starting pixel row number, starting pixel column number, ending pixel row number, ending pixel column number, and size of the largest target region in the grayscale image. Based on the starting pixel row number, starting pixel column number, ending pixel row number, and ending pixel column number of the largest target region, the center pixel row number and center pixel column number of the largest target region are calculated. The position of the intersection of the center pixel row number and center pixel column number is the initial position of the target.

[0122] The target center location calculation submodule is used for:

[0123] Using the initial target position of the previous grayscale image as the center position, the target range is defined in the current grayscale image; the target range is centered on the aforementioned center position; the size of the target range is N times the largest target area of ​​the previous grayscale image;

[0124] Within the target range, threshold segmentation is performed using the background threshold of the previous grayscale image to obtain target pixels and background pixels;

[0125] Calculate the average row number and average column number of all target pixels within the target area; the average row number and average column number constitute the center position of the detected target;

[0126] Please see Figure 5 Calculate the average row number and average column number of all target pixels within the target range; the average row number and average column number constitute the center position of the detected target.

[0127] In one example, based on the characteristic of CMOS sensor images being output sequentially line by line, and considering the processing method of FPGA, target line segments are extracted line by line, and then the target line segments between lines are aggregated.

[0128] To avoid repeated access to the image data stream in the buffer and to simplify the process design, the processing of the current frame uses the background threshold calculation result of the previous frame image. Those skilled in the art can flexibly design the value of N, and the commonly used range is 1 to 3.

[0129] Using the initial position of the target in the previous frame as the center, the target range is defined in the row and column directions of the current frame by a size twice the number of target pixels calculated in the previous frame, to ensure that the target is within the defined range and to eliminate the influence of interfering targets.

[0130] like Figure 6 As shown in the above description, the entire process of determining the background threshold, extracting and aggregating the target line segments, and calculating the target center position is performed on three consecutive frames of images. The background threshold is determined in the first frame image and used for extracting and aggregating the target line segments in the second frame image to obtain the initial target position and the number of target pixels in the second frame image. Combined with the background threshold of the second frame image, the target range is delineated and the target center position is calculated in the third frame image.

[0131] This processing flow ensures a unidirectional data stream, from camera image input at the CMOS sensor to processing and buffering at the FPGA, and then output. As CMOS data is input line by line, the processes of background threshold determination, target line segment extraction and aggregation, and target center position calculation are completed sequentially in one go. This eliminates the need for repeated reading and processing of the buffered image, greatly simplifying the process and improving processing speed and reliability. This is provided that, under conditions of stable lighting and visual axis, the target's state remains essentially unchanged between adjacent image frames.

[0132] Window opening parameter calculation module 25 is used for:

[0133] Calculate the image's autonomous windowing parameters based on the center position;

[0134] The window opening parameter calculation module 25 specifically includes: the window starting row calculation unit and the window starting column calculation unit.

[0135] The window start row calculation unit is used to calculate the window start row m in the image autonomous windowing parameters based on the center position of the detected target. W The formula is as follows:

[0136]

[0137] The window start column calculation unit is used to calculate the window start column n in the image autonomous windowing parameters based on the center position of the detected target. W The formula is as follows:

[0138]

[0139] Where M is the total number of rows in the full-frame color image, and N is the total number of columns in the full-frame color image. W N is the number of rows in the window. W m is the number of columns in the window. t n is the row number of the center position. t The column number is the center position; the number of rows in the window is M. W And the number of columns in the window N W This is the default value.

[0140] Image output module 26 is used for:

[0141] Based on the window's starting line m W Window starting column n W Number of rows in window M W And the number of columns in the window N W Read and output windowed color image data from the full-frame color image data cached in image buffer 3.

[0142] Window opening parameters include the window's starting row, starting column, number of rows, and number of columns.

[0143] In one example, since the shape and attitude of the asteroid are unknown, a square window is generally chosen as the windowing parameter, that is, the number of rows and columns of the window are equal.

[0144] There are two ways to determine the number of rows and columns of the window: one is to calculate the number of rows and columns of the window automatically based on the number of target pixels obtained during the extraction and aggregation of target line segments, while leaving sufficient margin; the other is to estimate the size of the asteroid target in the image based on prior information about the size of the asteroid and the distance between the camera and the asteroid, combined with the parameters of the lens and CMOS image sensor, and manually set the size of the window in advance.

[0145] The first method has the advantage of requiring no manual intervention and executing entirely autonomously. Its disadvantage is that the image processing result is affected by factors such as lighting and random noise, potentially fluctuating within a certain range, leading to dynamic changes in the output image size. This makes subsequent image compression, transmission, and display restoration inconvenient. The second method, because the image size is pre-set manually, has the advantage of a fixed output image size, and autonomous windowing only affects the windowing position. Its disadvantage is that it requires prior information about the asteroid target.

[0146] After obtaining the windowing parameters, the address range for reading the cached image can be determined based on the correspondence between the image address in the cache and the image row and column number. The FPGA reads the image data from the corresponding address range, packages it into a specified format, and outputs it, thus realizing the autonomous windowing process of the asteroid deep space exploration camera.

[0147] Image buffer 3 is connected to programmable gate array chip 2, and image buffer 3 is used for:

[0148] Stores full-frame color image data.

[0149] In one example, image buffer 3 uses a single-chip SDRAM as the image data cache. The SDRAM model is 3DSD2G64VB4488, which has enhanced radiation resistance, a capacity of 2Gbit, and a 64-bit bus width. Different storage areas are partitioned within the single SDRAM chip to implement PING-PONG caching for image read / write operations.

[0150] The working process of the deep space exploration camera is as follows: the FPGA drives and controls the CMOS image sensor chip to work, the CMOS image sensor outputs images line by line, the FPGA receives and processes the images line by line, writes the images into the SDRAM buffer, and reads the image data from the buffered SDRAM according to the instructions and packages and outputs it.

[0151] The deep-space asteroid exploration camera in this embodiment of the invention uses a single FPGA architecture. A common alternative is an FPGA+DSP architecture. The FPGA handles image sensor driving and data interface processing, while the DSP performs image data calculations. The DSP has strong data processing capabilities and can implement relatively complex algorithms through software programming, but the processing time is relatively longer, and the system complexity, size, weight, and power consumption also increase accordingly.

[0152] In summary, in this embodiment of the invention, the image sensor outputs full-frame color image data captured by the deep space exploration camera according to the output command. The programmable gate array (PGA) chip includes: an image sensor driving module, an image data receiving module, an image preprocessing module, a center position calculation module, a windowing parameter calculation module, and an image output module. The application of the PGA chip enables the camera to autonomously identify and calculate the position of asteroid targets in the image without human intervention, thus improving the efficiency of the deep space exploration camera. Furthermore, the parallel execution of each module and pipelined processing greatly simplify the control process and improve processing speed and reliability. The PGA chip can realize all control, processing, and transmission functions of the camera, simplifying the camera's circuit and mechanical structure design, saving size, weight, and power consumption, improving reliability, enhancing the camera's adaptability to operating temperature, mechanical environment, and radiation environment, and extending the camera's lifespan, which is in line with the limited resources of deep space exploration.

[0153] The image sensor driving module sends driving signals to drive the image sensor; the image data receiving module receives full-frame color image data, and the image preprocessing module converts the full-frame color image data into a grayscale image; the center position calculation module calculates the center position of the detected target based on the grayscale image; the windowing parameter calculation module calculates the autonomous windowing parameters of the image based on the center position; the system autonomously calculates the target position and windowing parameters without manual intervention, overcoming the disadvantages of long-distance deep space exploration and large telemetry and control delays. The calculated target position results can be used not only for determining the autonomous windowing parameters but also as a reference for automatic target exposure control and for guiding the target's pointing position during multi-payload joint detection.

[0154] The windowing parameter calculation module outputs windowed color image data based on the autonomous windowing parameters of the image. Considering the relatively small size of the target in deep space asteroid exploration, the windowing parameter calculation module enables the autonomous windowing output of the image, outputting a windowed image containing the target, discarding meaningless dark background image data of deep space, effectively reducing the amount of data transmitted, and making full use of the limited data bandwidth of deep space exploration.

[0155] In one example, the full-frame image size of the camera is 5120×3840. The maximum window size is set to 1920×1920 and the minimum window size is 240×240. Under the maximum window size, the data volume is reduced by 81.25%, and under the minimum window size, the data volume is reduced by 99.71%.

[0156] To achieve the above objectives, embodiments of the present invention also provide the following solutions:

[0157] Please see Figure 7 An autonomous windowing control method for deep space exploration camera images, comprising:

[0158] Step 1: Send a drive signal to drive the image sensor to acquire full-frame color image data that is continuously output at a fixed frame rate.

[0159] Step 2: Write the full-frame color image data to the image buffer cache;

[0160] Step 3: Convert the full-color image data into a grayscale image.

[0161] Convert full-frame color image data from RGB color space to YUV color space;

[0162] The Y component of the YUV color space represents the grayscale information of a grayscale image; the formula for calculating the Y component is:

[0163] Y = 0.299R + 0.587G + 0.114B

[0164] Where Y is the Y component, R is the R component of the RGB color space, G is the G component of the RGB color space, and B is the B component of the RGB color space.

[0165] Step 4: Calculate the center position of the target based on the grayscale image; specifically including:

[0166] Step 41: Divide the target pixels and background pixels in each frame of the grayscale image according to the background threshold; the background threshold is determined based on the image background acquisition value and threshold parameters of the grayscale image; the specific calculation process is as follows:

[0167] Background threshold = Image background acquisition value + Threshold parameter;

[0168] The image background acquisition value is determined by acquiring grayscale information at the edges of the grayscale image; the threshold parameter is a preset value.

[0169] Step 42: Extract the target line segment from any row of pixels in the grayscale image based on the background threshold; the target line segment is a continuous target pixel; the target pixel is a pixel whose grayscale value is greater than the background threshold;

[0170] Sort all target line segments in any row of pixels in the grayscale image in descending order of length to obtain the three longest target line segments; and record the starting pixel position, ending pixel position and length of any of the three longest target line segments.

[0171] Based on the positional relationship of the three longest target line segments in any two adjacent rows of pixels in the grayscale image, the connectivity of the target line segments is determined; if the starting pixel position and the ending pixel position of the target line segments between any two rows are adjacent or coincident, then the two target line segments are determined to be connected; otherwise, the two target line segments are determined to be disconnected; all connected target line segments are combined into a target region; and the starting pixel row number, starting pixel column number, ending pixel row number, ending pixel column number, and the size of the target region are recorded.

[0172] The target line segments are extracted row by row from all pixel rows in the grayscale image, and their connectivity is determined to obtain multiple target regions. The sizes of the multiple target regions are compared to obtain the starting pixel row number, starting pixel column number, ending pixel row number, ending pixel column number, and region size of the largest target region in the grayscale image. Based on the starting pixel row number, starting pixel column number, ending pixel row number, and ending pixel column number of the largest target region, the center pixel row number and center pixel column number of the largest target region are calculated. The position of the intersection of the center pixel row number and center pixel column number is the initial position of the target.

[0173] Step 43: Using the initial target position of the previous grayscale image as the center position, define the target range in the current grayscale image; the target range is centered on the center position; the size of the target range is N times the maximum target area of ​​the previous grayscale image;

[0174] Within the target range, threshold segmentation is performed using the background threshold of the previous frame grayscale image to obtain target pixels and background pixels;

[0175] Calculate the average row number and average column number of all target pixels within the target range; the average row number and average column number constitute the center position of the detected target.

[0176] Step 5: Calculate the autonomous windowing parameters of the image based on the center position;

[0177] The window start row m in the image autonomous windowing parameters is calculated based on the center position of the detected target. W The formula is as follows:

[0178]

[0179] The window start column n in the image autonomous windowing parameters is calculated based on the center position of the detected target. W The formula is as follows:

[0180]

[0181] Where M is the total number of rows in the full-frame color image, and N is the total number of columns in the full-frame color image.W N is the number of rows in the window. W m is the number of columns in the window. t n is the row number of the center position. t The column number is the center position; the number of rows in the window is M. W And the number of columns in the window N W This is the default value.

[0182] Step 6: Read and output windowed color image data from the cached full-frame color image data according to the image autonomous windowing parameters; specifically including:

[0183] Based on the window's starting line m W Window starting column n W Number of rows in window M W And the number of columns in the window N W Output windowed color image data.

[0184] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0185] This document uses specific examples to illustrate the principles and implementation methods of the embodiments of the present invention. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of the embodiments of the present invention. At the same time, for those skilled in the art, there will be changes in specific implementation methods and application scope based on the ideas of the embodiments of the present invention. In summary, the content of this specification should not be construed as a limitation on the embodiments of the present invention.

Claims

1. An autonomous windowing control system for deep space exploration camera images, characterized in that, include: Image sensors, programmable gate array chips, and image buffers; The programmable gate array chip drives the image sensor to continuously output full-frame color image data at a fixed frame rate; The programmable gate array chip includes: an image sensor driving module, an image data receiving module, an image preprocessing module, a center position calculation module, a windowing parameter calculation module, and an image output module; The image sensor driving module is used to: send driving signals to drive the image sensor; The image data receiving module is used to: receive the full-frame color image data, write it into the image buffer, and output it to subsequent modules for processing; The image preprocessing module is used to: convert the full-frame color image data into a grayscale image; The center position calculation module is used to: calculate the center position of the detected target based on the grayscale image; The windowing parameter calculation module is used to: calculate the autonomous windowing parameters of the image based on the center position; The image output module is used to: read windowed color image data from the full-frame color image data stored in the image buffer according to the image autonomous windowing parameters and output it; The image buffer is connected to the programmable gate array chip, and the image buffer is used to store the full-frame color image data; The center position calculation module includes: The background threshold calculation submodule is used to determine the background threshold; the background threshold is determined based on the image background acquisition value of the grayscale image and a threshold parameter; the specific calculation process is as follows: Background threshold = Image background acquisition value + Threshold parameter; The image background acquisition value is determined by acquiring grayscale information at the edges of the grayscale image; the threshold parameter is a preset value. The target line segment extraction and aggregation submodule is used for: Target line segments are extracted from any row of pixels in the grayscale image based on a background threshold; the target line segments are continuous target pixels; the target pixels are pixels with grayscale values ​​greater than the background threshold. Sort all target line segments in any row of pixels in the grayscale image in descending order of length to obtain the three longest target line segments; and record the starting pixel position, ending pixel position and length of any of the three longest target line segments. Based on the positional relationship of the three longest target line segments in any two adjacent rows of pixels in the grayscale image, the connectivity of the target line segments is determined; if the starting pixel position and the ending pixel position of the target line segments between any two rows are adjacent or coincident, then the two target line segments are determined to be connected; otherwise, the two target line segments are determined to be disconnected; all connected target line segments are combined into a target region; and the starting pixel row number, starting pixel column number, ending pixel row number, ending pixel column number, and the size of the target region are recorded. The target line segments are extracted row by row from all pixels in the grayscale image, and their connectivity is determined to obtain multiple target regions. The sizes of the multiple target regions are compared to obtain the starting pixel row number, starting pixel column number, ending pixel row number, ending pixel column number, and size of the largest target region in the grayscale image. Based on the starting pixel row number, starting pixel column number, ending pixel row number, and ending pixel column number of the largest target region, the center pixel row number and center pixel column number of the largest target region are calculated. The position of the intersection of the center pixel row number and center pixel column number is the initial position of the target. The target center location calculation submodule is used for: Using the initial target position of the previous grayscale image as the center position, a target range is defined in the current grayscale image; the target range is centered on the center position; the size of the target range is N times the largest target area of ​​the previous grayscale image; Within the target range, threshold segmentation is performed using the background threshold of the previous frame grayscale image to obtain target pixels and background pixels; Calculate the average row number and average column number of all target pixels within the target range; the average row number and average column number constitute the center position of the detected target; The entire process of determining the background threshold, extracting and aggregating the target line segments, and calculating the target center position is performed on three consecutive frames of images. The background threshold is determined in the first frame of the image and used for extracting and aggregating the target line segments in the second frame of the image to obtain the initial target position and the number of target pixels in the second frame of the image. Combined with the background threshold of the second frame of the image, the target range is delineated and the target center position is calculated in the third frame of the image. The window opening parameter calculation module specifically includes: The window start row calculation unit is used to calculate the window start row m in the image autonomous windowing parameters based on the center position of the detected target. W The formula is as follows: ； The window start column calculation unit is used to calculate the window start column n in the image autonomous windowing parameters based on the center position of the detected target. W The formula is as follows: ； Where M is the total number of rows in the full-frame color image, and N is the total number of columns in the full-frame color image. W N is the number of rows in the window. W m is the number of columns in the window. t n is the row number of the center position. t The center position column number; the number of rows in the window M W And the number of columns in the window N W This is the default value.

2. The autonomous windowing control system for deep space exploration camera images according to claim 1, characterized in that, The image preprocessing module includes: The format conversion unit is used to convert full-frame color image data from RGB color space to YUV color space; The Y component of the YUV color space represents the grayscale information of the grayscale image; the formula for calculating the Y component is: ； Wherein, Y is the Y component, R is the R component of the RGB color space, G is the G component of the RGB color space, and B is the B component of the RGB color space.

3. The autonomous windowing control system for deep space exploration camera images according to claim 2, characterized in that, The image output module is used for: According to the window start line m W The starting column n of the window W Number of rows in window M W And the number of columns in the window N W It reads and outputs windowed color image data from the cached full-frame color image data.

4. A method for autonomous windowing control of images from a deep space exploration camera, characterized in that, include: Send a drive signal to drive the image sensor to acquire full-frame color image data that is continuously output at a fixed frame rate; The full-frame color image data is written into the image buffer cache; Convert the full-frame color image data into a grayscale image; Calculate the center position of the target based on the grayscale image; Calculate the image autonomous windowing parameters based on the center position; The windowed color image data is read from the cached full-frame color image data and output according to the image autonomous windowing parameters; Calculating the center position of the detected target based on the grayscale image includes: The target pixels and background pixels in each frame of the grayscale image are divided according to a background threshold; the background threshold is determined based on the image background acquisition value and threshold parameters of the grayscale image; the specific calculation process is as follows: Background threshold = Image background acquisition value + Threshold parameter; The image background acquisition value is determined by acquiring grayscale information at the edges of the grayscale image; the threshold parameter is a preset value. Target line segments are extracted from any row of pixels in the grayscale image based on a background threshold; the target line segments are continuous target pixels; the target pixels are pixels with grayscale values ​​greater than the background threshold. Sort all target line segments in any row of pixels in the grayscale image in descending order of length to obtain the three longest target line segments; and record the starting pixel position, ending pixel position and length of any of the three longest target line segments. Based on the positional relationship of the three longest target line segments in any two adjacent rows of pixels in the grayscale image, the connectivity of the target line segments is determined; if the starting pixel position and the ending pixel position of the target line segments between any two rows are adjacent or coincident, then the two target line segments are determined to be connected; otherwise, the two target line segments are determined to be disconnected; all connected target line segments are combined into a target region; and the starting pixel row number, starting pixel column number, ending pixel row number, ending pixel column number, and the size of the target region are recorded. The target line segments are extracted row by row from all pixel rows in the grayscale image, and their connectivity is determined to obtain multiple target regions. The sizes of the multiple target regions are compared to obtain the starting pixel row number, starting pixel column number, ending pixel row number, ending pixel column number, and region size of the largest target region in the grayscale image. Based on the starting pixel row number, starting pixel column number, ending pixel row number, and ending pixel column number of the largest target region, the center pixel row number and center pixel column number of the largest target region are calculated. The position of the intersection of the center pixel row number and center pixel column number is the initial position of the target. Using the initial target position of the previous grayscale image as the center position, a target range is defined in the current grayscale image; the target range is centered on the center position; the size of the target range is N times the largest target area of ​​the previous grayscale image; Within the target range, threshold segmentation is performed using the background threshold of the previous frame grayscale image to obtain target pixels and background pixels; Calculate the average row number and average column number of all target pixels within the target range; the average row number and average column number constitute the center position of the detected target; The entire process of determining the background threshold, extracting and aggregating the target line segments, and calculating the target center position is performed on three consecutive frames of images. The background threshold is determined in the first frame of the image and used for extracting and aggregating the target line segments in the second frame of the image to obtain the initial target position and the number of target pixels in the second frame of the image. Combined with the background threshold of the second frame of the image, the target range is delineated and the target center position is calculated in the third frame of the image. The calculation of image autonomous windowing parameters based on the center position specifically includes: The window start row m in the image autonomous windowing parameters is calculated based on the center position of the detected target. W The formula is as follows: ； The window start column n in the image autonomous windowing parameters is calculated based on the center position of the detected target. W The formula is as follows: ； Where M is the total number of rows in the full-frame color image, and N is the total number of columns in the full-frame color image. W N is the number of rows in the window. W m is the number of columns in the window. t n is the row number of the center position. t The center position column number; the number of rows in the window M W And the number of columns in the window N W This is the default value.

5. The autonomous windowing control method for deep space exploration camera images according to claim 4, characterized in that, Converting the full-frame color image data to a grayscale image includes: Convert full-frame color image data from RGB color space to YUV color space; The Y component of the YUV color space represents the grayscale information of the grayscale image; the formula for calculating the Y component is: ； Wherein, Y is the Y component, R is the R component of the RGB color space, G is the G component of the RGB color space, and B is the B component of the RGB color space.

6. The autonomous windowing control method for deep space exploration camera images according to claim 5, characterized in that, The specific steps for outputting windowed color image data based on the image autonomous windowing parameters include: According to the window start line m W The starting column n of the window W Number of rows in window M W And the number of columns in the window N W Read and output windowed color image data from cached full-frame color image data.

Images