Data acquisition system for observing high-dynamic target image under limited bandwidth condition
By employing a combined system of a visible light camera, an image compression device, and a wireless telemetry terminal under limited bandwidth conditions, real-time acquisition and transmission of high dynamic range target images were achieved, solving the problems of image loss and color distortion in the high dynamic range and ensuring target attitude acquisition during flight missions.
Patent Information
- Application Number
- CN202610083800.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-22
- Publication Date
- 2026-04-17
AI Technical Summary
Under limited bandwidth conditions, traditional methods struggle to capture and reconstruct high-dynamic target images while maintaining a high dynamic range, and simultaneously achieve real-time wireless telemetry downlink. Image loss or color distortion is particularly likely to occur at the moment of ignition and separation.
A combined system employing a visible light camera, image compression device, network switch, wireless telemetry terminal, and storage device ensures real-time acquisition and transmission of high dynamic range target images under limited bandwidth through high frame rate image data compression, color restoration, and automatic exposure technology.
It achieves the ability to capture and restore targets within a high dynamic range, while enabling real-time downlink under limited bandwidth transmission conditions, ensuring that the target attitude can be acquired throughout the flight mission, and solving the problems of image loss and color distortion.
Smart Images
Figure CN121887958A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image data processing technology, and in particular relates to a data acquisition system for observing high dynamic target images under limited bandwidth conditions. Background Technology
[0002] During flight, aircraft need to observe some high-dynamic scene targets such as ignition and separation. The traditional method is to acquire real-time images of the observation points and transmit them to the ground station for analysis via wireless telemetry. In actual use, the following problems are faced: (1) During the high-dynamic separation moment, due to the limitation of wireless downlink bandwidth, the video bit rate is generally low while ensuring clarity, which makes it difficult for the camera device to capture instantaneous attitude details. (2) During the ignition process, due to the influence of the flame, overexposure and other phenomena are easy to occur, resulting in the loss of the observation point image or color distortion. (3) The wireless telemetry downlink bit rate bandwidth is limited, and the telemetry downlink bit rate left for the image is generally no more than 1Mbps. Summary of the Invention
[0003] The technical problem solved by this invention is to overcome the shortcomings of the prior art and provide a data acquisition system for observing high dynamic target images under limited bandwidth conditions. This system ensures both the ability to capture and reconstruct targets under high dynamic range conditions and the requirement for real-time downlink under limited bandwidth transmission conditions, thereby enabling the acquisition of target attitude throughout the flight mission.
[0004] The objective of this invention is achieved through the following technical solution: a data acquisition system for observing high-dynamic target images under limited bandwidth conditions, comprising: a visible light camera, an image compression device, a network switch, a wireless telemetry terminal, and a storage device; wherein, the visible light camera acquires high frame rate image data of the target and transmits the high frame rate image data to the image compression device; the image compression device receives the high frame rate image data, compresses the high frame rate image data to obtain low frame rate image data, performs color restoration on the high frame rate image data to obtain color-restored high frame rate image data, and transmits both the color-restored high frame rate image data and low frame rate image data to the network switch; the network switch receives the color-restored high frame rate image data and low frame rate image data, transmits the low frame rate image data to the wireless telemetry terminal, and transmits the color-restored high frame rate image data to the storage device; the wireless telemetry terminal receives the low frame rate image data and wirelessly downloads the low frame rate image data to the ground; the storage device receives the color-restored high frame rate image data and stores the color-restored high frame rate image data.
[0005] In the data acquisition system for observing high dynamic target images under the aforementioned limited bandwidth conditions, the number of visible light cameras is two.
[0006] In the data acquisition system for observing high dynamic range target images under the above limited bandwidth conditions, the number of photogenerated charges in the image of the target captured by the visible light camera is: ; ; in, Let λ be the number of photogenerated charges in the image of the target captured by the visible light camera, h be Planck's constant, c be the speed of light, and λ be the angle between the target and the normal of the visible light camera. For target illuminance, For visible light cameras, F-number For signal-to-noise ratio, This refers to the gain of a visible light camera.
[0007] In the data acquisition system for observing high dynamic target images under the above limited bandwidth conditions, the signal-to-noise ratio of the visible light camera is 33.3dB, and the F-number of the surface visible light camera is 4.2.
[0008] In the data acquisition system for observing high dynamic target images under the above limited bandwidth conditions, the high frame rate image data is 300 frames per second, and the low frame rate image data is 24 frames per second.
[0009] In the data acquisition system for observing high dynamic target images under the aforementioned limited bandwidth conditions, during color restoration, the red component value of the current pixel is estimated using the average value of the existing red component pixels surrounding the current pixel. If the estimated red component value differs from the existing red component value by less than 60, it indicates that the current pixel is located in a flat area in the entire image. Conversely, if the estimated red component value differs from the existing red component value by more than 60, then the current pixel is very likely to be located in an edge and texture area in the entire image. The estimated result is corrected by adding a correction value to the estimated green component of the current pixel, and this correction term is provided by the red component information of the current pixel and its surrounding pixels.
[0010] In the data acquisition system for observing high dynamic target images under the aforementioned limited bandwidth conditions, the visible light camera uses a weighted average method to automatically expose high frame rate image data.
[0011] In the data acquisition system for observing high dynamic target images under the aforementioned limited bandwidth conditions, different weights are assigned to different imaging scenes during automatic exposure. Specifically, in scenes with a dark background and a bright target, the weight of areas with higher grayscale is set higher than that of other areas to prevent the target from being overexposed. In scenes with a bright background and a dark target, the weight of areas with lower grayscale is set higher than that of other areas to ensure clear imaging of the target even if some background is sacrificed. When the target is always located in the center of the field of view, the weight of the central area is set higher than that of other areas.
[0012] A method for acquiring data from high-dynamic target images under limited bandwidth conditions includes: a visible light camera acquiring high frame rate image data of the target, and transmitting the high frame rate image data to an image compression device; the image compression device receiving the high frame rate image data, compressing the high frame rate image data to obtain low frame rate image data, performing color restoration on the high frame rate image data to obtain color-restored high frame rate image data, and transmitting both the color-restored high frame rate image data and the low frame rate image data to a network switch; the network switch receiving the color-restored high frame rate image data and the low frame rate image data, transmitting the low frame rate image data to a wireless telemetry terminal, and transmitting the color-restored high frame rate image data to a storage device; the wireless telemetry terminal receiving the low frame rate image data and wirelessly downloading the low frame rate image data to the ground; and the storage device receiving the color-restored high frame rate image data and storing the color-restored high frame rate image data.
[0013] An electronic device includes: a memory for storing computer-readable instructions; and a processor for executing the computer-readable instructions to perform a data acquisition method for observing high dynamic target images under limited bandwidth conditions.
[0014] Compared with the prior art, the present invention has the following advantages: This invention ensures both the ability to acquire and reconstruct targets under high dynamic range conditions and the requirement for real-time downlink transmission under limited bandwidth conditions, enabling the acquisition of target attitude throughout the flight mission. Attached Figure Description
[0015] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a schematic diagram of the structure of a data acquisition system for observing high dynamic target images under limited bandwidth conditions provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of solar radiation illuminance and solar altitude angle at an altitude of 40km provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the Bayer data format provided in an embodiment of the present invention; Figure 4 This is a block diagram of automatic exposure adjustment provided in an embodiment of the present invention. Detailed Implementation
[0016] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.
[0017] Figure 1 This is a schematic diagram of the data acquisition system for observing high dynamic target images under limited bandwidth conditions provided in an embodiment of the present invention. Figure 1 As shown, the data acquisition system for observing high-dynamic target images under limited bandwidth conditions includes: a visible light camera, an image compression device, a network switch, a wireless telemetry terminal, and a storage device. The visible light camera acquires high-frame-rate image data of the target and transmits this high-frame-rate image data to the image compression device. The image compression device receives the high-frame-rate image data, compresses it to obtain low-frame-rate image data, performs color restoration on the high-frame-rate image data to obtain color-restored high-frame-rate image data, and transmits both the color-restored high-frame-rate image data and low-frame-rate image data to the network switch. The network switch receives the color-restored high-frame-rate image data and low-frame-rate image data, transmits the low-frame-rate image data to the wireless telemetry terminal, and transmits the color-restored high-frame-rate image data to the storage device. The wireless telemetry terminal receives the low-frame-rate image data and wirelessly downloads it to the ground. The storage device receives the color-restored high-frame-rate image data and stores it.
[0018] The system uses two visible light cameras. Two image sensors with high-sensitivity chips and color imaging capabilities serve as visible light cameras, enabling observation at two independent measurement points. Image compression equipment compresses the imaging data from the two visible light cameras into multiple data streams at 24 frames per second and 300 frames per second, which are then output to a network switch via a network interface. The network switch transmits the low-frame-rate image data to the telemetry terminal equipment and multiplexes the telemetry channels for real-time wireless downlink to the ground, transmitting the two visible light image data streams in a time-division multiplexing manner. Simultaneously, the network switch transmits the high-frame-rate image data to a telemetry storage device for generating high-bitrate, self-exposure, and color-self-recovery image data, such as... Figure 1 As shown.
[0019] The number of photogenerated charges in the image of the target captured by the visible light camera is: ; ; in, Let λ be the number of photogenerated charges in the image of the target captured by the visible light camera, h be Planck's constant, c be the speed of light, and λ be the angle between the target and the normal of the visible light camera. For target illuminance, For visible light cameras, F-number For signal-to-noise ratio, Gain of visible light cameras The signal-to-noise ratio of the visible light camera is 33.3dB, and the F-number of the surface visible light camera is 4.2.
[0020] This embodiment is used to detect the characteristics of a target. Therefore, the core of the electronic design of a visible light camera is to select a suitable visible light camera to acquire high-quality images and extract target feature information for specific target characteristics.
[0021] The imaging energy was analyzed, and the variation of the illuminance at an altitude of 40 km in the 0.45μm~0.7μm visible light band with the solar elevation angle was obtained using PcModWin atmospheric transmission simulation software. Due to the high altitude of the target and the scattering of solar radiation energy by the atmosphere, the ambient illuminance around the target did not change significantly when the solar elevation angle changed.
[0022] Taking the solar altitude angle as 30°, the solar irradiance at this point is 369.7 W / m². 2 Assume the target is a Lambertian surface with a surface reflectivity of 0.2, an optical system transmittance of 0.65, an atmospheric transmittance of 0.9, and a detector quantum efficiency of 0.55. Then, the integration time is set to 50 μs, the center wavelength to 0.55 μm, the F-number to 4.2, h to Planck's constant, c to the speed of light, and the area of each pixel to 54.76 μm². 2 At that time, the number of photogenerated charges in the image formed of the target is:
[0023]
[0024] In the formula, λ is the angle between the line connecting the two bodies and the x-axis, such as... Figure 2 As shown.
[0025] With the detector's internal gain set to K=1 and ignoring the effects of dark current noise (120e / s), the calculated signal-to-noise ratio is 33.3dB. The surface visible light camera's F-number of 4.2 can meet the imaging energy requirements.
[0026] High frame rate image data is 300 frames per second, and low frame rate image data is 24 frames per second. The visible light camera generates a 300-frame-per-second data stream in real time, which is then compressed into a 24-frame-per-second Ethernet data stream. The real-time bitrate is 640*480*24*16 / 1024 / 1024 / 200=0.63Mbps. The two cameras share the same channel and enter the telemetry frame format in a time-division manner. The real-time bitrate remains 0.63Mbps. The data is then sent to the wireless telemetry equipment via the network switch of the telemetry system, meeting the telemetry data volume requirement for a telemetry bitrate of less than 1Mbps for image data. Meanwhile, the compressor generates an Ethernet data stream with a color restoration and automatic exposure at 300 frames per second, with a real-time bitrate of 640*480*300*16 / 1024 / 1024 / 200=7.91Mbps. The bitrate of the two cameras is 15.82Mbps. The data is then sent to the storage device via the network switch of the telemetry system to store the high-frame-rate, high-fidelity image data.
[0027] In color restoration, the red component value of the current pixel is estimated by using the average value of the existing red component pixels around the current pixel. If the estimated red component value differs from the existing red component value by 60 or less, it means that the current pixel is in a flat area in the entire image. Conversely, if the estimated red component value differs from the existing red component value by 60 or less, then the current pixel is likely to be in an edge or textured area in the entire image. The estimated result is corrected by adding a correction value to the estimated green component of the current pixel, and this correction term is provided by the red component information of the current pixel and its surrounding pixels.
[0028] The compression device performs color restoration on the image data. The acquired Bayer format data provides only one color data for each pixel (red R, green G, or blue B). It is necessary to restore the RGB color information of each pixel, such as... Figure 3 As shown. Traditional hardware color interpolation uses bilinear interpolation. This interpolation algorithm typically performs interpolation on the RGB components of a 3x3 pixel block. While simple to implement, this method generally doesn't improve moiré pattern accuracy and image restoration.
[0029] Improving traditional methods can utilize color gradient information. In luminance-color space, since the luminance components of image edges and texture regions are much larger than the color difference components, the interpolation of the color difference components should be emphasized during interpolation. If transformed to the RGB color space, it is equivalent to emphasizing the interpolation information provided by color components other than the current interpolation component.
[0030] Specifically, taking the green component of a pixel with existing red component information (the "+" pixel in the image) as an example, the average of the information from the four pixels surrounding this pixel (top, bottom, left, and right) is typically used as the estimated value of the green component. The interpolation process does not utilize the existing red component pixel values and the information jointly provided by the surrounding pixels with existing red component information. In fact, the existing red component pixel values around the pixel to be interpolated provide a lot of useful information. Using the average of the surrounding pixels with existing red component information to estimate the red component value of the current pixel, if the estimated red component value is not significantly different from the existing red component value, it indicates that the current pixel is located in a relatively flat area in the image. Conversely, if the estimated red component value differs significantly from the existing red component value, then the current pixel is likely located in an edge or textured area in the image. In this case, a correction value can be added to the estimated green component of the current pixel to correct the estimation result. This correction term is provided by the red component information of the current pixel and its surrounding pixels.
[0031] Visible light cameras employ a weighted average method for automatic exposure of high frame rate image data. In automatic exposure, different weights are assigned based on the imaging scene. Specifically, in scenes with a dark background and a bright target, the weight of areas with higher grayscale is set higher than that of other areas to prevent overexposure of the target. In scenes with a bright background and a dark target, the weight of areas with lower grayscale is set higher than that of other areas, ensuring clear target imaging even at the cost of some background exposure. When the target is consistently located in the center of the field of view, the weight of the central area is set higher than that of other areas.
[0032] Visible light imaging devices use a weighted average method to automatically expose image data, assigning different weights to different imaging scenes, such as... Figure 4 As shown. Common scenarios include dark background with a bright target, bright background with a dark target, and a central target. In a dark background with a bright target, the weight of areas with higher grayscale is set higher to prevent the target from being overexposed. In a bright background with a dark target, the weight of areas with lower grayscale is set higher to ensure the target is clearly imaged even if some background is sacrificed. When the target is always in the center of the field of view, the weight of the central area is set higher. Automatic exposure and automatic gain are adjusted together to obtain the best visual image of the target.
[0033] This embodiment also provides a data acquisition method for observing high dynamic target images under limited bandwidth conditions, the method comprising: The visible light camera captures high frame rate image data of the target and transmits the high frame rate image data to the image compression device; The image compression device receives high frame rate image data, compresses the high frame rate image data to obtain low frame rate image data, restores the color of the high frame rate image data to obtain color-restored high frame rate image data, and transmits both the color-restored high frame rate image data and the low frame rate image data to the network switch. The network switch receives high-frame-rate image data and low-frame-rate image data after color restoration, transmits the low-frame-rate image data to the wireless telemetry terminal, and transmits the high-frame-rate image data after color restoration to the storage device. The wireless telemetry terminal receives low frame rate image data and wirelessly transmits the low frame rate image data to the ground. The storage device receives the color-restored high-frame-rate image data and stores the color-restored high-frame-rate image data.
[0034] This embodiment acquires high dynamic target image data under limited bandwidth conditions, and includes the following steps: Step 1: Select a suitable visible light camera to acquire high-quality images and extract target feature information based on the specific characteristics of the target.
[0035] Step 2: Use bilinear interpolation optimized by color gradient information to restore the color of real-time images.
[0036] Step 3: Assign different weights to different imaging scenarios and use the weighted average method to automatically expose the image data.
[0037] Step 4: The high frame rate data generated by the visible light camera is split into two Ethernet image data streams with high and low frame rates by the image compression device.
[0038] Step 5: The low frame rate Ethernet image data stream enters the wireless telemetry terminal through the telemetry system network switch, and is then transmitted to the ground via time-sharing wireless telemetry.
[0039] Step 6: The high frame rate and high fidelity Ethernet image data stream enters the storage device through the telemetry system network switch, and is used to restore the high dynamic separation moment afterward.
[0040] This embodiment also provides an electronic device, including: a memory for storing computer-readable instructions; and a processor for running the computer-readable instructions to execute a data acquisition method for observing high dynamic target images under limited bandwidth conditions.
[0041] This embodiment ensures both the ability to acquire and reconstruct targets under high dynamic range conditions and the requirement for real-time downlink under limited bandwidth transmission conditions, enabling the acquisition of target attitude throughout the flight mission.
[0042] This embodiment has the advantages of good real-time performance, strong fidelity in dynamic scenes, high smoothness, good versatility, and wide applicability.
[0043] This embodiment ensures real-time data acquisition while acquiring a set of high bitrate and high fidelity image data streams, which can achieve the following effects: (1) High dynamism: High bitrate image data can be used to analyze the real-time performance of high dynamic images and highly restore the details of the aircraft separation action; (2) High fidelity: Image data after color restoration and automatic exposure can be used to restore image distortion problems caused by overexposure or color mismatch due to engine flames and high-altitude lighting, so that the overall team and other professionals can grasp the separation process; (3) Real-time performance: Low frame rate image data of telemetry channel multiplexing can be wirelessly transmitted to the ground station in real time, and telemetry images of the moment of separation can be acquired in real time during flight tests; (4) Reliability of data acquisition: In cases where the memory cannot be recovered, the aircraft separation image can still be acquired by wireless telemetry.
[0044] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention by utilizing the methods and techniques disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.
Claims
1. A data acquisition system for observing high dynamic target images under limited bandwidth conditions, characterized in that... include: Visible light camera, image compression device, network switch, wireless telemetry terminal, and storage device; among which, The visible light camera acquires high frame rate image data of the target and transmits the high frame rate image data to the image compression device. The image compression device receives high frame rate image data, compresses the high frame rate image data to obtain low frame rate image data, restores the color of the high frame rate image data to obtain color-restored high frame rate image data, and transmits both the color-restored high frame rate image data and the low frame rate image data to the network switch. The network switch receives high-frame-rate image data and low-frame-rate image data after color restoration, transmits the low-frame-rate image data to the wireless telemetry terminal, and transmits the high-frame-rate image data after color restoration to the storage device. The wireless telemetry terminal receives low frame rate image data and wirelessly transmits the low frame rate image data to the ground. The storage device receives the color-restored high-frame-rate image data and stores the color-restored high-frame-rate image data.
2. The data acquisition system for observing high dynamic target images under limited bandwidth conditions according to claim 1, characterized in that: The number of visible light cameras is two.
3. The data acquisition system for observing high dynamic target images under limited bandwidth conditions according to claim 1 or 2, characterized in that: The number of photogenerated charges in the image of the target captured by the visible light camera is: ； ； in, Let λ be the number of photogenerated charges in the image of the target captured by the visible light camera, h be Planck's constant, c be the speed of light, and λ be the angle between the target and the normal of the visible light camera. For target illuminance, For visible light cameras, F-number For signal-to-noise ratio, This refers to the gain of a visible light camera.
4. The data acquisition system for observing high dynamic target images under limited bandwidth conditions according to claim 1 or 2, characterized in that: The signal-to-noise ratio of the visible light camera is 33.3dB, and the F-number of the surface visible light camera is 4.
2.
5. The data acquisition system for observing high dynamic target images under limited bandwidth conditions according to claim 1, characterized in that: High frame rate image data is 300 frames per second, and low frame rate image data is 24 frames per second.
6. The data acquisition system for observing high dynamic target images under limited bandwidth conditions according to claim 1, characterized in that: In color restoration, the red component value of the current pixel is estimated by using the average value of the existing red component pixels around the current pixel. If the estimated red component value differs from the existing red component value by less than 60, it means that the current pixel is in a flat area in the entire image. Conversely, if the estimated red component value differs from the existing red component value by more than 60, then the current pixel is likely to be in an edge or textured area in the entire image. The estimated result is corrected by adding a correction value to the estimated green component of the current pixel, and this correction term is provided by the red component information of the current pixel and its surrounding pixels.
7. The data acquisition system for observing high dynamic target images under limited bandwidth conditions according to claim 1, characterized in that: Visible light cameras use a weighted average method to automatically expose high frame rate image data.
8. The data acquisition system for observing high dynamic target images under limited bandwidth conditions according to claim 7, characterized in that: In automatic exposure, different weights are assigned to different imaging scenes. Specifically, in a scene with a dark background and a bright target, the weight of the area with higher grayscale is set to be greater than that of the other areas so that the target will not be overexposed. In a scene with a bright background and a dark target, the weight of the area with lower grayscale is set to be greater than that of the other areas so that the target can be clearly imaged even if some background is sacrificed. When the target is always located in the center of the field of view, the weight of the center area is set to be greater than that of the other areas.
9. A method for acquiring data from high dynamic target images under limited bandwidth conditions, characterized in that... include: The visible light camera captures high frame rate image data of the target and transmits the high frame rate image data to the image compression device; The image compression device receives high frame rate image data, compresses the high frame rate image data to obtain low frame rate image data, restores the color of the high frame rate image data to obtain color-restored high frame rate image data, and transmits both the color-restored high frame rate image data and the low frame rate image data to the network switch. The network switch receives high-frame-rate image data and low-frame-rate image data after color restoration, transmits the low-frame-rate image data to the wireless telemetry terminal, and transmits the high-frame-rate image data after color restoration to the storage device. The wireless telemetry terminal receives low frame rate image data and wirelessly transmits the low frame rate image data to the ground. The storage device receives the color-restored high-frame-rate image data and stores the color-restored high-frame-rate image data.
10. An electronic device, characterized in that, include: Memory: Used to store computer-readable instructions; and Processor: configured to execute the computer-readable instructions to perform the method as described in claim 9.