Outdoor optical motion capture image enhancement method and system based on FPGA

Abstract

The present application discloses an outdoor optical motion capture image enhancement method and system based on FPGA, which relates to the field of data display. The method includes obtaining a first frame image captured outdoors when the infrared light is turned off, presetting infrared light parameter one, infrared light parameter two, infrared light parameter three and target infrared light parameter, capturing second frame images one, two, three and second target frame images according to the preset infrared light parameters one, two, three and target infrared light parameters, presetting a second difference judgment threshold, performing a differential calculation between the second frame images one, two, three and second target frame images and the second difference judgment threshold, adjusting the pixel value of a first pre-processed difference image obtained by comparing and analyzing the first frame image and the second target frame image, and performing a nonlinear transformation process on the obtained foreground target image to obtain a foreground brightness enhanced image. The outdoor optical motion capture image enhancement method and system based on FPGA provided by the present invention improves the image capture quality in outdoor environments.

Description

Technical Field

[0001] The present application relates to the technical field of optical motion capture cameras, and in particular to an outdoor optical motion capture image enhancement method and system based on FPGA. Background Art

[0002] Optical motion capture camera technology has been widely used in recent years in the fields of motion capture, virtual reality, animation production, etc. Although optical motion capture technology has significant advantages in accuracy and detail capture, there are still many problems in its performance in outdoor environments.

[0003] In the prior art, there are data error problems in the image capture and processing process of outdoor optical motion capture images, and no verification step is added in the processing work, which further increases the probability of data processing errors, resulting in low efficiency of the entire image processing work and waste of resources. Summary of the invention

[0004] In order to overcome the above-mentioned deficiencies of the prior art, the present application provides an outdoor optical motion capture image enhancement method and system based on FPGA.

[0005] In a first aspect, the present application provides an outdoor optical motion capture image enhancement method based on FPGA, the method comprising:

[0006] Acquire a first frame image captured outdoors when the infrared light is turned off, preset infrared light parameter one, infrared light parameter two, infrared light parameter three and target infrared light parameter, and capture a second frame image one, a second frame image two, a second frame image three and a second target frame image according to the infrared light parameter one, infrared light parameter two, infrared light parameter three and target infrared light parameter;

[0007] Compare the difference image 1 and the difference image 2 obtained by respectively subtracting the first frame image from the second frame image 2 and the second frame image 3 with a preset first difference judgment threshold to obtain a preprocessing condition 1;

[0008] A second difference determination threshold is preset, and the difference image 1 and the difference image 2 are ratioed with the second difference determination threshold to obtain a preprocessing ratio coefficient 1;

[0009] Adjusting the pixel values ​​of a first preprocessed difference image obtained by comparing and analyzing the first frame image with the second target frame image to obtain a foreground target image;

[0010] If it is determined that the pixel value of one of the difference images, the difference image 1 and the difference image 2, is less than the first difference judgment threshold, the preprocessing condition 2 is output, and according to the preprocessing condition 2, a reserved difference image is extracted from the difference image 1 and the difference image 2, and a preprocessing ratio coefficient 2 is obtained by statistics according to the reserved difference image and the second difference judgment threshold;

[0011] According to the preprocessing ratio coefficient 2, a nonlinear transformation process for foreground brightness enhancement is performed on the foreground target image to obtain a foreground brightness enhanced image.

[0012] Preferably, a first frame of image captured outdoors when the infrared light is turned off is obtained;

[0013] Preset infrared light parameter 1, infrared light parameter 2, infrared light parameter 3 and target infrared light parameter, wherein the parameter values ​​of infrared light parameter 1, infrared light parameter 2, infrared light parameter 3 and target infrared light parameter are decreased in sequence;

[0014] According to the infrared light parameter 1, obtaining a second frame image 1 captured outdoors when the infrared light is turned on and in the infrared light parameter 1 state;

[0015] According to the infrared light parameter 2, obtaining a second frame image 2 captured outdoors when the infrared light is turned on and in the infrared light parameter 2 state;

[0016] According to the infrared light parameter three, obtaining a second frame image three captured outdoors when the infrared light is turned on and in the infrared light parameter three state;

[0017] According to the target infrared light parameters, a second target frame image captured outdoors when the infrared light is turned on and in the target infrared light parameter state is obtained.

[0018] Preferably, the first frame image and the second frame image are subtracted to obtain a difference image 1;

[0019] Subtract the first frame image from the second frame image to obtain a difference image 2;

[0020] A first difference determination threshold is preset, and if it is determined that the pixel values ​​of the difference image 1 and the difference image 2 are both greater than the first difference determination threshold, the preprocessing condition 1 is output.

[0021] Preferably, a second difference determination threshold is preset, and according to the preprocessing condition one, a first ratio coefficient is obtained by calculating a ratio between the difference image one and the second difference determination threshold;

[0022] Comparing the difference image 2 and the second difference determination threshold value to obtain a second ratio coefficient;

[0023] The first ratio coefficient and the second ratio coefficient are averaged to obtain a preprocessing ratio coefficient one.

[0024] Preferably, the first frame image and the second target frame image are subtracted to obtain a difference image to be adjusted;

[0025] According to the preprocessing ratio coefficient one, adjusting the pixel value error of the difference image to be adjusted to obtain a first preprocessing difference image;

[0026] All pixel values ​​in the first preprocessed difference image that are less than the second difference determination threshold are set to zero to obtain a foreground target image.

[0027] Preferably, if it is determined that the pixel value of one of the difference image 1 and the difference image 2 is less than the first difference determination threshold, the preprocessing condition 2 is output;

[0028] According to the second preprocessing condition, retaining a reserved difference image corresponding to the first difference image and the second difference image, the pixel value of which is greater than the first difference determination threshold;

[0029] Difference is calculated between the first frame image and the second frame image to obtain a difference image three;

[0030] Ratioing the reserved difference image and the second difference determination threshold to obtain a third ratio coefficient;

[0031] Ratioing the third difference image to the second difference determination threshold to obtain a fourth ratio coefficient;

[0032] The third ratio coefficient and the fourth ratio coefficient are averaged to obtain a preprocessing ratio coefficient two.

[0033] Preferably, according to the second preprocessing ratio coefficient, pixel value error adjustment is performed on the difference image to be adjusted to obtain a second preprocessing difference image;

[0034] Setting all pixel values ​​in the second preprocessed difference image that are less than a second difference determination threshold to zero to obtain a foreground target image;

[0035] The foreground target image is subjected to nonlinear transformation processing for foreground brightness enhancement to obtain a foreground brightness enhanced image.

[0036] The second aspect is the FPGA-based outdoor optical motion capture image enhancement system, including:

[0037] An image capture module is used to obtain a first frame image captured outdoors when the infrared light is turned off, preset infrared light parameter one, infrared light parameter two, infrared light parameter three and target infrared light parameter, and capture a second frame image one, a second frame image two, a second frame image three and a second target frame image according to the infrared light parameter one, the infrared light parameter two, the infrared light parameter three and the target infrared light parameter;

[0038] A condition one judgment module, used for comparing a difference image one and a difference image two obtained by respectively performing a difference operation on the first frame image from the second frame image two and the second frame image three with a preset first difference judgment threshold to obtain a preprocessing condition one;

[0039] A threshold processing module, used for presetting a second difference determination threshold, calculating the ratio of the difference image 1 and the difference image 2 to the second difference determination threshold to obtain a preprocessing ratio coefficient 1;

[0040] An image acquisition module, used for adjusting the pixel values ​​of a first pre-processed difference image obtained by comparing and analyzing the first frame image and the second target frame image to obtain a foreground target image;

[0041] The condition 2 judgment module is used for outputting the preprocessing condition 2 if it is judged that the pixel value of one of the difference images, the difference image 1 and the difference image 2, is less than the first difference judgment threshold value, and according to the preprocessing condition 2, extracting the reserved difference image from the difference image 1 and the difference image 2, and obtaining the preprocessing ratio coefficient 2 by statistics according to the reserved difference image and the second difference judgment threshold value;

[0042] The image processing module is used to perform nonlinear transformation processing of foreground brightness enhancement on the foreground target image according to the preprocessing ratio coefficient 2 to obtain a foreground brightness enhanced image.

[0043] Compared with the prior art, the present invention has the following characteristics and beneficial effects:

[0044] By capturing outdoor optical images with infrared lights turned off and on, and capturing multiple images when the infrared lights are turned on under preset parameters, it is possible to reduce the occurrence of various situations that affect the accuracy of image analysis due to blur in the captured images, and timely image replacement can be performed for such situations, that is, differential analysis under the two conditions is performed to connect the next image processing step and improve the efficiency of the entire image processing work. Finally, the pixel value error adjustment is performed on the foreground target image analyzed under the two conditions to improve the accuracy of the foreground target image acquisition. The obtained difference image is thresholded to ensure the accuracy and clarity of the captured target, so that the subsequent processing can be more focused on useful information. After the threshold processing is completed, a nonlinear transformation is applied to the processed image. This process aims to enhance the brightness of the foreground, making the foreground target more prominent in the image, while making the background darker and forming a sharp contrast. This nonlinear transformation can significantly improve the visibility of the target in a complex environment and ensure the capture effect of the target in a dynamic scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flowchart of the steps of the FPGA-based outdoor optical motion capture image enhancement method mainly embodied in this embodiment.

[0046] Figure 2 It is a structural block diagram of the FPGA-based outdoor optical motion capture image enhancement system mainly embodied in this embodiment.

[0047] Figure 3 This is a flow chart of infrared lamp control mainly embodied in this embodiment.

[0048] Figure 4 This is a system block diagram of data stream caching and processing mainly embodied in this embodiment. DETAILED DESCRIPTION

[0049] The present invention is further described in detail below with reference to the following examples.

[0050] Reference Figure 1 , an outdoor optical motion capture image enhancement method based on FPGA, the method comprises the following steps:

[0051] S1. Acquire the first frame image captured outdoors when the infrared light is off, preset infrared light parameter one, infrared light parameter two, infrared light parameter three and target infrared light parameters, and capture the second frame image one, the second frame image two, the second frame image three and the second target frame image according to the infrared light parameter one, the infrared light parameter two, the infrared light parameter three and the target infrared light parameters.

[0052] S2. Compare the difference image 1 and the difference image 2 obtained by subtracting the first frame image from the second frame image 2 and the second frame image 3 with the preset first difference judgment threshold to obtain the preprocessing condition 1.

[0053] S3. Preset a second difference determination threshold, calculate the ratio of difference image 1 and difference image 2 to the second difference determination threshold to obtain a preprocessing ratio coefficient 1.

[0054] S4. Adjust the pixel values ​​of the first preprocessed difference image obtained by comparing and analyzing the first frame image and the second target frame image to obtain a foreground target image.

[0055] S5. If it is determined that the pixel value of one of the difference images, difference image 1 and difference image 2, is less than the first difference judgment threshold, then preprocessing condition 2 is output. According to preprocessing condition 2, a reserved difference image is extracted from difference image 1 and difference image 2. According to the reserved difference image and the second difference judgment threshold, preprocessing ratio coefficient 2 is obtained by statistics.

[0056] S6. According to the preprocessing ratio coefficient 2, a nonlinear transformation process for foreground brightness enhancement is performed on the foreground target image to obtain a foreground brightness enhanced image.

[0057] Specifically, by capturing outdoor optical images with infrared lights turned off and on, and capturing multiple images when the infrared lights are turned on under preset parameters, the accuracy of image analysis can be reduced due to the blur of the captured images, and timely image replacement can be performed for such situations, that is, differential analysis under the two conditions is performed to connect the next image processing step and improve the efficiency of the entire image processing work. Finally, the pixel value error adjustment is performed on the foreground target image obtained by the analysis under the two conditions to improve the accuracy of the foreground target image acquisition. The obtained difference image is thresholded to ensure the accuracy and clarity of the captured target, so that the subsequent processing can be more focused on useful information. After the threshold processing is completed, a nonlinear transformation is applied to the processed image. This process aims to enhance the brightness of the foreground, making the foreground target more prominent in the image, while making the background darker to form a sharp contrast. This nonlinear transformation can significantly improve the visibility of the target in a complex environment and ensure the capture effect of the target in a dynamic scene.

[0058] The specific step S1 includes the following sub-steps:

[0059] Get the first frame of image captured outdoors with the infrared light turned off.

[0060] The infrared light parameter one, the infrared light parameter two, the infrared light parameter three and the target infrared light parameter are preset, and the parameter values ​​of the infrared light parameter one, the infrared light parameter two, the infrared light parameter three and the target infrared light parameter are decreased in sequence.

[0061] According to infrared light parameter one, a second frame image one captured outdoors when the infrared light is turned on and in the infrared light parameter one state is obtained.

[0062] According to infrared light parameter 2, a second frame image 2 captured outdoors when the infrared light is turned on and in the infrared light parameter 2 state is obtained.

[0063] According to the infrared light parameter three, a second frame image three captured outdoors when the infrared light is turned on and in the infrared light parameter three state is obtained.

[0064] According to the target infrared light parameters, a second target frame image captured outdoors when the infrared light is turned on and in the target infrared light parameter state is obtained.

[0065] Specifically, the devices used in the present invention for capturing images include: CMOS sensor, Mark ball sensor, wherein the CMOS sensor: a CMOS camera with high frame rate (400FPS) and high resolution (1936×1464) is selected for image capture. This configuration can not only capture the details of fast motion, but also ensure the clarity of image quality. The high frame rate enables the camera to effectively capture more picture information in dynamic scenes, while the high resolution ensures the retention of image details, making subsequent analysis and processing more accurate. In addition, this camera configuration can perform well under different lighting conditions and has strong adaptability. Mark ball sensor: Mark ball sensor is used to track special markers (marker balls) on target objects. These spheres usually reflect infrared light, and the reflected signal is captured by the camera to locate the position and movement trajectory of the target. Under the irradiation of infrared light, the light reflected by the Mark ball can pass through the 940nm filter to ensure its accurate capture in complex environments. This configuration improves the robustness and anti-interference ability of the system, so that the motion capture system can still accurately track the infrared light in the outdoor environment. The infrared light is a 940nm infrared light, and the FPGA is used to control the switch of the 940nm infrared light to provide precise lighting control. The 940nm filter is also used. The combination of the 940nm infrared light and the filter makes the system perform well in the outdoor environment. The advantage of the 940nm filter is that it can effectively filter most of the outdoor light, reduce environmental interference, and thus improve the image quality. Compared with the traditional 850nm filter, the 940nm filter performs better in practical applications and can produce less visible light. This feature significantly reduces the impact on the shooting environment. During the image capture process, the 940nm infrared light and the filter work together. The use of the filter ensures that only light with a wavelength of 940nm can pass through, further enhancing the significance of the foreground target in the image. Through this combination, the system can effectively suppress background noise, improve the signal-to-noise ratio of the image, make the foreground target more prominent, and ensure that high-quality image data can still be captured under complex lighting conditions. The implementation of this technology enables optical motion capture cameras to perform well in outdoor environments, significantly improving the accuracy and reliability of target capture, and adopts a dual-frame image capture strategy. First, the first frame of the image is captured, and the infrared light is turned off at this time to obtain background information in the natural environment. Then, the second frame of the image is captured with the infrared light turned on (in order to reduce the blurriness of the captured image, and the pixel values ​​of the image captured under a single infrared light parameter are likely to have errors, multiple frames of images are captured under different infrared light parameters to facilitate subsequent comparison and reduce errors). In this way, the first frame provides a reference background, while the second frame includes the foreground target under the infrared light. This capture strategy effectively reduces the interference of external light sources and lays a good foundation for subsequent image processing.Other algorithm implementations: Infrared light control: Infrared light control is achieved through the exposure signal provided by the motion capture system and the frame number corresponding to each exposure. Figure 3 As shown. The FPGA captures the edge change of the exposure signal and latches the sequence number of the current frame at that moment. Subsequently, the FPGA performs parity judgment on the latched frame number and generates a corresponding enable signal to control the switch of the infrared light. Specifically, when the frame number is odd, the infrared light is turned on; when the frame number is even, the infrared light is turned off. The entire logic consumes only two clock cycles: the first clock cycle is used to latch the frame number, and the second clock cycle is used to judge the parity and control the state switching of the infrared light. This control method ensures that in dynamic scenes, the infrared light can accurately respond to the exposure signal and cooperate with the image capture process to perform effective front and back frame processing.

[0066] The specific step S2 includes the following sub-steps:

[0067] The first frame image and the second frame image 2 are subtracted to obtain a difference image 1.

[0068] The first frame image and the second frame image are subtracted to obtain a difference image 2.

[0069] A first difference determination threshold is preset, and if it is determined that the pixel values ​​of the difference image 1 and the difference image 2 are both greater than the first difference determination threshold, the preprocessing condition 1 is output.

[0070] Specifically, for example, the second frame image 2 is subtracted from the first frame image to obtain a difference image. This process aims to eliminate the interference of background light through the difference algorithm to make the foreground target more obvious. Through this processing, the system can effectively identify and highlight the foreground target, suppress background noise, thereby improving image quality and providing more reliable data for subsequent image analysis. Preprocessing condition 1 (the first difference judgment threshold, that is, the critical value used to determine whether the captured image is available, such as whether the clarity of the image meets the standard, and if both the difference image 1 and the difference image 2 meet the standard, the difference image 1 and the difference image 2 are subjected to the next step of image processing). Other algorithm implementations: Frame image and difference processing: The image data stream output by the CMOS sensor directly enters the FPGA. Due to the large amount of image data, memory address space is required to cache data. In response to this demand, the VDMA IP in the traditional Xilinx VIVADO can move the video data stream to the memory address space, but lacks flexibility in register configuration, which increases the complexity of implementation. In order to simplify the design and improve control flexibility, a custom AXI4IP module is used to achieve efficient read and write access to the memory address space. Figure 4The system block diagram of data stream caching and processing is shown. In the figure, the video stream first passes through the AXI4 write module and enters the memory space through the FIFO cache buffer. With the help of the AXI interconnect IP, the system simplifies the 4KB boundary alignment problem and improves the smoothness of data transmission. After subsequent difference processing, threshold processing and nonlinear transformation, the processed image is sent back to the video output through the AXI4 read module. The entire process makes full use of the parallel processing capability of the FPGA, improves processing efficiency and reduces the consumption of BRAM resources. The algorithm implementation of data stream caching and processing includes even frame storage and odd frame processing. Even frame storage (infrared light off, background frame): When the frame number is detected to be even, the image data stream (video stream) output by the CMOS sensor will be written to the WR_FIFO buffer of the FPGA. When the data in the buffer reaches the length of a row of pixels, the AXI4 bus starts the write operation and writes the pixel data to the specified address space in the system memory. To ensure synchronization, whenever the frame synchronization signal (VSYNC) ends, the system triggers an 8-clock reset signal to clear the FIFO and prepare the data for the next frame. Odd frame processing (infrared light on, foreground frame): When the frame number is odd, the FPGA reads the even frame (background frame) data from the memory through the AXI4 interface and loads it into the RD_FIFO buffer. At the same time, the FPGA calculates the difference between the image data of the current odd frame and the even frame read from the memory. During this process, the AXI4 read operation accurately schedules the data according to the row and column information of the image to ensure the timing consistency during the reading and writing process.

[0071] The specific step S3 includes the following sub-steps:

[0072] A second difference determination threshold is preset, and according to preprocessing condition one, a first ratio coefficient is obtained by calculating a ratio between the difference image one and the second difference determination threshold.

[0073] The second difference image and the second difference determination threshold are compared to obtain a second ratio coefficient.

[0074] The first ratio coefficient and the second ratio coefficient are averaged to obtain a preprocessing ratio coefficient of one.

[0075] Specifically, for example, threshold processing is performed on the difference image, and a suitable threshold is set to distinguish the foreground and the background. The difference image 1 and the difference image 2 are respectively subjected to pixel value analysis processing with the second difference judgment threshold, and the results of the separate analysis are comprehensively analyzed to evaluate the pixel value error of the captured image. Specific difference calculation algorithm implementation: After the difference calculation is completed, the FPGA passes the difference image to the subsequent image processing module to perform threshold processing. The threshold processing sets the pixels in the difference image that are less than the set threshold to zero, thereby eliminating background noise.

[0076] The specific step S4 includes the following sub-steps:

[0077] The first frame image and the second target frame image are subtracted to obtain a difference image to be adjusted.

[0078] According to the preprocessing ratio coefficient one, pixel value error adjustment is performed on the difference image to be adjusted to obtain a first preprocessing difference image.

[0079] All pixel values ​​in the first preprocessed difference image that are less than the second difference determination threshold are set to zero to obtain a foreground target image.

[0080] Specifically, if the difference image to be adjusted is an image captured and processed under the specified conditions that need to be detected, the subsequent process is mainly to perform a series of image brightness enhancement processing on the difference image to be adjusted. Through the image pixel value error coefficient value evaluated above, the difference image to be adjusted is compensated for the slight difference pixel value, and all pixel values ​​in the first pre-processed difference image that are less than the set threshold are set to zero, thereby further eliminating the background noise and forming a clear foreground target image. This step ensures the accuracy and clarity of the captured target, so that subsequent processing can be more focused on useful information. Specific difference calculation algorithm implementation: After the difference calculation is completed, the FPGA passes the difference image to the subsequent image processing module to perform threshold processing. The threshold processing sets the pixels in the difference image that are less than the set threshold to zero, thereby eliminating background noise.

[0081] The specific step S5 includes the following sub-steps:

[0082] If it is determined that the pixel value of one of the difference images, the difference image 1 and the difference image 2, is less than the first difference determination threshold, the preprocessing condition 2 is output.

[0083] According to the second preprocessing condition, a reserved difference image corresponding to the pixel value of one of the difference image 1 and the difference image 2 is retained, which is greater than the first difference determination threshold.

[0084] The first frame image and the second frame image are subtracted to obtain a difference image three.

[0085] The reserved difference image and the second difference determination threshold are compared to obtain a third ratio coefficient.

[0086] The fourth ratio coefficient is obtained by comparing the difference image 3 and the second difference determination threshold.

[0087] The third ratio coefficient and the fourth ratio coefficient are averaged to obtain the preprocessing ratio coefficient two.

[0088] Specifically, as in preprocessing condition two (the second frame image one is used as a substitute image, if it is determined that the pixel value of the captured first frame image two is less than the first difference judgment threshold, the first frame image two is determined to be unusable, and then a series of image processing is performed on the second frame image one and the second frame image three, so as to reduce the time required to re-capture the image and repeat the processing steps, improve the efficiency of the image processing work, and avoid the interference of the error image on the subsequent image processing results), the difference image is threshold processed, and a suitable threshold is set to distinguish between the foreground and the background, and the reserved difference image and the difference image three are respectively subjected to pixel value analysis processing with the second difference judgment threshold, and the results of the separate analyses are comprehensively analyzed to evaluate the pixel value error of the captured image.

[0089] The specific step S6 includes the following sub-steps:

[0090] According to the preprocessing ratio coefficient 2, pixel value error adjustment is performed on the difference image to be adjusted to obtain a second preprocessing difference image.

[0091] All pixel values ​​in the second preprocessed difference image that are less than the second difference determination threshold are set to zero to obtain a foreground target image.

[0092] The foreground target image is subjected to nonlinear transformation processing for foreground brightness enhancement to obtain a foreground brightness enhanced image.

[0093] Specifically, after completing the threshold processing, a nonlinear transformation is applied to the processed image. This process aims to enhance the brightness of the foreground, making the foreground target more prominent in the image, while making the background darker and forming a sharp contrast. This nonlinear transformation can significantly improve the visibility of the target in a complex environment and ensure the capture effect of the target in a dynamic scene. The specific algorithm implementation of the nonlinear transformation processing steps: the nonlinear transformation further enhances the brightness of the foreground target, while darkening the background, enhancing the image contrast, and making the foreground target more prominent. The specific implementation of the nonlinear transformation adopts the ARM combined with the lookup table (LUT) method. Under the ZYNQ architecture, through the LUT lookup operation, the system can flexibly adjust the parameters of the nonlinear transformation, and it is convenient to modify the data of the lookup table in real time according to the needs, which improves the flexibility and processing efficiency of the system. It should be noted that all image processing algorithms are deployed on the XILINX FPGA, using the high parallel processing capability of the FPGA, and processing pixel data through 8 lanes in parallel. The pixel clock is 237.6M, ensuring the high-speed processing and real-time response of the system. The FPGA is responsible for controlling the switch of the infrared light, accurately controlling the image capture process of each frame, and using the double-frame difference algorithm to process the foreground and background in real time. At the same time, FPGA processes the positioning information of the Mark ball in real time to improve the overall performance and capture efficiency of the system. Specific FPGA parallel processing and system optimization algorithm implementation: To improve processing efficiency, FPGA processes 8 lane pixel data in parallel, and the bit width of each video stream is 64 bits. The advantage of parallel processing is that it can process multiple data channels at the same time, which greatly improves the response speed of the system. Through the customized AXI4 interface and FIFO mechanism, the system realizes efficient storage and reading of large-scale image data streams, ensuring the real-time capture and processing of double-frame images; compared with the traditional VDMA module, the customized AXI4IP has significant advantages in resource utilization and flexibility. Although the traditional VDMA module can realize image storage and reading functions when processing video streams, its resource consumption is large, especially the BRAM resource requirements are high. The customized AXI4 IP module only consumes 2 36k BRAMs to cache image data under the same configuration, while the VDMA module requires 4 36k BRAMs. In the AXI4 protocol, the problem of 4KB boundary alignment will be encountered when accessing memory. Through the AXI interconnect IP, the system simplifies the handling of this problem, making data transmission smoother while reducing design complexity.

[0094] The FPGA-based outdoor optical motion capture image enhancement system, by applying the above-mentioned FPGA-based outdoor optical motion capture image enhancement method, includes an image capture module, a condition one judgment module, a threshold processing module, an image acquisition module, a condition two judgment module and an image processing module, referring to Figure 2, obtain the first frame image captured outdoors when the infrared light is turned off through the image capture module, preset infrared light parameter one, infrared light parameter two, infrared light parameter three and target infrared light parameter, and capture the second frame image one, the second frame image two, the second frame image three and the second target frame image according to the infrared light parameter one, the infrared light parameter two, the infrared light parameter three and the target infrared light parameter; compare the difference image one and the difference image two obtained by subtracting the first frame image from the second frame image two and the second frame image three with the preset first difference judgment threshold through the condition one judgment module to obtain the preprocessing condition one; preset the second difference judgment threshold through the threshold processing module, and compare the difference image one and the difference image two with the second difference judgment threshold The ratio is synthesized to obtain a preprocessing ratio coefficient 1; the first preprocessing difference image obtained by comparing and analyzing the first frame image and the second target frame image is adjusted by the image acquisition module to obtain a foreground target image; if the condition 2 judgment module determines that the pixel value of one of the difference images, the difference image 1 and the difference image 2, is less than the first difference judgment threshold, the preprocessing condition 2 is output, and according to the preprocessing condition 2, a reserved difference image is extracted from the difference image 1 and the difference image 2, and the preprocessing ratio coefficient 2 is obtained by statistics according to the reserved difference image and the second difference judgment threshold; the image processing module performs a nonlinear transformation process for foreground brightness enhancement on the foreground target image according to the preprocessing ratio coefficient 2 to obtain a foreground brightness enhanced image.

[0095] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. An outdoor optical motion capture image enhancement method based on FPGA, characterized in that: The following steps are involved: Acquire a first frame image captured outdoors when the infrared light is turned off, preset infrared light parameter one, infrared light parameter two, infrared light parameter three and target infrared light parameter, and capture a second frame image one, a second frame image two, a second frame image three and a second target frame image according to the infrared light parameter one, infrared light parameter two, infrared light parameter three and target infrared light parameter; The first frame image is subtracted from the second frame image 2 and the second frame image 3 to obtain a difference image 1 and a difference image 2, and if it is determined that the pixel values ​​of the difference image 1 and the difference image 2 are both greater than a preset first difference judgment threshold, then a preprocessing condition 1 is output; A second difference determination threshold is preset, and according to the preprocessing condition one, the difference image one and the difference image two are ratioed with the second difference determination threshold to obtain a preprocessing ratio coefficient one; Adjusting the pixel values ​​of a first preprocessed difference image obtained by comparing and analyzing the first frame image with the second target frame image to obtain a foreground target image; If it is determined that the pixel value of one of the difference images, the difference image 1 and the difference image 2, is less than the first difference judgment threshold, the preprocessing condition 2 is output, and according to the preprocessing condition 2, a reserved difference image is extracted from the difference image 1 and the difference image 2, and a preprocessing ratio coefficient 2 is obtained by performing statistics based on the reserved difference image and the second difference judgment threshold; According to the preprocessing ratio coefficient 2, a nonlinear transformation process for foreground brightness enhancement is performed on the foreground target image to obtain a foreground brightness enhanced image.

2. The FPGA-based outdoor optical motion capture image enhancement method according to claim 1, characterized in that: The steps of obtaining a first frame image captured outdoors when the infrared light is turned off, presetting infrared light parameter one, infrared light parameter two, infrared light parameter three and target infrared light parameter, and capturing a second frame image one, a second frame image two, a second frame image three and a second target frame image according to the infrared light parameter one, infrared light parameter two, infrared light parameter three and target infrared light parameter are specifically as follows: Acquire the first frame of image captured outdoors when the infrared light is turned off; Preset infrared light parameter 1, infrared light parameter 2, infrared light parameter 3 and target infrared light parameter, wherein the parameter values ​​of infrared light parameter 1, infrared light parameter 2, infrared light parameter 3 and target infrared light parameter are decreased in sequence; According to the infrared light parameter 1, obtaining a second frame image 1 captured outdoors when the infrared light is turned on and in the infrared light parameter 1 state; According to the infrared light parameter 2, obtaining a second frame image 2 captured outdoors when the infrared light is turned on and in the infrared light parameter 2 state; According to the infrared light parameter three, obtaining a second frame image three captured outdoors when the infrared light is turned on and in the infrared light parameter three state; According to the target infrared light parameters, a second target frame image captured outdoors when the infrared light is turned on and in the target infrared light parameter state is obtained.

3. The FPGA-based outdoor optical motion capture image enhancement method according to claim 1, characterized in that: The step of presetting a second difference determination threshold and obtaining a preprocessing ratio coefficient 1 by calculating the ratio of the difference image 1 and the difference image 2 to the second difference determination threshold according to the preprocessing condition 1 is specifically as follows: Preset a second difference determination threshold, and calculate the ratio of the difference image 1 and the second difference determination threshold according to the preprocessing condition 1 to obtain a first ratio coefficient; Comparing the difference image 2 and the second difference determination threshold value to obtain a second ratio coefficient; The first ratio coefficient and the second ratio coefficient are averaged to obtain a preprocessing ratio coefficient one.

4. The FPGA-based outdoor optical motion capture image enhancement method according to claim 3, characterized in that: The step of adjusting the pixel values ​​of a first pre-processed difference image obtained by comparing and analyzing the first frame image with the second target frame image to obtain a foreground target image is specifically: Subtracting the first frame image from the second target frame image to obtain a difference image to be adjusted; According to the preprocessing ratio coefficient one, adjusting the pixel value error of the difference image to be adjusted to obtain a first preprocessing difference image; All pixel values ​​in the first preprocessed difference image that are less than the second difference determination threshold are set to zero to obtain a foreground target image.

5. The FPGA-based outdoor optical motion capture image enhancement method according to claim 4, characterized in that: If it is determined that the pixel value of one of the difference images, the difference image 1 and the difference image 2, is less than the first difference judgment threshold, the preprocessing condition 2 is output, and according to the preprocessing condition 2, a reserved difference image is extracted from the difference image 1 and the difference image 2, and the step of obtaining the preprocessing ratio coefficient 2 is performed by statistics based on the reserved difference image and the second difference judgment threshold, specifically, the step of: If it is determined that the pixel value of one of the difference images, the difference image 1 and the difference image 2, is less than the first difference determination threshold, then the preprocessing condition 2 is output; According to the second preprocessing condition, retaining a reserved difference image corresponding to the first difference image and the second difference image, the pixel value of which is greater than the first difference determination threshold; Difference is calculated between the first frame image and the second frame image to obtain a difference image three; Ratioing the reserved difference image and the second difference determination threshold to obtain a third ratio coefficient; Ratioing the third difference image to the second difference determination threshold to obtain a fourth ratio coefficient; The third ratio coefficient and the fourth ratio coefficient are averaged to obtain a preprocessing ratio coefficient two.

6. The FPGA-based outdoor optical motion capture image enhancement method according to claim 5, characterized in that: The step of performing nonlinear transformation processing for foreground brightness enhancement on the foreground target image according to the preprocessing ratio coefficient 2 to obtain a foreground brightness enhanced image is specifically as follows: According to the second preprocessing ratio coefficient, adjusting the pixel value error of the difference image to be adjusted to obtain a second preprocessing difference image; Setting all pixel values ​​in the second preprocessed difference image that are less than a second difference determination threshold to zero to obtain a foreground target image; The foreground target image is subjected to nonlinear transformation processing for foreground brightness enhancement to obtain a foreground brightness enhanced image.

7. FPGA-based outdoor optical motion capture image enhancement system, characterized in that: The system is used to implement the FPGA-based outdoor optical motion capture image enhancement method according to any one of claims 1 to 6, comprising: An image capture module is used to obtain a first frame image captured outdoors when the infrared light is turned off, preset infrared light parameter one, infrared light parameter two, infrared light parameter three and target infrared light parameter, and capture a second frame image one, a second frame image two, a second frame image three and a second target frame image according to the infrared light parameter one, the infrared light parameter two, the infrared light parameter three and the target infrared light parameter; The condition one judgment module is used to obtain a difference image one and a difference image two by respectively calculating the difference between the first frame image and the second frame image two and the second frame image three, and output the preprocessing condition one if it is determined that the pixel values ​​of the difference image one and the difference image two are both greater than a preset first difference judgment threshold; A threshold processing module, used for presetting a second difference determination threshold, and according to the preprocessing condition one, calculating the ratio of the difference image one and the difference image two to the second difference determination threshold to obtain a preprocessing ratio coefficient one; An image acquisition module, used for adjusting the pixel values ​​of a first pre-processed difference image obtained by comparing and analyzing the first frame image and the second target frame image to obtain a foreground target image; The condition two judgment module is used for outputting the preprocessing condition two if it is judged that the pixel value of one of the difference images, the difference image one and the difference image two, is less than the first difference judgment threshold value, and according to the preprocessing condition two, extracting the reserved difference image from the difference image one and the difference image two, and obtaining the preprocessing ratio coefficient two by statistics according to the reserved difference image and the second difference judgment threshold value; The image processing module is used to perform nonlinear transformation processing of foreground brightness enhancement on the foreground target image according to the preprocessing ratio coefficient 2 to obtain a foreground brightness enhanced image.

Images