Self-adaptive threshold radiance quantification method and device for infrared imaging simulation and mobile terminal

Abstract

The invention discloses a self-adaptive threshold radiance quantization method and device for infrared imaging simulation and a mobile terminal, and mainly solves the problems that in the prior art, the quantization effect is not ideal, and quantization real-time performance is difficult to consider, and the implementation scheme comprises the steps that quantization parameters are initialized; a frame of radiance value image is obtained, and the maximum radiance value of the frame of image is searched for and used for calculating and updating the quantization threshold value; carrying out normalization processing on the radiance value of the frame image based on a quantization threshold value, and mapping a normalization result to a gray value range; performing data type conversion on the mapping result to obtain a quantized gray value result as a quantized result of the frame of image; and acquiring a next frame of image, repeating the process until all image frames are processed, and obtaining a quantization result of all radiance value images. According to the method, a better radiance quantization effect can be obtained without manually adjusting the quantization threshold value, the processing performance is good, and the method is a quantization solution considering quantization quality and real-time performance and can be used for real-time simulation of infrared imaging.

Description

Technical Field

[0001] The present invention belongs to the technical field of infrared simulation, and particularly relates to an adaptive threshold radiance quantization method, device and mobile terminal, which can be used for real-time infrared imaging simulation. Background Technique

[0002] Infrared imaging simulation is an important part of infrared simulation technology, mainly used for infrared scene simulation and imaging device simulation in industrial inspection, scientific research, military and other fields. Its core purpose is to provide infrared imaging data close to actual physical phenomena by simulating infrared radiation characteristics to meet application requirements in different scenarios. The key links in the simulation process usually include: scene modeling, physical property rendering, imaging device simulation, that is, detection, sampling, quantization, etc., and image output and other steps.

[0003] According to the actual application scenario, infrared imaging simulation can be divided into two types: non-real-time simulation and real-time simulation.

[0004] Non-real-time simulation: mainly applied to the research, testing and verification stages, usually with relatively high requirements for simulation accuracy. Such simulation allows the use of more complex algorithms and processing flows, but its execution time is long and it is not suitable for scenarios that require dynamic response.

[0005] Real-time simulation: emphasizes the real-time nature of simulation processing and is widely used in scenarios such as device online testing and algorithm verification. This type of simulation needs to complete complex calculations in a short time to provide the response speed of the device under test close to the actual environment, and it has higher requirements for the efficiency of the algorithm and the hardware performance.

[0006] In real-time simulation, gray-scale quantization is one of the key steps. This process converts infrared radiation data into gray-scale values of images through a specific mapping method for subsequent display or further processing. However, there is a contradiction between real-time performance and quantization effect, which will directly affect the quality and application effect of the simulation.

[0007] The patent document with the application number CN202410337209.0 discloses "a method for simulating infrared remote sensing background images and establishing a database". First, it analyzes and calculates the heat exchange between the ground and the external environment and the infrared radiation of objects to model the ground temperature field. Secondly, it uses a pulse-coupled neural network to segment different material regions in the visible light remote sensing image and produce label data. Then, it calculates the radiation effect of each part according to the label data, quantifies the radiation brightness to replace the label, and outputs the simulated infrared image. Finally, it enhances the authenticity of the simulated infrared image through normalization and detail modulation processing to construct an infrared remote sensing background sample library. This method generates various infrared remote sensing scenarios based on visible light images. The obtained infrared simulation data has high details and authenticity, and can continuously increase the number of samples, effectively reducing the cost of obtaining such infrared data. However, since this method requires complex processing of data through a neural network based on known visible light images, the simulation conditions are relatively high and the processing delay is relatively large, so its application scenarios have great limitations and it is difficult to be used for real-time infrared simulation.

[0008] The patent document with the application number CN201610613691.1 discloses "an infrared imaging method based on digital domain TDI". In order to solve the problem that the conventional infrared imaging method cannot achieve flexible and continuous selection of integration levels and image acquisition in a large target range, it converges the target radiation signal onto a focal plane array infrared detector through an optical lens. The focal plane array infrared detector converts the radiation signal of the target into an analog voltage signal. This signal undergoes impedance matching and proportional amplification through an operational amplifier and outputs an analog signal that matches the input end of the A / D converter. This signal is digitally quantified by the A / D converter and then output to the FPGA circuit. The quantified image data is stored in the FPGA circuit in frames, completing the caching of multi-frame image data and the TDI superposition algorithm, and finally outputting the processed image data. Although this method can effectively increase the exposure time and improve the sensitivity of the system, and can detect and reconnoiter weak radiation targets. However, since it is mainly designed and optimized for actual infrared imaging devices, it cannot be directly used for real-time infrared imaging simulation.

[0009] Currently, there are mainly two common quantization methods directly applied to real-time infrared imaging simulation: the gray-scale quantization method of manually adjusting the threshold and the gray-scale quantization method of complete linear mapping. Among them:

[0010] The gray-scale quantization method of manually adjusting the threshold refers to setting the upper and lower thresholds of gray-scale quantization in advance through methods such as simulation experience judgment before infrared imaging simulation, and performing a linear mapping of the radiance values within the threshold to the gray-scale domain, so as to obtain a high-quality quantization result that can reflect most of the information of the radiance image. The quantization effect of this method mainly depends on the experience judgment of the simulation personnel, and its reliability and rationality are relatively poor.

[0011] The grayscale quantization method of a complete linear mapping refers to directly mapping the input radiance value to the closest grayscale value. If the input amplitude exceeds or far exceeds the maximum grayscale value, it will be mapped to the maximum grayscale value. This operation can only achieve a good quantization effect when the input radiation intensity range is very close to the grayscale value range. Otherwise, a large amount of information will inevitably be lost. Summary of the Invention

[0012] The purpose of the present invention is to propose an adaptive threshold radiance quantization method, device, and mobile terminal for infrared imaging simulation to avoid relying on the experience of simulation personnel and improve the real-time simulation quantization effect of infrared imaging through automatic threshold comparison and dynamic update during the simulation process.

[0013] To achieve the above object, the technical solutions of the adaptive threshold radiance quantization method, device, mobile terminal, and readable storage medium for infrared imaging simulation are realized as follows:

[0014] 1. An adaptive threshold radiance quantization method for infrared imaging simulation, characterized by comprising:

[0015] 1) Initialize the quantization accuracy γ, overexposure prevention coefficient α, and quantization threshold ρ;

[0016] 2) Obtain the radiance value image represented by single-precision floating-point numbers of the nth frame, and search for the maximum radiance value of this frame of image n ∈ [1, m], where m is the total number of image frames to be processed in this simulation;

[0017] 3) Normalize the radiance values of the nth frame radiance value image based on the quantization threshold ρ to obtain a normalization result And use this result and the quantization accuracy γ to calculate the result mapped to the grayscale value range

[0018] 4) For Perform data type conversion to obtain the grayscale quantization result in the form of an unsigned integer type As the quantization result of this frame of radiance value image;

[0019] 5) Update the quantization threshold ρ according to the maximum radiance value searched for in this frame of image And the overexposure prevention coefficient α, let n = n + 1, and repeat steps 2)-4) to obtain the quantization result of the images obtained this time;

[0020] 6) Repeat steps 2)-5) until all image frames are processed to obtain the quantization results of all radiance value images.

[0021] Further, in step 2), search for the maximum radiance value of the nth frame image Its implementation includes: defining the variables required for the search and their representation methods, and setting the temporary value of the nth frame radiance image as Compare the radiance values and the temporary value to determine their magnitude relationship, and based on the comparison result, judge whether to update the temporary value; then traverse all the radiance values in the image, perform the above operations, and let So as to obtain the maximum radiance value of the nth frame image

[0022] Further, in step 3), normalize the radiance value of the nth radiance value image based on the quantization threshold ρ, and the formula is as follows:

[0023]

[0024] Among them, is the normalization result of the radiance value , and its value range is [0, 1]; is the median radiance value of the radiance value image, M i is the mantissa part of i is the exponent part of ρ ρ is the quantization threshold, M ρ is the mantissa part of ρ, and its range is [0, 1), E

[0025] Further, in step 3), calculate the result of mapping the normalization result to the gray value range, and the formula is as follows:

[0026]

[0027] Among them γ is the quantization precision, which is set during parameter initialization.

[0028] Further, in step 4), perform data type conversion on the result in the gray value range to obtain the unsigned integer form of and its formula is:

[0029]

[0030] Among them is the intermediate variable required for data type conversion, and its data type is integer, and the value range is [0, γ].

[0031] Further, the maximum radiance value searched according to the frame image in step 5) and the overexposure prevention coefficient α are used to update the quantization threshold ρ, and the formula is:

[0032]

[0033] wherein, the overexposure prevention coefficient α is set during parameter initialization.

[0034] 2. An adaptive threshold radiance quantization device for infrared imaging simulation, implemented by FPGA, which is characterized by including:

[0035] An image data receiving module, configured to receive an externally input radiance image stream, and complete functions of signal type conversion, data transmission protocol parsing, frame start signal generation, and data bit-width conversion, obtain radiance value data and output it;

[0036] A quantization threshold calculation module, configured to receive the radiance value data and the image frame start signal from the image data receiving module, and perform radiance value comparison based on a floating-point operation core. Until the image frame start signal is high, update the maximum radiance value, and then calculate and update the quantization threshold from the maximum radiance value, and output it to the normalization calculation module;

[0037] A normalization calculation module, configured to receive the radiance value data from the image data receiving module and the quantization threshold from the quantization threshold calculation module, and process the two based on a floating-point divider to implement normalization processing of the input radiance value;

[0038] A quantization mapping module, configured to map the normalization processing result obtained by the normalization calculation module to the gray value range based on a floating-point multiplier, obtain a quantization mapping result, and transmit it to the data type conversion module;

[0039] A data type conversion module, configured to convert the data type of the quantization mapping result to an unsigned integer type based on a floating-point operation core to obtain a converted quantization mapping result;

[0040] A quantization result output module, configured to perform data caching, UDP data packet generation, and Ethernet sending operations on the converted quantization mapping result output by the data type conversion module in sequence.

[0041] 3. A mobile terminal, including a processing chip, a memory, an input / output interface, and a program file stored in the memory and loadable to run on the processing chip, which is characterized in that:

[0042] The processing chip adopts an FPGA chip or a ZYNQ series heterogeneous chip, and is configured to run the program file to execute any step in the above infrared imaging simulation adaptive threshold radiance quantization method and any module function in the device;

[0043] The memory is a non-volatile readable memory for storing the program file;

[0044] The input / output interface includes an optical fiber interface and a gigabit Ethernet interface for inputting and outputting images or videos;

[0045] The program file is stored in the memory and can be loaded into the processing chip for running, and is used to implement any step in the above infrared imaging simulation adaptive threshold radiance quantization method and any module function in the device.

[0046] 4. A non-volatile readable storage medium, characterized in that it stores program instructions, and the program instructions are run to execute any step in the above infrared imaging simulation adaptive threshold radiance quantization method and any module function in the device. Description of the Drawings

[0047] Figure 1 is the overall flowchart for implementing the adaptive threshold radiance quantization method of infrared imaging simulation of the present invention;

[0048] Figure 2 is the sub-flowchart for implementing the maximum radiance value search in the method of the present invention;

[0049] Figure 3 is the block diagram of the modules of the adaptive threshold radiance quantization device for infrared imaging simulation of the present invention;

[0050] Figure 4 The sub-block diagram of the image data receiving module in the device of the present invention;

[0051] Figure 5 is the block diagram of the mobile terminal of the present invention. Detailed Embodiments

[0052] The following describes the embodiments of the present invention in detail with reference to the drawings.

[0053] Embodiment 1: Adaptive Threshold Radiance Quantization Method for Infrared Imaging Simulation

[0054] Refer to Figure 1 , the implementation steps of this example are as follows:

[0055] Step 1, initialize parameters

[0056] The required initialization parameters include the quantization accuracy γ, the overexposure prevention coefficient α, and the quantization threshold ρ. The specific implementation includes:

[0057] 1.1) Initialize the quantization accuracy γ:

[0058] The quantization precision γ represents the precision level of the grayscale value result of this quantization output and is used for subsequent radiance value normalization operations. The unit of this parameter is bit, and the common values are 8bit, 10bit, 14bit, and 16bit. The larger the value of the quantization precision, the more accurate the quantization result.

[0059] In this example, γ supports the quantization precision range of 8bit - 16bit. The initialization operation of the quantization precision γ is to select a specific precision value within this range according to the actual application requirements. This parameter value is a fixed value in one simulation.

[0060] 1.2) Initialize the overexposure prevention coefficient α:

[0061] The overexposure prevention coefficient α is an empirical coefficient introduced to avoid overexposure or distortion of the quantized image. This coefficient will be specifically used to calculate and update the quantization threshold operation. The recommended value range of this coefficient is from 0.8 to 0.9. If the value is lower than the recommended value range, it will cause excessive loss of image information after quantization and poor quantization effect; if the value exceeds the recommended value range, it may cause overexposure of the quantized image and also cannot obtain a good quantization effect.

[0062] This value is a fixed value in one simulation.

[0063] 1.3) Initialize the quantization threshold ρ:

[0064] The quantization threshold ρ is a key parameter for radiance value normalization operations. It is a dynamically changing parameter value. During the subsequent quantization process, this threshold will be continuously updated and iterated according to the real-time input image information, so as to ensure that it can conform to the characteristics of the dynamically changing input scene image and guarantee the quantization quality.

[0065] In this step, the initial value of this parameter is set to 1.

[0066] Step 2: Obtain the radiance value image represented by single-precision floating-point numbers for the nth frame, and search for the maximum radiance value of this frame of image

[0067] Refer to Figure 2 , the implementation of this step includes the following operations:

[0068] 2.1) Obtain the radiance value image represented by single-precision floating-point numbers for the nth frame:

[0069] Read the radiance value image of the nth frame from the memory and parse the radiance value represented by single-precision floating-point numbers for subsequent maximum radiance value search operations.

[0070] 2.2) Before performing the search, first perform relevant variable definitions, that is, define the required variables and their representation methods:

[0071] 2.2.1) Definition of Radiance Value:

[0072] The radiance value of an infrared image is an important physical quantity characterizing the intensity of infrared radiation. It is used to describe the radiation power density passing through a unit projected area within a unit solid angle per unit time, and its unit is usually W / m 2 / sr. The expression of its physical definition is:

[0073]

[0074] where x represents the radiance value, P represents the radiation energy, A represents the area of the emitting or receiving surface, Ω represents the solid angle, and θ represents the direction angle from which the radiation comes;

[0075] Due to the large dynamic range of the infrared radiance value, single-precision floating-point numbers are usually used for numerical representation in simulations to effectively balance the numerical range and representation accuracy.

[0076] According to the standard format of single-precision floating-point numbers: Value = (-1) S ·(1 + M)·2 E-127 , the radiance value is defined as

[0077]

[0078] where Value is the variable name of the single-precision floating-point number; S is the sign bit; M is the mantissa part, with a range of [0, 1); E is the exponent part, with an offset value of 127; n is the frame number of the image; i is the sequence number of the radiance value in the nth-frame radiance image, i ∈ [1, a×b], where a and b are the length and width of the radiance value image respectively; S i is 's sign bit; M i is 's mantissa part; E i is 's exponent part.

[0079] 2.2.2) Definition of Temporary Value

[0080] The temporary value is the intermediate value required to execute this step and is used to temporarily store the calculation intermediate results in the maximum radiance value search operation. Its initial value is 0;

[0081] According to the standard format Value of single-precision floating-point numbers, the temporary value of the nth-frame radiance image is defined as

[0082]

[0083] where S temp is The sign bit determines its positive or negative sign; M temp is the mantissa part, with a range of [0, 1), E temp is the exponent part, with an exponent offset value of 127.

[0084] 2.2.3) Definition of the maximum radiance value:

[0085] According to the standard format Value of single-precision floating-point numbers, the maximum radiance value, which is the maximum radiance value in a frame of radiance image, is defined as

[0086]

[0087] where S max is the sign bit, M max is the mantissa part, with a range of [0, 1), E max is the exponent part, with an exponent offset value of 127.

[0088] 2.3) Compare the E i described in 2.2.1) with the E temp described in 2.2.2) to obtain the size relationship between the radiance value and the temporary value :

[0089] If E i > E temp , it can be obtained that then execute step 2.5);

[0090] If E i < E temp , it can be obtained that then execute step 2.5);

[0091] If E i = E max , then the size relationship between and cannot be directly determined, and step 2.4) is executed.

[0092] 2.4) Compare the M i described in 2.2.1) with the M temp described in 2.2.2):

[0093] If M i > M temp , it can be obtained that

[0094] If Mi = M temp , it can be obtained that

[0095] If M i < M temp , it can be obtained that

[0096] 2.5) According to the comparison results obtained in steps 2.3) and 2.4), determine whether to update the temporary value: and to determine whether to update the temporary value:

[0097] If then update the temporary value to

[0098] Otherwise, do not update the temporary value;

[0099] At this time, the obtained temporary value is the maximum radiance value among the first i radiance values of the image.

[0100] 2.6) Traverse all radiance values in the image:

[0101] If the serial number i of the current radiance value < a × b, then let i = i + 1, and return to step 2.3);

[0102] If the serial number i of the current radiance value ≥ a × b, it means that all radiance values in the nth frame image have been traversed. At this time, the temporary value is the maximum radiance value of the nth frame image, then let Complete the operation of the maximum radiance value of the nth frame image operation.

[0103] Step 3, update the quantization threshold.

[0104] The quantization threshold update introduces an improvement operation on the basis of the existing gray quantization method with manual adjustment of the threshold. Its specific implementation is that after the quantization processing of the current frame is completed, according to the maximum radiance value searched in the current frame image and the overexposure prevention coefficient α, calculate and update the quantization threshold to obtain the current quantization threshold ρ′. The implementation formula is as follows:

[0105]

[0106] Step 4, normalize the radiance values of the nth frame radiance value image based on the quantization threshold to obtain the normalization result and use this result and the quantization accuracy γ to calculate the result of mapping it to the gray value range

[0107] 4.1) Based on the value of n, use the following normalization formula to normalize the radiance value of the n-th radiance value image according to the quantization threshold for normalization:

[0108] When n = 1, use the initial quantization threshold ρ for normalization, and the formula is as follows:

[0109]

[0110] When n > 1, use the current quantization threshold ρ′ for normalization, and the formula is as follows:

[0111]

[0112] Among them, is the normalization result of the radiance value , and the value range is [0, 1]; n is the image frame number; is the radiance value in the radiance value image, and M is the mantissa part of i and E is the exponent part of ρ ; ρ is the initial quantization threshold, and M ρ is the mantissa part of ρ, with a range of [0, 1), and E ρ′ is the exponent part of ρ; ρ′ is the current quantization threshold, and M ρ′ is the mantissa part of ρ′, with a range of [0, 1), and E

[0113] By normalizing the input radiance value, this example can be applied to meet the quantization requirements of different scenarios and different bands for infrared real-time simulation of radiance value images with different ranges.

[0114] 4.2) Calculate the result of mapping the normalization result to the gray value range

[0115] The quantization output result of this example needs to be a gray value with a specified quantization accuracy, which is specifically reflected in that the data size is within the specified gray value range and the data type is in the form of an unsigned integer;

[0116] Since the normalization result obtained in 4.1) still does not meet the above requirements, to meet this requirement, the normalization result needs to be further mapped to the gray value range and then the data type is converted to the unsigned integer form;

[0117] This operation will complete the gray value range calculation based on the quantization accuracy and implement the mapping to the gray value range;

[0118] 4.2.1) Calculate the grayscale value range based on the quantization precision:

[0119] Take the quantization precision γ as the precision level of the grayscale value result of this quantization output, and there is the following fixed relationship between it and the grayscale value range R:

[0120] R = [0, Max gray = [0, (2 γ - 1)]

[0121] where R is the grayscale value range, and Max gray is the upper limit of the grayscale value;

[0122] For example, when the quantization precision is 8 bits, the grayscale value range is [0, 255]; when the quantization precision is 16 bits, the grayscale value range is [0, 65535].

[0123] 4.2.2) Linearly map the normalization result to the grayscale value range according to the following formula:

[0124]

[0125] where is the result mapped to the grayscale value range, and the data type is single-precision floating-point number, Max_gray is the upper limit of the grayscale value, and Max gray = (2 γ - 1);

[0126] In the existing grayscale quantization method of complete linear mapping, it does not include the normalization operation or similar operations described in the steps of this example, but directly maps the radiance value to the grayscale value range according to the following formula:

[0127]

[0128] This forced mapping operation will cause relatively large precision loss and information loss;

[0129] For example, when using the existing grayscale quantization method of complete linear mapping to quantize a set of radiance values represented by floating-point numbers {0.1, 10.11, 36.813, 316.43, 1000.13, 2137.81} with a quantization precision of 8 bits and a grayscale value range of 0 to 255, the corresponding mapping result is {0, 10, 37, 255, 255, 255};

[0130] As can be seen from the above examples, when the original size of the radiance value is within the grayscale value range, this method can obtain a good quantization effect. For example, mapping 0.1 to 0 and 10.11 to 10. When the original size of the radiance exceeds the grayscale value range, this method will directly map it to 255 by force, and this mapping result will completely fail to reflect the characteristics of the original radiance value. For example, two radiance values with a large difference in the original values of 316.43 and 1000.13 are both mapped to 255.

[0131] It can be seen that since the grayscale quantization method of complete linear mapping does not adopt the normalization operation in the present invention, when mapping radiance values with a large dynamic range, it cannot retain the distribution relationship of the original radiance values and there is a large amount of information loss. In actual infrared imaging simulation, the radiance values in the simulation image usually have a large dynamic range. Therefore, the method containing the normalization operation in the present invention has great advantages compared with the grayscale quantization method of complete linear mapping.

[0132] Step 5, perform data type conversion to obtain the grayscale quantization result in the form of unsigned integer as the quantization result of the nth frame radiance value image.

[0133] In step 4, the processing result with a value range within the specified grayscale value range has been obtained However, its data type is still single-precision floating-point number, which does not meet the output result requirements. Therefore, it is necessary to convert the data type to unsigned integer form through this step to obtain the quantization result that meets the requirements. The implementation is as follows:

[0134] 5.1) Use to calculate the intermediate variable m:

[0135]

[0136] where m is the intermediate variable required for data type conversion, and its data type is integer;

[0137] 5.2) Calculate the grayscale quantization result of the nth frame image

[0138]

[0139] Step 6, obtain the quantization results of all images.

[0140] The object processed in this example is an image sequence composed of several frames of radiance images. When executing this step, the quantization process of the nth frame radiance image has been completed and the quantization threshold has been updated. Therefore, at this time, it is also necessary to obtain the subsequent (n + 1)th frame image and perform quantization operations on it. The implementation is as follows:

[0141] 6.1) Perform Steps 2 to 5 on the (n + 1)-th frame image to obtain the quantization result of the (n + 1)-th frame image;

[0142] 6.2) Continuously repeat Step 6.1) until all image frames are processed to obtain the quantization results of all radiance value images.

[0143] Embodiment 2: An adaptive threshold radiance quantization device for infrared imaging simulation.

[0144] Refer to Figure 3 , the device of this embodiment includes an image data receiving module 1, a quantization threshold calculation module 2, a normalization calculation module 3, a quantization mapping module 4, a data type conversion module 5, and a quantization result output module 6. Among them, the image data receiving module 1 includes an optical module control sub-module 11, a protocol parsing sub-module 12, and a bit width conversion sub-module 13, as Figure 4 shown.

[0145] The optical module control sub-module 11 uses the GTX transceiver inside the FPGA chip as the core component to receive the high-speed image data stream input from the optical fiber, and completes data decoding, clock recovery, and preliminary data verification to ensure data integrity and stability.

[0146] The protocol parsing sub-module 12 uses the Aurora 64b / 66b IP core as the core component to decode the serial data into a 64-bit parallel data stream, and extracts the data after a specific valid data identifier as the valid image data output.

[0147] The bit width conversion sub-module 13 consists of a memory read / write logic and a memory. It realizes data bit width conversion through the memory caching method. The memory read / write logic is implemented through the VDMA IP core, and the memory is a DDR memory.

[0148] The quantization threshold calculation module 2 uses a floating-point arithmetic core and a floating-point multiplier as the core components. Among them, the floating-point arithmetic core is used for maximum radiance value search, and the floating-point multiplier is used for quantization threshold calculation based on the maximum radiance value.

[0149] The normalization calculation module 3 consists of a floating-point divider and a register cache, and is used to complete the normalization calculation operation.

[0150] The quantization mapping module 4 uses a look-up table and a floating-point divider as the core components. The look-up table stores the correspondence between the quantization precision and the upper limit value of the gray value. The corresponding upper limit value of the gray value can be obtained according to the set specific precision and then used in the mapping operation. This quantization mapping operation is implemented through the floating-point divider.

[0151] The data type conversion module 5 uses a floating-point operation core as its core component. This operation core is configured in the floating-point to fixed-point mode and is used to implement data type conversion.

[0152] The quantization result output module 6 consists of a data cache, a data packet generation state machine, and an Ethernet transmission state machine, and is used to output quantization results based on the Ethernet protocol. The data cache can be implemented by on-chip memories such as FIFO and RAM.

[0153] The working principle of the entire device is as follows:

[0154] The optical module control sub-module 11 receives the optical fiber input data and transmits it to the protocol parsing sub-module 12 for transmission protocol parsing, obtaining a 64-bit wide radiance image data stream and a frame start signal indicating the frame header position. Then, the 64-bit wide radiance image data stream is converted to a 32-bit wide radiance image data stream through the bit width conversion sub-module 13 and output to the quantization threshold calculation module 2.

[0155] The quantization threshold calculation module 2 receives the radiance value data from the bit width conversion sub-module 13 and searches for the maximum radiance value. The obtained maximum radiance value is further used for quantization threshold calculation. When the quantization threshold calculation module 2 detects the image frame start signal generated by the protocol parsing sub-module 12, it outputs the threshold calculation result to the normalization calculation module 3.

[0156] The normalization calculation module 3 caches the radiance value data received from the bit width conversion sub-module 13 and the quantization threshold received from the quantization threshold calculation module 2 in registers, and then transmits them to the input port of the floating-point divider inside the module. The output result of the floating-point divider is the normalization processing result of the input radiance value, and this result will be transmitted to the quantization mapping module 4.

[0157] After receiving the normalization processing result from the normalization calculation module 3, the quantization mapping module 4 maps it to the gray value range corresponding to the specified quantization accuracy based on the floating-point multiplier, obtaining a mapping result in floating-point form. This floating-point form mapping result will be further converted to an unsigned integer type through the data type conversion module 5 to obtain a quantization gray value that meets the quantization output requirements, which is used as the quantization output result. This quantization output result will be transmitted to the quantization result output module 6 for caching until the cached data volume meets the data packet generation condition, at which time the cached data is read, and a UDP data packet is generated based on the state machine, encapsulated according to the Ethernet protocol standard, and sent through the network port to achieve real-time and efficient output of the quantization result.

[0158] Embodiment 3: A mobile terminal.

[0159] Refer to Figure 5, the mobile terminal provided in this embodiment includes a processing chip, a memory, an input / output interface, and a program file stored in the memory and loadable into the processing chip for running.

[0160] The processing chip can be implemented in ways such as a CPU, a microprocessor, an application-specific chip, an FPGA chip, or a ZYNQ chip, and is used to execute relevant programs to implement the technical solution provided in this embodiment.

[0161] The memory can be implemented using specific memory types such as ROM, RAM, DDR, QSPI, etc. Software programs, driver programs, or firmware programs can be stored in the memory to implement the technical solution provided in this embodiment.

[0162] The input / output interface can use interfaces such as an optical fiber interface, an Ethernet interface, etc. as the input / output interface of the mobile terminal described in this embodiment, and is used to transmit information such as images, videos, and parameters required for program operation;

[0163] The program file is stored in the memory and loadable into the processing chip for running, and is used to implement the technical solution provided in this embodiment.

[0164] It should be noted that although the above device only shows a processing chip, a memory, an input / output interface, and a program file, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device may also only include the components necessary for implementing the method of this embodiment, and do not necessarily include Figure 5 all the components shown.

[0165] Embodiment 4: Non-volatile readable storage medium.

[0166] The non-volatile readable storage medium is a storage device that can retain data after power-off and allows users to read data multiple times, such as a solid-state drive, a mechanical hard drive, an optical disc, a flash memory, a read-only memory, an electrically erasable programmable read-only memory, a FLASH chip, an SD card, etc. Program instructions are stored in the medium, and the program instructions can exist in forms such as binary, assembly language, hardware description language, or high-level programming language, and are run to execute any step in the above infrared imaging simulation adaptive threshold radiance quantization method and any module function in the device.

[0167] The above description is only several specific examples of the present invention and does not constitute any limitation to the present invention. Obviously, for professionals in the field, after understanding the content and principle of the present invention, various modifications and changes in form and details may be made without departing from the principle and structure of the present invention. However, these corrections and changes based on the idea of the present invention are still within the scope of protection of the claims of the present invention.

[0168] It should be noted that the step numbers in the specification and claims of the present invention are only for clearly describing the implementation embodiments of the present invention for easy understanding, and their sequence numbers are not limited.

Claims

1. An adaptive threshold radiance quantification method for infrared imaging simulation, characterized in that: including: 1) Initialize the quantization precision γ, overexposure prevention coefficient α, and quantization threshold ρ; 2) Get the radiance value image represented by single-precision floating-point numbers for the nth frame, and search for the maximum radiance value of the frame image n∈[1,m], m is the total number of image frames that need to be processed in this simulation; 3) Based on the quantization threshold ρ, the radiance value of the n-th frame radiance value image is normalized to obtain the normalized result And use this result and the quantization accuracy γ to calculate the result of mapping it to the gray value range 4) Yes Perform data type conversion to obtain the grayscale quantization result in unsigned integer form As the quantization result of the radiance value image of the frame; 5) The maximum radiance value found based on the frame image Update the quantization threshold ρ with the anti-overexposure coefficient α, set n=n+1, repeat steps 2)-4), and obtain the quantization result of the image acquired this time; 6) Repeat steps 2) - 5) until all image frames are processed to obtain the quantization result of the entire radiance value image.

2. The method according to claim 1, characterized in that: In step 2), search for the maximum radiance value of the nth frame image It includes: 2a) Define the required variables and their representation methods: 2a1) Assume that the resolution of the radiance image is a×b, the image frame number is n, and the radiance value of the nth frame of the radiance image is defined as Among them, M i yes The mantissa of is in the range [0,1); E i yes The exponent part of , the exponent offset value is 127; i is the radiance value sequence number; 2a2) Let the temporary storage value of the radiance image of the nth frame be Among them, M temp yes The mantissa of is in the range [0,1); E temp yes The exponent part of the index is 127. The initial value of is 0; 2a3) Assume the current maximum radiance value is Among them, M max yes The mantissa of is in the range [0,1); E max yes The exponent part of , the exponent offset value is 127; 2b) E described in 2a1) i E described in 2a2) temp Compare and obtain the radiance value and the temporary value Size relationship: If E i >E temp , can be obtained Then execute step 2d); If E i <E temp , can be obtained Then execute step 2d); If E i =E max , then execute step 2c); 2c) For the M described in 2a1) i and M described in 2a2) temp For comparison: If M i >M temp , can be obtained If M i =M temp , can be obtained If M i <M temp , can be obtained 2d) obtained according to step 2b) and step 2c) and Compare the results to determine whether to update the temporary value: like The temporary value Updated to Otherwise, do not update the temporary value; 2e) Traverse all radiance values in the image: If the current radiance value sequence number i < a × b, then set i = i + 1 and return to step 2b); If the current radiance value number i ≥ a×b, then let Complete the maximum radiance value of the nth frame image operate.

3. The method according to claim 1, characterized in that: In step 3), the radiance value of the nth radiance value image is calculated based on the quantization threshold ρ. Normalize it, the formula is as follows: in, is the radiance value The normalized result of is in the range of [0, 1]; is the median radiance value of the radiance value image, M i for The mantissa part, E i for The exponential part; ρ is the quantization threshold, M ρ is the mantissa of ρ, ranging from [0,1), E ρ is the exponential part of ρ.

4. The method according to claim 1, characterized in that: Calculate the normalized result in step 3) The result of mapping to the grayscale value range The formula is as follows: in γ is the quantization precision, which is set when the parameters are initialized.

5. The method according to claim 1, characterized in that The result of gray value range in step 4) Perform data type conversion to obtain an unsigned integer The formula is: in It is the intermediate variable required for data type conversion. Its data type is integer and its value range is [0,γ].

6. The method according to claim 1, characterized in that The maximum radiance value found in step 5) based on the frame image And the anti-overexposure coefficient α updates the quantization threshold ρ, and the formula is: Among them, the overexposure prevention coefficient α is set during parameter initialization.

7. An adaptive threshold radiance quantization device for infrared imaging simulation, implemented by FPGA, characterized in that: including: An image data receiving module, used to receive the externally input radiance image stream, and complete the functions of signal type conversion, data transmission protocol parsing, frame start signal generation, and data bit-width conversion, to obtain radiance value data and output it; A quantization threshold calculation module, used to receive the radiance value data and the image frame start signal from the image data receiving module, and perform radiance value comparison based on a floating-point operation core. Until the image frame start signal is high, update the maximum radiance value, and then calculate and update the quantization threshold from the maximum radiance value, and output it to the normalization calculation module; A normalization calculation module, used to receive the radiance value data from the image data receiving module and the quantization threshold from the quantization threshold calculation module, and process the two based on a floating-point divider to achieve the normalization processing of the input radiance value; A quantization mapping module, used to map the normalization processing result obtained by the normalization calculation module to the grayscale value range based on a floating-point multiplier to obtain a quantization mapping result, and transmit it to the data type conversion module; A data type conversion module, used to convert the data type of the quantization mapping result to an unsigned integer type based on a floating-point operation core to obtain the converted quantization mapping result; A quantization result output module, used to perform data caching, UDP packet generation, and Ethernet sending operations on the converted quantization mapping result output by the data type conversion module in sequence.

8. The device according to claim 7, characterized in that The described image data receiving module includes: An optical module control sub-module, used to convert the received optical signal into an electrical signal and transmit it to the protocol parsing sub-module; A protocol parsing sub-module, used to parse the signal output by the optical module control sub-module according to the Aurora64b / 66b protocol standard to obtain a 64-bit wide data stream containing radiance information, and identify specific identifiers therein to generate a frame start signal; A bit-width conversion sub-module, used to convert the data stream input by the protocol parsing sub-module from 64-bit width to 32-bit width through off-chip high-speed memory caching to obtain a correctly formatted radiance value data stream.

9. A mobile terminal, including a processing chip, a memory, an input / output interface, and a program file stored in the memory and loadable for running on the processing chip, characterized in that: The processing chip uses an FPGA chip or a ZYNQ series heterogeneous chip, used to run the program file to execute the steps of the adaptive threshold radiance quantization method for infrared imaging simulation and the device design method described in any one of claims 1 to 8; The memory, which is a non-volatile readable memory, is used to store the program file; The input and output interface includes an optical fiber interface and a Gigabit Ethernet interface, which are used for input and output of images or videos; The program file is stored in the memory and can be loaded into the processing chip for execution, and is used to implement the adaptive threshold radiance quantification method steps and device design for infrared imaging simulation as described in any one of claims 1 to 8.

10. A non-volatile readable storage medium, characterized in that: Program instructions are stored, and the program instructions are run to execute the steps of the adaptive threshold radiance quantization method for infrared imaging simulation as described in any one of claims 1 to 6, or to realize the functions of each module of the quantization device as described in claim 7.

Images