Modulation imaging system and adaptive image gray scale control method and system

Abstract

The invention provides a modulation imaging system and a self-adaptive gray control method and system, and relates to the field of optical detection imaging, and the self-adaptive gray control method comprises the steps: determining the mapping relation between the on-off state of each lens in a digital micromirror array and an image through a trained neural network model; an image of the imaging unit is obtained, the gray level of each pixel in the image is detected, overexposure pixels in the image are recognized, the overexposure pixels comprise absolute overexposure pixels and influence overexposure pixels, the absolute overexposure pixels are pixels with the gray level exceeding a first gray level threshold value, the influence overexposure pixels are pixels with the gray level exceeding a second gray level threshold value, and the absolute overexposure pixels are pixels with the gray level exceeding a second gray level threshold value; the first gray threshold is smaller than the first gray threshold, absolute overexposure pixels exist in a preset range influencing the overexposure pixels, and the first gray threshold is larger than the second gray threshold; the target lens is determined based on the overexposure pixel, the target lens in the digital micromirror array is controlled to be closed, the requirement of pixel-level mapping of an imaging system is met, and interference of a highlight target in a specific area is effectively restrained.

Description

Technical Field

[0001] This invention relates to the field of optical detection imaging, and more particularly to a modulation imaging system, adaptive image grayscale control and system. Background Technology

[0002] In order to control the imaging effect through digital micromirror array (DMD), pixel-level mapping algorithms and DMD micromirror control methods need to be designed. The mapping relationship between the DMD micromirror and the pixels of the image sensor (CCD) with charge-coupled device must be accurately calculated in order to accurately control the deflection state of the DMD micromirror and thus modulate the incident light intensity in each region of the imaging system. However, the relevant technologies do not have a specific control process for the DMD and the amount of computation required for control is large. Summary of the Invention

[0003] To address the above problems, this invention provides a modulation imaging system, an adaptive image control method, and a system. It proposes a precise mapping algorithm between DMD and CCD pixels, utilizing an improved attention neural network to learn the mapping relationship between the DMD and CCD, thereby enhancing the accuracy and generalization of the mapping and meeting the pixel-level mapping requirements of the imaging system. To address the critical issue of high-brightness interference, a gray-level control method based on DMD is specifically designed to control and adjust the state of the micromirrors on the DMD corresponding to the target pixels on the CCD, effectively suppressing high-brightness interference.

[0004] The first aspect of this invention provides an adaptive image grayscale control method, which is applied to a modulation imaging system. The modulation imaging system includes a light source, a lens group, a digital micromirror array, and an imaging unit. The light emitted by the light source passes through the lens group and is directed towards the digital micromirror array. The digital micromirror array reflects the light to the imaging unit so that the imaging unit generates an image. The adaptive image grayscale control method includes: determining the mapping relationship between the on / off state of each lens in the digital micromirror array and the image through a trained neural network model; acquiring the image of the imaging unit, detecting the grayscale of each pixel in the image, and identifying overexposed pixels in the image, wherein the overexposed pixels include absolutely overexposed pixels and influencing overexposed pixels, the absolutely overexposed pixels are pixels whose grayscale exceeds a first grayscale threshold, and the influencing overexposed pixels are pixels whose grayscale exceeds a second grayscale threshold, is less than the first grayscale threshold, and exists within a preset range of the absolutely overexposed pixels, wherein the first grayscale threshold is greater than the second grayscale threshold; determining the target lens based on the overexposed pixels, and controlling the target lens in the digital micromirror array to turn off.

[0005] In some implementations, detecting the grayscale of each pixel in the image and identifying overexposed pixels in the image includes: determining the absolutely overexposed pixels in the image; and determining the pixels affecting overexposure within the preset range of each absolutely overexposed pixel.

[0006] In some implementations, detecting the grayscale of each pixel in the image and identifying overexposed pixels in the image includes: selecting a pixel in the image; determining the pixel as a candidate overexposed pixel when the pixel's grayscale is between a second grayscale threshold and a first grayscale threshold, and determining the pixel as an absolutely overexposed pixel when the pixel's grayscale is greater than the first grayscale threshold; acquiring the grayscale of each pixel within a preset range of the candidate overexposed pixel, and determining the candidate overexposed pixel as the influencing overexposed pixel when there are pixels within the preset range whose grayscale exceeds the first grayscale threshold, and determining the pixels whose grayscale exceeds the first grayscale threshold as the absolutely overexposed pixel. Overexposed pixels; Select another unselected pixel in the image, and repeat the process of determining the phase as a candidate overexposed pixel when the gray level of the pixel is between the second gray level threshold and the first gray level threshold, and when the gray level of each pixel within a preset range of the candidate overexposed pixels is such that there is a pixel within the preset range whose gray level exceeds the first gray level threshold, determine the candidate overexposed pixel as the overexposed pixel that affects the image, and determine the pixel whose gray level exceeds the first gray level threshold as the absolutely overexposed pixel. In the process of selecting another pixel in the image, skip the pixels that have been determined as the absolutely overexposed pixels until all pixels that have not been determined as the absolutely overexposed pixels are selected.

[0007] In some implementations, determining the target lens based on the overexposed pixel and controlling the target lens in the digital micromirror array to turn off includes: determining the target lens corresponding to the overexposed pixel according to the mapping relationship; and controlling the target lens to turn off.

[0008] In some implementations, the neural network model includes an input layer, multiple convolutional layers, pooling layers, a Transformer encoding layer, and a fully connected layer.

[0009] In some implementations, the formula for the improved multi-head self-attention mechanism in the Transformer coding layer is: In the formula, Q, K, and V represent the query weight matrix, key weight matrix, and value weight matrix, respectively; Softmax is the softmax activation function; k represents the number of heads in the self-attention mechanism; c is the scaling factor; and IN is instance normalization.

[0010] A second aspect of this invention provides an adaptive image grayscale control system, comprising: a mapping module, configured to determine the mapping relationship between the on / off state of each lens in the digital micromirror array and the image through a trained neural network model; a detection module, configured to acquire an image of the imaging unit, detect the grayscale of each pixel in the image, and identify overexposed pixels in the image, wherein the overexposed pixels include absolutely overexposed pixels and influencing overexposed pixels, the absolutely overexposed pixels being pixels whose grayscale exceeds a first grayscale threshold, and the influencing overexposed pixels being pixels whose grayscale exceeds a second grayscale threshold, is less than the first grayscale threshold, and exists within a preset range of the absolutely overexposed pixels, wherein the first grayscale threshold is greater than the second grayscale threshold; and a control module, configured to determine a target lens based on the overexposed pixels and control the target lens in the digital micromirror array to turn off.

[0011] In some embodiments, the detection module is further configured to determine the absolutely overexposed pixels in the image; and to determine the pixels affecting overexposure within the preset range of each of the absolutely overexposed pixels.

[0012] In some embodiments, the detection module is further configured to: select a pixel of the image; determine the pixel as a candidate overexposed pixel when the pixel's grayscale is between the second grayscale threshold and the first grayscale threshold, and determine the pixel as an absolutely overexposed pixel when the pixel's grayscale is greater than the first grayscale threshold; acquire the grayscale of each pixel within a preset range of the candidate overexposed pixel, and determine the candidate overexposed pixel as the influencing overexposed pixel and the pixel whose grayscale exceeds the first grayscale threshold as the absolutely overexposed pixel when there are pixels within the preset range whose grayscale exceeds the first grayscale threshold; and select the... Another unselected pixel in the image is selected, and the process is repeated whereby the phase is determined to be a candidate overexposed pixel when the gray level of the pixel is between the second gray level threshold and the first gray level threshold. Furthermore, if, within a preset range of gray levels of the candidate overexposed pixels, there is a pixel whose gray level exceeds the first gray level threshold, the candidate overexposed pixel is determined to be the overexposed pixel that affects the image, and the pixel whose gray level exceeds the first gray level threshold is determined to be the absolutely overexposed pixel. During the selection of another pixel in the image, pixels already determined to be absolutely overexposed pixels are skipped until all pixels not determined to be absolutely overexposed pixels are selected.

[0013] A third aspect of this invention provides a modulation imaging system, comprising: a light source, a first lens group, a total internal reflection prism, a digital micromirror array, a second lens group, and an imaging unit; the light source outputs light rays, which sequentially pass through the first lens group and the total internal reflection prism before being directed toward the digital micromirror array; the digital micromirror array reflects the light rays, which then pass through the total internal reflection prism and the second lens group before being directed toward the imaging unit to generate an image; the total internal reflection prism directs the light rays perpendicularly to the digital micromirror array before being directed toward the digital micromirror array; the total internal reflection prism also directs the light rays perpendicularly to the imaging unit before being directed toward the imaging unit.

[0014] This invention provides an adaptive image grayscale control method. A trained neural network obtains the mapping relationship between a digital micromirror array (DMI) and the image. Overexposed pixels in the image are identified, and the corresponding target micromirrors in the DMI array are turned off to remove these overexposed pixels, improving image clarity. Furthermore, by separating absolute overexposed pixels from those that affect overexposed pixels, the method accurately suppresses overexposed pixels while preserving pixels whose grayscale is unaffected by overexposed pixels. Additionally, by improving the instance normalization of the multi-head self-attention architecture in the neural network, the computational load of the control method is reduced. Attached Figure Description

[0015] Figure 1 A flowchart illustrating the first adaptive image grayscale control method provided in an embodiment of the present invention; Figure 2 A flowchart illustrating the second adaptive image grayscale control method provided in an embodiment of the present invention; Figure 3 A flowchart illustrating the third adaptive image grayscale control method provided in this embodiment of the invention; Figure 4 A schematic diagram of the neural network model in the adaptive image grayscale control method provided in this embodiment of the invention; Figure 5 for Figure 4 A schematic diagram of the architecture of the multi-head self-attention extraction layer in the neural network model; Figure 6 This is a schematic diagram of the structure of an adaptive image grayscale control system provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of a modulation imaging system provided in an embodiment of the present invention.

[0016] Explanation of reference numerals in the attached figures 1. Input layer; 2. Convolutional layer; 3. Pooling layer; 4. Transformer encoding layer; 5. Multi-head self-attention extraction layer; 6. Feedforward network; 7. Fully connected layer; 51. 1x1 Convolutional Layer; 52. View Operation Layer; 53. Matrix Multiplication Layer; 54. Scaling Layer; 55. Softmax Activation Layer; 56. Instance Normalization Layer; 10. Light source; 20. First lens group; 30. All-internal emission prism; 40. Digital micromirror array; 50. Second lens group; 60. Imaging unit; 100. Mapping module; 200. Detection module; 300. Control module. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] The specific technical features described in the various embodiments in the detailed implementation can be combined in various ways without contradiction. For example, different implementation methods can be formed by combining different specific technical features. In order to avoid unnecessary repetition, the various possible combinations of the specific technical features in this invention will not be described separately.

[0019] It should also be noted that, in order to avoid obscuring the present invention with unnecessary details, only the structures and / or processing steps closely related to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0020] Additionally, it should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. In the following description, the terms "first," "second," etc., are used merely to distinguish different objects and do not indicate any similarity or connection between them. It should be understood that the directional descriptions such as "above," "below," "inside," and "outside" refer to the orientation under normal use conditions.

[0021] In some embodiments, the adaptive image grayscale control method is applied to a modulation imaging system, which includes a light source, a lens group, a digital micromirror array (DMI), and an imaging unit. The light source outputs light, which, after passing through the lens group, is directed to the DMI. The DMI reflects the light to the imaging unit, which then forms an image corresponding to the received light. See also... Figure 1, Figure 1 This is a flowchart illustrating the first adaptive image grayscale control method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the adaptive image grayscale control method includes: Step S101: Determine the mapping relationship between the on / off state of each lens in the digital micromirror array and the image through the trained neural network model.

[0022] This can be understood as follows: An initial neural network model is trained using a large amount of labeled data. This model takes the on / off states of each lens in the digital micromirror array (DMI) as input and the image formed by the imaging unit as output. The labeled data includes the on / off states of each lens in the DMI and the actual image formed by the imaging unit in each on / off state. The on / off states of the DMI in the training data are input into the neural network model, which outputs a preset image. An evaluation function is constructed based on the difference between the pixels of the predicted image and the corresponding pixels of the actual image in the training data, thus obtaining a loss value. This loss value represents the difference between the predicted image and the actual image; the smaller the loss value, the smaller the difference. The weight and threshold parameters in the neural network are adjusted until the loss value reaches its current minimum value. This process is considered to complete one training iteration. The labeled data in the training data are used to iteratively train the neural network model until the loss value after multiple generations of training is consistently less than the preset loss threshold. At this point, the model is considered to have completed training.

[0023] The output of the neural network model also includes the correspondence between the position of the mirror in the digital micromirror array and the position of the pixel in the image. That is, it determines which pixels in the image are affected by the position of the mirror in the digital micromirror array. Through this correspondence, the position of the corresponding mirror can be deduced from the position of the pixel in the image.

[0024] The trained neural network model implicitly reflects the mapping relationship between the on / off state of each lens in the digital array and the image generated by the imaging unit through the weight parameters and the connection relationship of each node. This mapping relationship includes not only the overall correspondence between the on / off relationship of each lens and the generated image, but also the correspondence between the position of the mirror in the digital micromirror array and the position of the pixel in the image. Through this mapping relationship, it is possible not only to deduce the overall on / off state of each lens from the generated image, but also to deduce the position of the lens that affects the grayscale of the pixel from the position of the pixel in the generated image.

[0025] Step S102: Acquire the image of the imaging unit, detect the grayscale of each pixel in the image, and identify overexposed pixels in the image.

[0026] Overexposed pixels include absolutely overexposed pixels and pixels that affect overexposed pixels. Absolutely overexposed pixels are pixels whose grayscale exceeds a first grayscale threshold, meaning they are overexposed directly under strong light. Pixels that affect overexposed pixels are pixels whose grayscale exceeds a second grayscale threshold but is less than the first grayscale threshold, and exist within a preset range of overexposed pixels in the image. This preset range can be represented by pixel spacing; for example, the preset range of pixels affecting overexposed pixels is a range where the distance to the affected overexposed pixel is less than 7 pixels. Since the second grayscale threshold is less than the first grayscale threshold, pixels affecting overexposed pixels have a relatively large grayscale value, and there are absolutely overexposed pixels nearby. The larger grayscale value of the affected pixel is due to the influence of the brightness of the absolutely overexposed pixels.

[0027] Furthermore, for pixels with a gray level less than the first gray level threshold and greater than the second gray level threshold, if there are no absolutely overexposed pixels within the preset range of the pixel, it is considered that the large gray level of the pixel is not caused by overexposure. By not considering such pixels with large gray levels as overexposed pixels, it is possible to retain pixels with large gray levels that are not affected by overexposure.

[0028] Step S103: Determine the target lens based on the overexposed pixels and control the target lens in the digital micromirror array to turn off.

[0029] Optionally, the grayscale of overexposed pixels is adjusted to a second grayscale threshold to form a predicted image that suppresses overexposure. Based on the suppressed image and the mapping relationship, the overall lens switching state of the digital micromirror array is deduced. The overall lens switching state obtained is compared with the current lens switching state to obtain the target lens. Optionally, the lenses in the digital micromirror array that affect these overexposed pixels are deduced based on the position of the overexposed pixels as the target lenses. By controlling the target lenses to be turned off, overexposure in the image generated by the imaging unit can be suppressed, thereby making the generated image clearer.

[0030] In some embodiments, see Figure 2 , Figure 2 This is a flowchart illustrating the second adaptive image grayscale control method provided in this embodiment of the invention. Figure 1 The grayscale control method shown is different, Figure 1 Step S102 includes: Step S201: Determine the absolutely overexposed pixels in the image.

[0031] Specifically, overexposed pixels in the generated image are identified by determining the pixels whose gray levels exceed a first gray level threshold.

[0032] Step S202: Determine the pixels that affect overexposure within the preset range of each absolutely overexposed pixel.

[0033] Specifically, within the preset range of absolutely overexposed pixels, pixels with a gray level greater than the second gray level threshold and less than the first gray level threshold are searched, and these pixels are identified as pixels that affect overexposure. This can be understood as follows: if there are pixels with such a large gray level within the preset range of overexposed pixels, then such pixels with such a large gray level are also within the preset range of absolutely overexposed pixels, and thus such pixels with such a large gray level are pixels that affect overexposure.

[0034] This method for determining overexposed pixels can perform parallel calculations on each pixel simultaneously, thereby speeding up the determination of overexposed pixels.

[0035] In some embodiments, please refer to Figure 3 , Figure 3 This is a flowchart illustrating the third adaptive image grayscale control method provided in this embodiment of the invention, as shown below. Figure 3 As shown, with Figure 1 The adaptive image grayscale control method shown is different from the one described above. Figure 1 Step S102 includes: Step S301: Select a pixel.

[0036] Specifically, the selected pixel can be a randomly selected pixel or pixels selected in a certain order.

[0037] Step S302: When the gray level of a pixel is between the second gray level threshold and the first gray level threshold, the pixel is determined as a candidate overexposed pixel; when the gray level of a pixel is greater than the first gray level threshold, the pixel is determined as an absolutely overexposed pixel.

[0038] This can be understood as follows: if the gray level of a pixel is greater than the first gray level threshold, the pixel is directly identified as an absolutely overexposed pixel and proceeds to step S304; if the gray level of a pixel is greater than the second gray level threshold but less than the first gray level threshold, the pixel is identified as a candidate overexposed pixel and further judgment is required through step S303.

[0039] Step S303: Within the preset range of the candidate overexposed pixels, if there are pixels whose grayscale values ​​exceed the first grayscale threshold within the preset range, the candidate overexposed pixels are determined as overexposed pixels that affect the exposure, and the pixels whose grayscale values ​​exceed the first threshold are determined as absolutely overexposed pixels.

[0040] This can be understood as detecting whether there are pixels with gray levels exceeding a first gray level threshold within a preset range of candidate overexposed pixels. If there are pixels with gray levels exceeding the first gray level threshold, the candidate overexposed pixels are identified as pixels that affect overexposedness, and all of these pixels with gray levels exceeding the first gray level threshold are identified as absolutely overexposed pixels. If there are no pixels with gray levels exceeding the first gray level threshold, the candidate overexposed pixel is considered not to be an overexposed pixel.

[0041] Step S304: Select another unselected pixel in the image, and repeat steps S302 and S303, skipping pixels that have been determined to be absolutely overexposed pixels during the process of selecting another pixel in the image, until all pixels that have not been determined to be absolutely overexposed pixels are selected.

[0042] This can be understood as follows: by repeating steps S302 and S303 to detect the remaining pixels one by one, it is determined whether these pixels are overexposed. Since a portion of the absolutely overexposed pixels can be identified in step S302, there is no need to repeatedly detect these pixels. Therefore, when selecting another pixel for overexposed detection, pixels already identified as absolutely overexposed are skipped until all pixels not identified as absolutely overexposed are selected. By performing overexposed detection on each pixel in the image one by one, and skipping pixels already identified as absolutely overexposed during detection, the repetitive calculation of overexposed detection is minimized.

[0043] In some embodiments, such as Figure 4 As shown, Figure 1 The neural network model includes: input layer 1, multiple convolutional layers 2, pooling layer 3, Transformer encoding layer 4, and fully connected layer 7. Specifically, convolutional layer 2 is used to extract features from the data, effectively capturing important features; pooling layer 3 is used to reduce the size of the feature map and reduce computational cost; Transformer encoding layer 4 includes a multi-head self-attention extraction layer 5 and a feedforward network 6. The multi-head self-attention extraction layer 5 uses a multi-head self-attention mechanism to extract and refine feature information from a global perspective, and the feedforward network 6 consists of two fully connected layers and an activation function to enhance the model's expressive power. After passing through the multi-head self-attention extraction layer 5 and the feedforward network 6, residual connections and layer normalization are applied to ensure accelerated model training and prevent gradient vanishing; the fully connected layer 7 is located at the end of the network, integrating the extracted features and performing regression analysis. Convolutional neural networks focus on local information, while Transformers can capture long-distance dependencies of features and have efficient global information extraction capabilities, but they require large-scale data training and convergence. Combining the advantages of both is highly beneficial for the mapping needs of large-scale DMD to CCD data.

[0044] Among them, such as Figure 5As shown, the architecture of the multi-head self-attention extraction layer 5 includes: a 1x1 convolutional layer 51, a view operation layer 52, a matrix multiplication layer 53, a scaling layer 54, a softmax activation layer 55, and an instance normalization layer 56; wherein, the 1x1 convolutional layer 51 is used for feature extraction or feature dimensionality reduction, the view operation layer 52 is used for feature transformation, the matrix multiplication layer 53 is used for fusing attention features, the scaling layer 54 is used to reduce computation and prevent gradient explosion, the softmax activation layer 55 is used to balance attention weights through activation function features, and the instance normalization layer 56 is used to reduce computation through instance normalization.

[0045] The formula for the improved multi-head self-attention mechanism in the Transformer coding layer is as follows: In the formula, IN represents instance normalization, Q, K, and V represent the query weight matrix, key weight matrix, and value weight matrix, respectively, k represents the number of heads in the self-attention mechanism, and c is a scaling factor. Instance normalization is used to mitigate the performance degradation caused by MSA combining different attention feature inputs, and Conv represents a standard 1×1 convolution, simulating the interaction between various attention mechanisms. Figure 5 As shown, EMSA takes the features as input to a 1×1 convolutional layer to obtain the Q, K, V weight matrices, and performs dimensional transformation through the View operation, converting the (N, C, H, W) features into (N, 4, H / 4, W / 4), where N represents the batch size, C is the number of channels, and H and W are the feature map sizes. After dimensional transformation, Q and K perform matrix multiplication, and after scaling, one-dimensional convolution is used to further extract features and perform instance normalization. Finally, Q, K, and V are merged and dimensionally transformed to obtain enhanced attention features with the same dimensions as the input features. EMSA effectively solves the problem of increased computation and parameter count caused by MSA.

[0046] This invention also provides an adaptive image grayscale control system, which is used to achieve, for example... Figures 1 to 5 The adaptive image grayscale control method shown in any one of the images.

[0047] In some embodiments, such as Figure 6As shown, the adaptive image grayscale control system includes a mapping module 100, a detection module 200, and a control module 300. The mapping module 100 is used to determine the mapping relationship between the on / off state of each lens in the digital micromirror array and the image through a trained neural network model. The detection module 200 is used to acquire images of the imaging unit, detect the grayscale of each pixel in the image, and identify overexposed pixels in the image. Overexposed pixels include absolutely overexposed pixels and influencing overexposed pixels. Absolutely overexposed pixels are pixels whose grayscale exceeds a first grayscale threshold, and influencing overexposed pixels are pixels whose grayscale exceeds a second grayscale threshold, is less than the first grayscale threshold, and exists within a preset range of influencing overexposed pixels but are otherwise absolutely overexposed. The first grayscale threshold is greater than the second grayscale threshold. The control module 300 is used to determine the target lens based on the overexposed pixels and control the target lens in the digital micromirror array to turn off.

[0048] In some embodiments, such as Figure 6 As shown, the detection module 200 is also used to determine the absolutely overexposed pixels in the image; and to determine the pixels that affect the overexposure within a preset range of each absolutely overexposed pixel.

[0049] In some embodiments, such as Figure 6 As shown, the detection module 200 is further configured to: select a pixel in the image; determine a pixel as a candidate overexposed pixel when its grayscale value is between a second grayscale threshold and a first grayscale threshold, and determine a pixel as an absolutely overexposed pixel when its grayscale value is greater than the first grayscale threshold; acquire the grayscale values ​​of each pixel within a preset range of the candidate overexposed pixel, and determine the candidate overexposed pixel as an influencing overexposed pixel and the pixel whose grayscale value exceeds the first grayscale threshold as an absolutely overexposed pixel if there are pixels within the preset range whose grayscale value exceeds the first grayscale threshold; and select another pixel in the image. A pixel that has not been selected is repeatedly identified as a candidate overexposed pixel when its grayscale value is between a second grayscale threshold and a first grayscale threshold. The grayscale values ​​of each pixel within a preset range of the candidate overexposed pixel are also considered. If a pixel with a grayscale value exceeding the first grayscale threshold exists within the preset range, the candidate overexposed pixel is identified as an influencing overexposed pixel. Pixels with a grayscale value exceeding the first grayscale threshold are identified as absolutely overexposed pixels. Furthermore, pixels already identified as absolutely overexposed pixels are skipped during the selection of another pixel in the image until all pixels not identified as absolutely overexposed pixels are selected.

[0050] This invention also provides a modulation imaging system, which is an application scenario for adaptive image grayscale control method and adaptive image grayscale control system.

[0051] In some embodiments, such as Figure 7As shown, the modulation imaging system includes: a light source 10, a first lens group 20, a total internal reflection prism 30, a digital micromirror array 40, a second lens group 50, and an imaging unit 60. The light source 10 outputs light rays, which pass sequentially through the first lens group 20 and the total internal reflection prism 30 before reaching the digital micromirror array 40. The digital micromirror array 40 reflects the light rays, which then pass through the total internal reflection prism 30 and the second lens group 50 before reaching the imaging unit 60 to generate an image. The total internal reflection prism 30 directs the light rays perpendicularly to the digital micromirror array 40. The total internal reflection prism 30 also directs the light rays perpendicularly to the imaging unit 60. The imaging unit 60 can be a CCD camera.

[0052] Because the imaging system involves a secondary imaging process and the distance between the digital micromirror array 40 and the imaging unit 60 is limited, it is difficult to directly design an angle folding. Therefore, a total internal reflection prism 30 is added to change the direction of the optical path. After the addition, the overall optical path will undergo angle folding and transform into two perpendicular optical paths.

[0053] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An adaptive image grayscale control method, characterized in that, The adaptive image grayscale control method is applied to a modulation imaging system, which includes a light source, a lens group, the digital micromirror array, and an imaging unit. The light emitted by the light source passes through the lens group and is then directed to the digital micromirror array. The digital micromirror array reflects the light to the imaging unit so that the imaging unit generates an image. The adaptive image grayscale control method includes: The mapping relationship between the on / off state of each lens in the digital micromirror array and the image is determined by the trained neural network model. The image of the imaging unit is acquired, and the gray level of each pixel in the image is detected and overexposed pixels in the image are identified. The overexposed pixels include absolute overexposed pixels and influencing overexposed pixels. The absolute overexposed pixels are pixels whose gray level exceeds a first gray level threshold. The influencing overexposed pixels are pixels whose gray level exceeds a second gray level threshold, is less than the first gray level threshold, and exist within a preset range of the absolute overexposed pixels. The first gray level threshold is greater than the second gray level threshold. The target lens is determined based on the overexposed pixels, and the target lens in the digital micromirror array is controlled to turn off.

2. The adaptive image grayscale control method according to claim 1, characterized in that, The step of detecting the grayscale of each pixel in the image and identifying overexposed pixels in the image includes: Determine the absolutely overexposed pixels in the image; The overexposed pixels are determined within the preset range of each of the absolutely overexposed pixels.

3. The adaptive image grayscale control method according to claim 1, characterized in that, The step of detecting the grayscale of each pixel in the image and identifying overexposed pixels in the image includes: Select one pixel of the image; When the gray level of a pixel is between the second gray level threshold and the first gray level threshold, the pixel is determined to be a candidate overexposed pixel. When the gray level of a pixel is greater than the first gray level threshold, the pixel is determined to be an absolutely overexposed pixel. The grayscale of each pixel within a preset range of the candidate overexposed pixels is obtained. If there are pixels within the preset range whose grayscale exceeds the first grayscale threshold, the candidate overexposed pixels are determined as the overexposed pixels that affect the overexposure. The pixels whose grayscale exceeds the first grayscale threshold are determined as the absolutely overexposed pixels. Select another unselected pixel in the image, and repeat the process of determining the phase as a candidate overexposed pixel when the gray level of the pixel is between the second gray level threshold and the first gray level threshold, and the process of determining the candidate overexposed pixel as the overexposed pixel when there is a pixel with a gray level exceeding the first gray level threshold within a preset range of the candidate overexposed pixel, and the pixel with a gray level exceeding the first gray level threshold as the absolutely overexposed pixel. In the process of selecting another pixel in the image, skip the pixels that have been determined as the absolutely overexposed pixels until all pixels that have not been determined as the absolutely overexposed pixels are selected.

4. The adaptive image grayscale control method according to any one of claims 1 to 3, characterized in that, The step of determining the target lens based on the overexposed pixels and controlling the target lens in the digital micromirror array to turn off includes: The target lens corresponding to the overexposed pixel is determined based on the mapping relationship; Control the target lens to close.

5. The adaptive image grayscale control method according to any one of claims 1 to 3, characterized in that, The neural network model includes: The system consists of an input layer, multiple convolutional layers, pooling layers, a Transformer encoding layer, and a fully connected layer.

6. The adaptive image grayscale control method according to claim 5, characterized in that, The formula for the improved multi-head self-attention mechanism in the Transformer coding layer is as follows: In the formula, Q, K, and V represent the query weight matrix, key weight matrix, and value weight matrix, respectively; Softmax is the softmax activation function; k represents the number of heads in the self-attention mechanism; c is the scaling factor; and IN is instance normalization.

7. An adaptive image grayscale control system, characterized in that, The adaptive image grayscale control system includes: The mapping module is used to determine the mapping relationship between the on / off state of each lens in the digital micromirror array and the image through a trained neural network model; The detection module is used to acquire the image of the imaging unit, detect the gray level of each pixel in the image, and identify overexposed pixels in the image. The overexposed pixels include absolutely overexposed pixels and influencing overexposed pixels. The absolutely overexposed pixels are pixels whose gray level exceeds a first gray level threshold. The influencing overexposed pixels are pixels whose gray level exceeds a second gray level threshold, is less than the first gray level threshold, and exist within a preset range of the influencing overexposed pixels. The first gray level threshold is greater than the second gray level threshold. The control module is used to determine the target lens based on the overexposed pixels and control the target lens in the digital micromirror array to turn off.

8. The adaptive image grayscale control system according to claim 7, characterized in that, The detection module is further configured to determine the absolutely overexposed pixels in the image; and to determine the pixels affecting overexposure within the preset range of each absolutely overexposed pixel.

9. The adaptive image grayscale control system according to claim 7, characterized in that, The detection module is further configured to select a pixel of the image; to determine the pixel as a candidate overexposed pixel when the pixel's grayscale is between the second grayscale threshold and the first grayscale threshold, and to determine the pixel as an absolutely overexposed pixel when the pixel's grayscale is greater than the first grayscale threshold; and to obtain the grayscale of each pixel within a preset range of the candidate overexposed pixel, and to determine the candidate overexposed pixel as the overexposed pixel if there is a pixel within the preset range whose grayscale exceeds the first grayscale threshold, and to determine the pixel whose grayscale exceeds the first grayscale threshold as the absolutely overexposed pixel. It is also used to select another unselected pixel of the image, and repeat the process of determining the phase as a candidate overexposed pixel when the gray level of the pixel is between the second gray level threshold and the first gray level threshold, and the process of determining the candidate overexposed pixel as the affected overexposed pixel when there is a pixel with a gray level exceeding the first gray level threshold in the preset range of gray levels of each pixel in the preset range, and determining the pixel with a gray level exceeding the first gray level threshold as the absolutely overexposed pixel, and skipping pixels that have been determined as the absolutely overexposed pixel in the process of selecting another pixel of the image, until all pixels that have not been determined as the absolutely overexposed pixel are selected.

10. A modulation imaging system, characterized in that, The modulation imaging system includes: a light source, a first lens group, a total internal reflection prism, a digital micromirror array, a second lens group, and an imaging unit; The light source is used to output light, which passes sequentially through a first lens group and a total internal reflection prism before being directed toward the digital micromirror array. The digital micromirror array is used to reflect the light, which then passes through the total internal reflection prism and the second lens group before being directed toward the imaging unit to generate an image. The total internal reflection prism is used to direct the light perpendicularly to the digital micromirror array, and it is also used to direct the light perpendicularly to the imaging unit.

Images