FPGA-based CMOS image sensor exposure gain method and system

Abstract

The invention discloses an FPGA-based CMOS image sensor exposure gain method and system, and the method comprises the steps: collecting a frame of original image data through an FPGA, and calculating the average gray value of a current frame of image; comparing the average gray value of the current frame image with a set target gray threshold range, and judging whether an exposure parameter or a gain parameter of a current scene needs to be adjusted or not according to a comparison result; when the exposure parameter or the gain parameter needs to be adjusted, firstly judging whether the current gain parameter is an initial value, and selecting to subsequently adjust the exposure parameter or the gain parameter according to a judgment result; and performing adaptive adjustment on the exposure parameter or the gain parameter according to a set target gray threshold range, so that the average gray value of the current frame image reaches the set target gray threshold range, stopping adjustment, and outputting the current exposure parameter and the current gain parameter. The method is simple and efficient, automatic exposure can adapt to strong illumination and low illumination, and image flickering in the adjusting process is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of digital image processing, and in particular to an exposure gain method and system of a CMOS image sensor based on FPGA. Background Art

[0002] Exposure refers to the amount of light that enters the camera and shines on the image sensor. It is usually adjusted by the camera's shutter and is an important indicator that determines the brightness of the image captured by the camera. If too much light is irradiated on the sensor, it will cause "overexposure" and make the image too bright. If not enough light is irradiated on the sensor, it will cause "underexposure" and make the image too dark.

[0003] In the field of digital image processing, automatic exposure has always been a key technology, and with the development of machine vision, the requirements for image quality have gradually increased. Automatic exposure can automatically adjust the exposure according to the strength of external light to avoid overexposure or underexposure. A good automatic exposure algorithm not only improves the imaging effect of the camera, but also avoids the serious problem of image flickering during the adjustment process.

[0004] Limited by the convergence speed of the automatic exposure algorithm and the strong or weak ambient light, simply adjusting the exposure time of the image sensor can no longer achieve good imaging effects in extreme scenes. When the ambient light is weak, even if the exposure time is increased to saturation, the image is still in an "underexposed" state, resulting in dark imaging. In addition, the traditional automatic exposure algorithm uses a constant step size to adjust the exposure time, which has too many limitations and cannot meet the automatic exposure requirements of multiple scenes. Some algorithms use complex large models to predict exposure parameters, which consumes a lot of FPGA resources. Summary of the invention

[0005] The purpose of the present invention is to address the problems in the above-mentioned prior art and provide a CMOS image sensor exposure gain method and system based on FPGA, which uses FPGA to adaptively adjust the exposure time and gain value of the CMOS image sensor, which is simple and efficient, enables automatic exposure to adapt to strong light and low illumination, and avoids image flickering during the adjustment process.

[0006] In order to achieve the above object, the present invention has the following technical solutions: In a first aspect, a CMOS image sensor exposure gain method based on FPGA is provided, comprising: Collect a frame of original image data through FPGA and calculate the average gray value of the current frame image; Compare the average grayscale value of the current frame image with the set target grayscale threshold range, and determine whether the current scene needs to adjust the exposure parameter or gain parameter based on the comparison result; When the exposure parameter or gain parameter needs to be adjusted, first determine whether the current gain parameter is the initial value, and then choose to adjust the exposure parameter or gain parameter according to the determination result; According to the set target grayscale threshold range, the exposure parameter or gain parameter is adaptively adjusted to make the average grayscale value of the current frame image reach the set target grayscale threshold range, stop adjusting and output the current exposure parameter and gain parameter.

[0007] As a preferred solution, the method of collecting a frame of original image data through FPGA and calculating the average grayscale value of the current frame image includes: Calculate the total grayscale value of all pixels in the current frame image as follows:

[0008] The total number of pixels of a frame of image is counted as follows:

[0009] The average grayscale value of the current frame image is calculated by the sum of the grayscale values ​​of all pixels in the current frame image and the total number of pixels in a frame image as follows:

[0010] In the formula, Represents a pixel of a frame of image. Indicates the total number of pixels in a frame of image. Indicates the number of rows in a frame of image. Indicates the number of pixels in a line.

[0011] As a preferred solution, in the step of comparing the average grayscale value of the current frame image with the set target grayscale threshold range, and determining whether the current scene needs to adjust the exposure parameter or gain parameter based on the comparison result, if the average grayscale value of the current frame image is not within the set target grayscale threshold range, it is determined that the current scene needs to adjust the exposure parameter or gain parameter.

[0012] As a preferred solution, when the exposure parameter or gain parameter needs to be adjusted, it is first determined whether the current gain parameter is the initial value, and in the subsequent step of adjusting the exposure parameter or the gain parameter is selected according to the judgment result, if the current gain parameter is the initial gain value, the exposure parameter is selected for adjustment; if the current gain parameter is not the initial gain value, it is determined that the historical frame image has undergone gain adjustment, and when adjusting again, the state is preferentially pulled back to the gain parameter adjustment.

[0013] As a preferred solution, the step of adaptively adjusting the exposure parameter or gain parameter based on the set target grayscale threshold range uses a pre-established segmented exposure parameter adjustment model to predict the exposure parameter and the average grayscale value of the target frame image to achieve adaptive adjustment of the exposure parameter; and adopts a fixed step size to increase or decrease the gain value to achieve adaptive adjustment of the gain parameter.

[0014] As a preferred solution, the step of using a pre-established segmented exposure parameter adjustment model to predict the exposure parameters and average grayscale value of the target frame image to achieve adaptive adjustment of the exposure parameters includes: Under equal illumination intensity, when the grayscale value is in the middle area, the grayscale value and the exposure time are in a linear relationship. Then, according to the proportional relationship between the grayscale of the current frame and the target frame image, the exposure time of the current frame is proportionally enlarged and reduced to establish the first segmented exposure parameter adjustment model. The expression of the first segmented exposure parameter adjustment model is as follows:

[0015] In the formula, dsir_expo is the target frame exposure time predicted by the model, expo_time is the exposure time corresponding to the current frame, AVER_GRAY_OK is the optimal grayscale value of the target frame, avg_gray is the average gray value of the current frame.

[0016] As a preferred solution, the step of using a pre-established segmented exposure parameter adjustment model to predict the exposure parameters and average grayscale value of the target frame image to achieve adaptive adjustment of the exposure parameters includes: When the gray value is not in the middle area, the gray value and the exposure time have a nonlinear relationship. The second segmented exposure parameter adjustment model is established by the stepwise approximation method. The expression of the second segmented exposure parameter adjustment model is as follows:

[0017] In the formula, dsir_expo is the target frame exposure time predicted by the model, expo_time is the exposure time corresponding to the current frame, AVER_GRAY_OK is the optimal grayscale value of the target frame, avg_gray is the average gray value of the current frame.

[0018] As a preferred solution, in the step of adaptively adjusting the gain parameter by increasing or decreasing the gain value in a fixed step size, when the FPGA enters the gain parameter adjustment process, if it is determined that the average grayscale value of the current frame image is less than the target grayscale threshold range, the gain value is increased by configuring the value of the image register, otherwise the gain value is reduced.

[0019] In a second aspect, a CMOS image sensor exposure gain system based on FPGA is provided, comprising: The average gray value calculation module is used to collect a frame of original image data through FPGA and calculate the average gray value of the current frame image; A comparison and determination module is used to compare the average grayscale value of the current frame image with the set target grayscale threshold range, and determine whether the current scene needs to adjust the exposure parameter or gain parameter based on the comparison result; The adjustment object selection module is used to first determine whether the current gain parameter is an initial value when the exposure parameter or the gain parameter needs to be adjusted, and then select the subsequent adjustment of the exposure parameter or the gain parameter according to the judgment result; The adjustment output module is used to adaptively adjust the exposure parameter or gain parameter according to the set target grayscale threshold range, so that the average grayscale value of the current frame image reaches the set target grayscale threshold range, stop adjusting and output the current exposure parameter and gain parameter.

[0020] According to a third aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the FPGA-based CMOS image sensor exposure gain method is implemented.

[0021] Compared with the prior art, the present invention has at least the following beneficial effects: The FPGA-based CMOS image sensor exposure gain method of the present invention uses FPGA to adaptively adjust the exposure time and gain value of the CMOS image sensor, and can automatically adjust the exposure parameters and image gain values ​​according to different lighting scenes of the image to obtain the best shooting effect. By obtaining the average grayscale value of the current frame image and comparing it with the set target grayscale threshold range, if it exceeds the target grayscale threshold range, the exposure parameter or gain parameter is selected to be adjusted. When the exposure parameter or gain parameter needs to be adjusted, it is first determined whether the current gain parameter is the initial value, thereby determining whether the historical frame image has been gain adjusted, and the subsequent adjustment of the exposure parameter or gain parameter is selected according to the judgment result. Finally, according to the set target grayscale threshold range, the exposure parameter or gain parameter is adaptively adjusted so that the average grayscale value of the current frame image reaches the set target grayscale threshold range. The present invention realizes the process of continuously approaching the ideal grayscale value after a limited number of frame cycles, so that the existing automatic exposure can adapt to strong light and low illumination, and avoid image flickering during the adjustment process.

[0022] Furthermore, the present invention uses a pre-established segmented exposure parameter adjustment model to predict the exposure parameters and average grayscale values ​​of the target frame image, realizes adaptive adjustment of the exposure parameters, and adopts a fixed step size to increase or decrease the gain value to realize adaptive adjustment of the gain parameters. The present invention can complete the adaptive adjustment of the image exposure parameters and image gain values, and output the exposure parameters and gain values; by introducing the segmented modeling method, compared with the traditional automatic exposure scheme, the modeling prediction accuracy is higher, and the convergence speed of automatic exposure in different scenes is fast. At the same time, the high-precision modeling scheme can effectively avoid the oscillation in the exposure adjustment process, so that the exposure parameters can converge to the optimal value faster. On the basis of the traditional automatic exposure scheme, the automatic gain parameter is introduced to adjust the image brightness at the same time, so that the image can obtain the ideal grayscale value within a larger range of light intensity. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A schematic diagram of the implementation principle of the FPGA-based CMOS image sensor exposure gain method according to an embodiment of the present invention; Figure 2 Flowchart of a method for exposure gain of a CMOS image sensor based on FPGA according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, ordinary technicians in this field can also obtain other embodiments without making creative work.

[0026] In the field of digital image processing, automatic exposure has always been a key technology, and with the development of machine vision, the requirements for image quality have gradually increased. Automatic exposure can automatically adjust the exposure according to the intensity of external light, thereby avoiding overexposure or underexposure. A good automatic exposure algorithm not only improves the imaging effect of the camera, but also avoids the serious problem of image flickering during the adjustment process. A good automatic exposure algorithm can ensure that high-quality images can be obtained under various lighting conditions, which is particularly important for machine vision systems because image quality directly affects the accuracy of subsequent image analysis and recognition. During the automatic exposure adjustment process, if the algorithm is not designed properly, the image may flicker significantly when the exposure parameters are adjusted. This not only affects the user experience, but may also have a negative impact on the stability of the machine vision system. At the same time, with the continuous expansion of machine vision application scenarios, the automatic exposure algorithm needs to have stronger adaptability to cope with the challenges of different lighting conditions, different shooting objects and different shooting angles. For machine vision applications with high real-time requirements, such as autonomous driving and intelligent monitoring, the automatic exposure algorithm needs to achieve fast response while ensuring image quality.

[0027] See also Figure 1 and Figure 2 The embodiment of the present invention proposes an exposure gain method for a CMOS image sensor based on FPGA to solve the problem that the automatic exposure in the prior art cannot adapt to the requirements of strong light and low illumination, and the image flickers during the adjustment process. The method of the embodiment of the present invention can be summarized as follows: if the gray value of the current frame jumps out of the ideal gray value range, the automatic exposure parameter and gain parameter adjustment are turned on, and the process of continuously approaching the ideal gray value after a limited number of frame cycles.

[0028] Specifically, the FPGA-based exposure gain method for a CMOS image sensor according to an embodiment of the present invention includes: S1, collect a frame of original image data through FPGA, and calculate the average gray value of the current frame image; S2, comparing the average gray value of the current frame image with the set target gray threshold range, and determining whether the current scene needs to adjust the exposure parameter or gain parameter according to the comparison result; S3, when it is necessary to adjust the exposure parameter or the gain parameter, first determine whether the current gain parameter is an initial value, and select whether to subsequently adjust the exposure parameter or the gain parameter according to the determination result; S4, according to the set target grayscale threshold range, adaptively adjust the exposure parameter or gain parameter so that the average grayscale value of the current frame image is within the set target grayscale threshold range; S5, stop adjusting and output the current exposure parameters and gain parameters; S6. When the ambient light or the captured image changes, steps S1-S5 are repeated until appropriate exposure parameters and gain parameters are output.

[0029] In a possible implementation manner, the step S1 of collecting a frame of raw image data through FPGA and calculating the average grayscale value of the current frame image includes the following steps: The sum of the grayscale values ​​of all pixels in the current frame image is calculated as follows. Specifically, this embodiment takes 12-bit pixel depth as an example and counts the sum of the grayscale values ​​of all pixels:

[0030] The total number of pixels of a frame of image is counted as follows:

[0031] Specifically, taking an image with a resolution of 4096×3072 as an example, the total number of pixels in one frame is:

[0032] The average grayscale value of the current frame image is calculated by the sum of the grayscale values ​​of all pixels in the current frame image and the total number of pixels in a frame image according to the following formula: the average grayscale value of the current frame image = total grayscale value / total number of pixels:

[0033] In the formula, Represents a pixel of a frame of image. Indicates the total number of pixels in a frame of image. Indicates the number of rows in a frame of image. Indicates the number of pixels in a line.

[0034] In a possible implementation, step S2 compares the average grayscale value of the current frame image with the set target grayscale threshold range, and determines whether the current scene needs to adjust the exposure parameter or gain parameter according to the comparison result. If the average grayscale value of the current frame image is not within the set target grayscale threshold range, it is determined that the current scene needs to adjust the exposure parameter or gain parameter. Specifically, taking 12-bit pixel depth as an example, the maximum grayscale value of a single pixel is 4096, and the grayscale value of the image is about 2000, which is an image that conforms to human visual perception. Therefore, the grayscale value range of the image 1550-1950 is set as the upper and lower grayscale thresholds of the image target for determining whether the current image needs to adjust the parameters. Further, if the average grayscale value of the current frame image is calculated to be 1800, which is within the set upper and lower grayscale threshold range, it is determined that there is no need to adjust the parameters; if the average grayscale value of the current frame image is calculated to be 2500, which is not within the set upper and lower grayscale threshold range, it is necessary to adjust.

[0035] In a possible implementation, in step S3, when it is necessary to adjust the exposure parameter or the gain parameter, first determine whether the current gain parameter is the initial value, and select whether to subsequently adjust the exposure parameter or the gain parameter based on the determination result. If the current gain parameter is the initial gain value 1, the exposure parameter is selected for adjustment; if the current gain parameter is not the initial gain value 1, the historical frame image has already undergone gain adjustment, and when adjusting again, it is preferred to pull the state back to the gain parameter adjustment.

[0036] In a possible implementation, step S4 is a step of adaptively adjusting the exposure parameter or the gain parameter according to the set target grayscale threshold range: (1) Using the pre-established segmented exposure parameter adjustment model to predict the exposure parameters and average grayscale value of the target frame image, the adaptive adjustment of the exposure parameters is achieved; (2) Adopting a fixed step size to increase or decrease the gain value, adaptive adjustment of the gain parameter is achieved.

[0037] Furthermore, the step of using the pre-established segmented exposure parameter adjustment model in step (1) to predict the exposure parameters and average grayscale value of the target frame image to achieve adaptive adjustment of the exposure parameters includes: According to the photosensitivity characteristics of the CMOS image sensor, under equal light intensity, when the grayscale value is in the middle area (experiments show that under equal light intensity, the grayscale value and exposure time are positively correlated, forming an "S" curve, including a linear area and a nonlinear area; the middle area here refers to the linear area in the "S" curve), the grayscale value and exposure time are in a linear relationship. Then, according to the proportional relationship between the grayscale of the current frame and the target frame image, the exposure time of the future frame is proportionally enlarged and reduced in combination with the exposure time of the current frame, and the first segmented exposure parameter adjustment model is established. The expression of the first segmented exposure parameter adjustment model is as follows:

[0038] In the formula, dsir_expo is the target frame exposure time predicted by the model, expo_time is the exposure time corresponding to the current frame, AVER_GRAY_OK is the optimal grayscale value of the target frame, avg_gray is the average gray value of the current frame.

[0039] Specifically, for a pixel depth of 12 bits, the optimal grayscale value of the target frame is AVER_GRAY_OK When the average grayscale value of the current frame is 1800 and the exposure time is 1ms, the first segmented exposure parameter adjustment model predicts that the exposure time required for the next frame is 3ms. If the linearity of the exposure time and grayscale value of the image sensor in this area is high, the next frame can reach the target frame.

[0040] When the gray value is not in the middle area, the gray value and the exposure time have a nonlinear relationship. The second segmented exposure parameter adjustment model is established by the stepwise approximation method. The expression of the second segmented exposure parameter adjustment model is as follows:

[0041] In the formula, dsir_expo is the target frame exposure time predicted by the model, expo_time is the exposure time corresponding to the current frame, AVER_GRAY_OK is the optimal grayscale value of the target frame, avg_gray is the average gray value of the current frame.

[0042] Specifically, for a pixel depth of 12 bits, the optimal grayscale value of the target frame is AVER_GRAY_OK When the average grayscale value of the current frame is 1800, the average grayscale value of the current frame is 200, and the exposure time is 1ms, the model is adjusted according to the second segmented exposure parameter, and the exposure time required for the next frame is predicted to be (1800+200) / 2 / 200*1ms=5ms. After adjusting 5 times in this way, the lower limit of the set target grayscale threshold range can be reached, and the automatic exposure converges.

[0043] In the step (2) of adaptively adjusting the gain parameter by increasing or decreasing the gain value with a fixed step size, when the FPGA enters the gain parameter adjustment process, if it is determined that the average grayscale value of the current frame image is less than the target grayscale threshold range, the gain value is increased by configuring the value of the image register, otherwise the gain value is decreased.

[0044] Taking 12-bit pixel depth as an example, when the FPGA state enters the image gain adjustment, it is judged that the grayscale value of the current frame is 1000, which is less than the target grayscale value, and the exposure time has been adjusted to the saturation state. Therefore, before the next frame of valid signal of the image arrives, the gain value is increased by configuring the value of the image register, otherwise the gain is reduced.

[0045] In one possible implementation, if the average grayscale value of the current frame image reaches the set target grayscale threshold range, the adjustment of the exposure parameters and gain parameters is stopped, the state is switched to S1 and the process of S1-S5 is repeated until the average grayscale value of the output image is within the set target grayscale threshold range, then the automatic exposure gain adjustment is stopped; when the image, lighting, or scene changes, if the system detects that the average grayscale value obtained in S1 is not within the target range, the automatic exposure gain is triggered again, and the cycle continues.

[0046] Another embodiment of the present invention provides a CMOS image sensor exposure gain system based on FPGA, comprising: The average gray value calculation module is used to collect a frame of original image data through FPGA and calculate the average gray value of the current frame image; A comparison and determination module is used to compare the average grayscale value of the current frame image with the set target grayscale threshold range, and determine whether the current scene needs to adjust the exposure parameter or gain parameter based on the comparison result; The adjustment object selection module is used to first determine whether the current gain parameter is an initial value when the exposure parameter or the gain parameter needs to be adjusted, and then select the subsequent adjustment of the exposure parameter or the gain parameter according to the judgment result; The adjustment output module is used to adaptively adjust the exposure parameter or gain parameter according to the set target grayscale threshold range, so that the average grayscale value of the current frame image reaches the set target grayscale threshold range, stop adjusting and output the current exposure parameter and gain parameter.

[0047] Furthermore, the adjustment output module of the embodiment of the present invention uses a pre-established segmented exposure parameter adjustment model to predict the exposure parameters and average grayscale value of the target frame image, thereby realizing adaptive adjustment of the exposure parameters. The adjustment output module of the embodiment of the present invention uses a fixed step size to increase or decrease the gain value, thereby realizing adaptive adjustment of the gain parameter.

[0048] When realizing adaptive adjustment of exposure parameters, under equal light intensity, when the grayscale value is in the middle area, the grayscale value and the exposure time are in a linear relationship. Then, according to the proportional relationship between the grayscale of the current frame and the target frame image, the exposure time of the future frame is proportionally enlarged and reduced in combination with the exposure time of the current frame, and the first segmented exposure parameter adjustment model is established. The expression of the first segmented exposure parameter adjustment model is as follows:

[0049] In the formula, dsir_expo is the target frame exposure time predicted by the model, expo_time is the exposure time corresponding to the current frame, AVER_GRAY_OK is the optimal grayscale value of the target frame, avg_gray is the average gray value of the current frame.

[0050] When the gray value is not in the middle area, the gray value and the exposure time have a nonlinear relationship. The second segmented exposure parameter adjustment model is established by the stepwise approximation method. The expression of the second segmented exposure parameter adjustment model is as follows:

[0051] In the formula, dsir_expo is the target frame exposure time predicted by the model, expo_time is the exposure time corresponding to the current frame, AVER_GRAY_OK is the optimal grayscale value of the target frame, avg_gray is the average gray value of the current frame.

[0052] When implementing adaptive adjustment of the gain parameters, when the FPGA enters the gain parameter adjustment process, if it is determined that the average grayscale value of the current frame image is less than the target grayscale threshold range, the gain value is increased by configuring the value of the image register; if it is determined that the average grayscale value of the current frame image is greater than the target grayscale threshold range, the gain value is reduced by configuring the value of the image register.

[0053] Considering that gain will increase noise and reduce image quality, the present invention chooses to adjust the exposure parameters first, and then adjusts the sensor gain value when the exposure parameters reach saturation. The present invention proposes to adopt a segmented exposure parameter adjustment model, which accelerates the convergence speed of image automatic exposure and avoids the problem of image flicker to the greatest extent. When the gain parameter is adjusted, the present invention adopts a fixed step size, and in order to minimize noise, the gain parameter can only be adjusted within a smaller range. The present invention uses the powerful image data processing capability of FPGA to calculate the average grayscale value of the input frame image in real time, and combines feedback to complete the adaptive adjustment of image exposure and gain parameters, and can obtain better imaging effects in different scenarios.

[0054] Another embodiment of the present invention further provides an electronic device, comprising: A memory storing at least one instruction; and a processor executing the instruction stored in the memory to implement the FPGA-based CMOS image sensor exposure gain method.

[0055] Another embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the FPGA-based CMOS image sensor exposure gain method is implemented.

[0056] Exemplarily, the instructions stored in the memory may be divided into one or more modules / units, which are stored in a computer-readable storage medium and executed by the processor to implement the FPGA-based CMOS image sensor exposure gain method of the present invention. The one or more modules / units may be a series of computer-readable instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the server.

[0057] The electronic device may be a computing device such as a smart phone, a notebook, a PDA, and a cloud server. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the electronic device may also include more or fewer components, or a combination of certain components, or different components, for example, the electronic device may also include an input / output device, a network access device, a bus, etc.

[0058] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0059] The memory may be an internal storage unit of the server, such as a hard disk or memory of the server. The memory may also be an external storage device of the server, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (FlashCard), etc. equipped on the server. Furthermore, the memory may include both an internal storage unit of the server and an external storage device. The memory is used to store the computer-readable instructions and other programs and data required by the server. The memory may also be used to temporarily store data that has been output or is to be output.

[0060] It should be noted that the information interaction, execution process and other contents between the above-mentioned module units are based on the same concept as the method embodiment. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0061] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0062] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0063] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0064] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A CMOS image sensor exposure gain method based on FPGA, characterized in that: include: Collect a frame of original image data through FPGA and calculate the average gray value of the current frame image; Compare the average grayscale value of the current frame image with the set target grayscale threshold range, and determine whether the current scene needs to adjust the exposure parameter or gain parameter based on the comparison result; When the exposure parameter or gain parameter needs to be adjusted, first determine whether the current gain parameter is the initial value, and then choose to adjust the exposure parameter or gain parameter according to the determination result; According to the set target grayscale threshold range, the exposure parameter or gain parameter is adaptively adjusted to make the average grayscale value of the current frame image reach the set target grayscale threshold range, stop adjusting and output the current exposure parameter and gain parameter.

2. The FPGA-based CMOS image sensor exposure gain method according to claim 1, characterized in that: The method of collecting a frame of original image data through FPGA and calculating the average grayscale value of the current frame image includes: Calculate the total grayscale value of all pixels in the current frame image as follows: The total number of pixels of a frame of image is counted as follows: The average grayscale value of the current frame image is calculated by the sum of the grayscale values ​​of all pixels in the current frame image and the total number of pixels in a frame image as follows: In the formula, Represents a pixel of a frame of image. Indicates the total number of pixels in a frame of image. Indicates the number of rows in a frame of image. Indicates the number of pixels in a line.

3. The FPGA-based CMOS image sensor exposure gain method according to claim 1, characterized in that: In the step of comparing the average grayscale value of the current frame image with the set target grayscale threshold range, and determining whether the current scene needs to adjust the exposure parameter or the gain parameter based on the comparison result, if the average grayscale value of the current frame image is not within the set target grayscale threshold range, it is determined that the current scene needs to adjust the exposure parameter or the gain parameter.

4. The FPGA-based CMOS image sensor exposure gain method according to claim 1, characterized in that: When the exposure parameter or the gain parameter needs to be adjusted, it is first determined whether the current gain parameter is the initial value, and in the subsequent step of adjusting the exposure parameter or the gain parameter is selected according to the determination result, if the current gain parameter is the initial gain value, the exposure parameter is selected for adjustment; if the current gain parameter is not the initial gain value, it is determined that the historical frame image has been gain adjusted, and when adjusting again, the state is preferably pulled back to the gain parameter adjustment.

5. The FPGA-based CMOS image sensor exposure gain method according to claim 1, characterized in that: The step of adaptively adjusting the exposure parameter or gain parameter according to the set target grayscale threshold range uses a pre-established segmented exposure parameter adjustment model to predict the exposure parameter and average grayscale value of the target frame image, thereby achieving adaptive adjustment of the exposure parameter; The gain parameter can be adaptively adjusted by increasing or decreasing the gain value with a fixed step size.

6. The FPGA-based CMOS image sensor exposure gain method according to claim 5, characterized in that: The step of using the pre-established segmented exposure parameter adjustment model to predict the exposure parameters and average grayscale value of the target frame image to achieve adaptive adjustment of the exposure parameters includes: Under equal illumination intensity, when the grayscale value is in the middle area, the grayscale value and the exposure time are in a linear relationship. Then, according to the proportional relationship between the grayscale of the current frame and the target frame image, the exposure time of the current frame is proportionally enlarged and reduced to establish the first segmented exposure parameter adjustment model. The expression of the first segmented exposure parameter adjustment model is as follows: In the formula, dsir_expo is the target frame exposure time predicted by the model, expo_time is the exposure time corresponding to the current frame, AVER_GRAY_OK is the optimal grayscale value of the target frame, avg_gray is the average gray value of the current frame.

7. The FPGA-based CMOS image sensor exposure gain method according to claim 5, characterized in that: The step of using the pre-established segmented exposure parameter adjustment model to predict the exposure parameters and average grayscale value of the target frame image to achieve adaptive adjustment of the exposure parameters includes: When the gray value is not in the middle area, the gray value and the exposure time have a nonlinear relationship. The second segmented exposure parameter adjustment model is established by the stepwise approximation method. The expression of the second segmented exposure parameter adjustment model is as follows: In the formula, dsir_expo is the target frame exposure time predicted by the model, expo_time is the exposure time corresponding to the current frame, AVER_GRAY_OK is the optimal grayscale value of the target frame, avg_gray is the average gray value of the current frame.

8. The FPGA-based CMOS image sensor exposure gain method according to claim 5, characterized in that: In the step of adaptively adjusting the gain parameter by increasing or decreasing the gain value in a fixed step size, when the FPGA enters the gain parameter adjustment process, if it is determined that the average grayscale value of the current frame image is less than the target grayscale threshold range, the gain value is increased by configuring the value of the image register, otherwise the gain value is decreased.

9. A CMOS image sensor exposure gain system based on FPGA, characterized in that: include: The average gray value calculation module is used to collect a frame of original image data through FPGA and calculate the average gray value of the current frame image; A comparison and determination module is used to compare the average grayscale value of the current frame image with the set target grayscale threshold range, and determine whether the current scene needs to adjust the exposure parameter or gain parameter based on the comparison result; The adjustment object selection module is used to first determine whether the current gain parameter is an initial value when the exposure parameter or the gain parameter needs to be adjusted, and then select the subsequent adjustment of the exposure parameter or the gain parameter according to the judgment result; The adjustment output module is used to adaptively adjust the exposure parameter or gain parameter according to the set target grayscale threshold range, so that the average grayscale value of the current frame image reaches the set target grayscale threshold range, stop adjusting and output the current exposure parameter and gain parameter.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the FPGA-based CMOS image sensor exposure gain method according to any one of claims 1 to 8 is implemented.

Images