Automatic exposure and gain control method and apparatus for extracting physiological signals

Abstract

Disclosed in the present application are an automatic exposure and gain control method and apparatus for extracting physiological signals. The method comprises: detecting a target area; on the basis of spatiotemporal information, segmenting the target area to obtain a final target area, and calculating and obtaining the average brightness of the final target area; on the basis of the average brightness, calculating and obtaining an error; on the basis of the error, calculating and obtaining an adjustment amount for exposure time and gain; and if the error is greater than an error threshold value, then calculating and obtaining updated exposure time and gain. In the present application, physiological monitoring with a high signal-to-noise ratio is implemented by means of controlling the average brightness of the target area and synchronously adjusting the dynamic weight of the exposure time and analog gain; using dynamic step size adjustment allows image brightness to rapidly reach a balance, and physiological signals can be stably extracted under changing illumination conditions, thus effectively reducing the impact of ambient light changes, facial expressions, and bodily movement on the extraction of physiological signals, improving the accuracy and reliability of physiological parameter measurement.

Description

Automatic exposure and gain control method and device for extracting physiological signals Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an automatic exposure and gain control method and device for extracting physiological signals. Background Art

[0002] As my country gradually enters a stage of low fertility and an aging society, the health of newborns and the elderly population will become one of the most pressing social issues to be addressed. To achieve health monitoring and disease prevention for these two groups, there is an urgent need for accurate and continuous monitoring of physiological information such as heart rate, blood pressure, blood oxygen saturation, and respiratory rate. As a non-contact physiological parameter measurement method, camera-based remote photoplethysmography (RPPG) technology has attracted widespread attention in fields such as healthcare, health monitoring, psychological assessment, and human-computer interaction. When using a camera to extract physiological signals (such as pulse wave signals, respiratory signals, or cardiographic signals), controlling exposure parameters (such as exposure time and gain) plays a crucial role in ensuring image quality and accurate signal extraction.

[0003] Currently, exposure control methods for physiological signal extraction fall into two main categories: (i) using fixed exposure parameters. This method relies on pre-set empirical exposure parameters that are not adjusted once set. However, this method does not consider the impact of ambient light changes on image quality. Therefore, the accuracy and reliability of physiological signal monitoring may be affected by factors such as changes in light intensity, camera movement, changes in facial expression, and body movements. (ii) using the automatic exposure algorithm built into the camera's image signal processor (ISP). This method attempts to cope with changes in ambient light through automatic exposure algorithms. However, existing automatic exposure algorithms mainly focus on global brightness balance from the perspective of photographic aesthetics. In contrast, physiological signal extraction typically only focuses on brightness balance in the target area (such as the face, chest, abdomen, and soles of the feet). This makes it impossible for existing automatic exposure algorithms to achieve optimal image quality within the target area. Under special lighting conditions (such as backlighting), the brightness of the target area may be too bright or too dark, resulting in inaccurate physiological signal extraction. In addition, since existing automatic exposure algorithms usually use a fixed adjustment step size, under conditions of rapidly changing lighting (such as driver physiological monitoring), the algorithm's convergence speed may be too slow to achieve real-time stability, which may cause serious interference to the extraction and analysis of physiological signals.

[0004] Therefore, the existing technology needs to be improved and enhanced.

[0005] Summary of the Invention

[0006] The technical problem to be solved by this application is that the brightness adjustment of the target area is difficult in the existing technology, and the quality of the target area cannot be dynamically adjusted. In view of the shortcomings of the existing technology, an automatic exposure and gain control method and device for extracting physiological signals are provided.

[0007] In order to solve the above technical problems, a first aspect of an embodiment of the present application provides an automatic exposure and gain control method for extracting physiological signals, the automatic exposure and gain control method for extracting physiological signals comprising:

[0008] A method for automatically controlling exposure and gain of physiological signals, comprising:

[0009] Detect target area;

[0010] Segmenting the target area based on spatiotemporal information to obtain a final target area, and calculating and obtaining an average brightness of the final target area;

[0011] Calculate and obtain an error based on the average brightness;

[0012] Calculating and obtaining adjustment amounts of exposure time and gain based on the error;

[0013] If the error is greater than the error threshold, then the updated exposure time and gain are calculated and obtained.

[0014] Optionally, in the automatic exposure and gain control method for extracting physiological signals, the step of detecting the target area includes:

[0015] intercepting the target area;

[0016] If the target area is not captured, the global average brightness is calculated as the average brightness;

[0017] If the target area is captured, segmenting the target area based on the spatiotemporal information;

[0018] The segmenting the target area based on the spatiotemporal information to obtain the final target area, and calculating and obtaining the average brightness of the final target area include:

[0019] Extracting a plurality of image frames at consecutive times from the target area, and obtaining each pixel value and spatial coordinate of the image frames as features of the image frames;

[0020] Inputting the features into a clustering algorithm to obtain a series of superpixels;

[0021] Calculating the overlap rate of the superpixels and the target area, extracting superpixels with an overlap rate greater than an overlap threshold, and calculating and arranging the signal-to-noise ratios of the superpixels with an overlap rate greater than the overlap threshold in order of size;

[0022] Extracting a preset proportion of superpixels with the highest signal-to-noise ratio and determining them as the final target area;

[0023] Based on the final target area, an average brightness of the target area is calculated.

[0024] Optionally, the automatic exposure and gain control method for extracting physiological signals includes the following steps: presetting a target brightness, and calculating the difference between the average brightness and the target brightness as an error.

[0025] Optionally, before the step of calculating and obtaining the adjustment amount of exposure time and gain based on the error, the step further includes:

[0026] Performing dynamic weight adjustment based on the error;

[0027] When the error is greater than a preset error threshold, increasing the exposure time weight;

[0028] When the error is less than or equal to the error threshold and less than or equal to the negative value of the error threshold, increasing the gain weight;

[0029] When the error is less than or equal to the error threshold and greater than a negative value of the error threshold, calculating an average gradient of nearly N frames of the current frame;

[0030] When the average gradient is greater than a preset average gradient threshold, increasing the exposure time weight;

[0031] When the average gradient is less than or equal to a preset average gradient threshold, determining whether the average gradient is greater than a negative value of the average gradient threshold;

[0032] When the average gradient is greater than the negative value of the average gradient threshold, the gain weight is increased until the average gradient is less than or equal to the negative value of the average gradient threshold.

[0033] Optionally, the step of calculating the exposure time and the gain adjustment amount includes:

[0034] The error is divided into current frame error and historical frame error according to the acquisition time;

[0035] Calculating the sum of the current frame error and the historical frame error to obtain a cumulative error;

[0036] The current frame error, the accumulated error, and the error change rate are linearly combined according to preset parameters to calculate the output of the PID controller;

[0037] Get the exposure time and gain of the previous frame and the maximum exposure time;

[0038] The function of the relationship between gain and brightness and the function of the relationship between exposure time and brightness are obtained through experiments;

[0039] Fitting the function of the relationship between gain and brightness and the function of the relationship between exposure time and brightness into a mapping function through experiments;

[0040] The exposure time and gain adjustment amount are calculated based on the exposure time and gain of the previous frame, the maximum exposure time, the output of the PID controller and the mapping function.

[0041] Optionally, the calculation obtains the output of the PID controller and is performed according to the following formula: Δ=kp*error+Ki*∑(error)+Kd*(error-last_error)

[0042] Wherein, Δ is the output of the PID controller, Kp, Ki and Kd are the parameters of the PID controller, ∑(error) is the accumulated error, and last_error is the historical error of the previous frame.

[0043] Optionally, the step of calculating and obtaining the updated exposure time and gain includes:

[0044] Obtaining the current exposure time, and determining whether the current exposure time is less than the maximum exposure time;

[0045] If yes, calculate and obtain an updated exposure time and gain, where the updated exposure time is the sum of the exposure time of the previous frame and the exposure time adjustment amount;

[0046] The updated gain is the sum of the previous frame gain and the gain adjustment amount.

[0047] Optionally, the method further includes:

[0048] When the current exposure time is greater than or equal to the maximum exposure time, calculating the absolute value of the error based on the updated exposure time and gain;

[0049] If the absolute value of the error is greater than a preset adjustment threshold, applying the updated exposure time and gain adjustment amount to the image frame to obtain an adjusted image frame;

[0050] If the absolute value of the error is less than or equal to the adjustment threshold, the process returns to the step of detecting the target area.

[0051] Optionally, the method further includes:

[0052] A physiological signal of the adjusted image frame is extracted, and a physiological parameter is calculated based on the physiological signal.

[0053] In addition, to achieve the above object, the present invention further provides a device, wherein the device comprises:

[0054] An image acquisition module, used for acquiring continuous image frames containing human skin;

[0055] Target detection module, used to detect target areas;

[0056] A target area segmentation and brightness calculation module is used to segment the target area based on spatiotemporal information to obtain a final target area, and calculate and obtain the average brightness of the final target area;

[0057] an exposure-analog gain control module, configured to calculate and obtain an error based on the average brightness, perform dynamic weight adjustment, calculate and obtain exposure time and gain adjustment amounts based on the error, and if the error is greater than an error threshold, calculate and obtain updated exposure time and gain, and adjust the image frame;

[0058] The physiological signal extraction module is used to extract the physiological signal from the adjusted image frame.

[0059] As can be seen from the above, the present invention discloses an automatic exposure and gain control method and device for extracting physiological signals, including: initializing the system, detecting the target area, and intercepting the target area; segmenting the target area based on spatiotemporal information to obtain the final target area, calculating and obtaining the average brightness of the final target area; calculating and obtaining the error based on the average brightness, and performing dynamic weight adjustment; calculating and obtaining the adjustment amount of exposure time and gain based on the error; if the error is greater than the error threshold, then calculating and obtaining the updated exposure time and gain. The present invention controls the average brightness of the target area in real time, synchronously adjusts the exposure time and analog gain, and reasonably allocates the ratio between the two to achieve physiological monitoring with a high signal-to-noise ratio. In addition, the present invention also comprehensively considers the current and past lighting conditions, and adopts dynamic step size adjustment to achieve rapid equilibrium of image brightness. Physiological signals can be stably extracted under different lighting conditions, effectively reducing the influence of factors such as ambient light changes, facial expressions and body movements on physiological signal extraction, and improving the accuracy and reliability of physiological parameter measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without inventive work.

[0061] FIG1 is a flow chart of the automatic exposure and gain control method for extracting physiological signals provided in the present application.

[0062] FIG2 is a flow chart of dynamic weight adjustment provided by this application.

[0063] FIG3 is a structural principle diagram of the automatic exposure and gain control device for extracting physiological signals provided in this application. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solutions and advantages of the present invention more clear and distinct, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0065] The automatic exposure and gain control method for extracting physiological signals according to a preferred embodiment of the present invention is shown in FIG1 . The automatic exposure and gain control method for extracting physiological signals comprises the following steps:

[0066] Step S100: Detect the target area.

[0067] Specifically, before detecting the target area, the system is initialized. This initialization may include setting PID parameters (Proportional, Integral, Differential), presetting target brightness, initial exposure time, and gain weights. The weights can be set to an initial ratio, such as 50 (exposure time):50 (gain), to provide a balanced starting point for subsequent adjustment. This initial ratio can also be adjusted to other ratios, such as 30 (gain):70 (exposure). Detecting the target area can be performed using target detection technologies such as face detection and deep learning. The captured target area may include a region of interest (ROI) and a non-region of interest (NRI). For example, if the target area to be captured is the skin area of ​​a face, the target area can be a box provided by a face detection algorithm, containing both skin and non-skin areas (e.g., background). In this case, the ROI is the facial skin, and the NRI within the target area is the non-skin area within the face frame. The process of detecting the target area and intercepting the target area is a known implementation technology in the prior art and will not be described in detail here.

[0068] Step S200: Segment the target area based on the spatiotemporal information to obtain a final target area, and calculate and obtain the average brightness of the final target area.

[0069] Specifically, the target area is captured using existing target detection technology. This embodiment uses face detection technology to detect and capture the target area. If the target area is not captured, it is determined to be overexposed or underexposed, that is, the difference between the current average brightness and the preset target brightness is too large. The global average brightness is then calculated as the average brightness, which is used to further dynamically adjust the exposure time and gain. If the target area is captured, the captured area is preliminarily determined to be a preliminary target area, and the target area is segmented based on spatiotemporal information.

[0070] After determining the preliminary target area, the target area is segmented using a target area segmentation method with dynamic change characteristics, and the region of interest (RoI) and the non-region of interest therein are segmented for subsequent further processing. The segmentation method can be based on time information and spatial information to segment the target area. If the information of the current frame and the previous and next frames is considered, the time information is the value of the pixel at the same position in different frames, and the spatial information is the value of the pixel at different positions in the same frame, that is, within the same time. For example, for each pixel, its pixel intensity is the (R, G, B) value, the spatial information is the (x, y) coordinate, and the time information can be the t-th frame, or the time sliding window. If the 10 frames before and after are considered, the sliding window length is set to 20, and the current frame is the t-th frame in the sliding window.

[0071] Furthermore, several consecutive image frames are extracted from the target area, and the pixel values ​​and spatial coordinates of each of the image frames are obtained as features of the image frames. These features are input into a clustering algorithm to generate a series of superpixels. The overlap ratio between the superpixels and the target area is calculated, and superpixels with an overlap ratio greater than an overlap threshold are extracted. The signal-to-noise ratio of the superpixels with an overlap ratio greater than the overlap threshold is calculated and ranked by size. The superpixels with the highest signal-to-noise ratio are extracted, and a predetermined proportion of the superpixels are determined as the final target area.

[0072] For example, the overlap ratio between the superpixels and the target area can be calculated, and superpixels with an overlap ratio greater than 0.8 can be extracted. The signal-to-noise ratios of the superpixels with an overlap ratio greater than 0.8 can then be calculated and ranked by size. The top 50% of the superpixels with the highest signal-to-noise ratios are extracted and determined as the final target area. Based on the final target area, the average brightness of the final target area can be calculated.

[0073] Step S300: Calculate and obtain an error based on the average brightness.

[0074] Please refer to FIG. 2 . Specifically, a target brightness is preset, and the difference between the average brightness and the target brightness is calculated as the error.

[0075] The error can be calculated using the following formula: oeeor = target_brightness - current_brightness

[0076] Among them, error is the error, target_brightness is the preset target average brightness, and current_brightness is the average brightness of the target area in the current frame.

[0077] Specifically, before the step of calculating and obtaining the adjustment amount of exposure time and gain based on the error, the method further includes: performing dynamic weight adjustment, and the step of dynamic weight adjustment includes:

[0078] Preset an error threshold and an average gradient threshold;

[0079] First, determine whether the error is greater than the error threshold;

[0080] If so, increase the exposure time weight;

[0081] If not, further determine whether the error is less than or equal to the negative value of the error threshold: if so, increase the gain weight; if not, calculate the average gradient of the current frame over nearly N frames;

[0082] Secondly, determining whether the average gradient is greater than the average gradient threshold;

[0083] If so, increase the exposure time weight;

[0084] If not, the system further determines whether the average gradient is greater than the negative of the average gradient threshold. If so, the gain weight is increased; if not, the exposure time and gain adjustments are calculated. This step analyzes the error trend over several consecutive frames. If the error continues to increase, the exposure time weight is increased; if the error continues to decrease, the gain weight is increased. This allows for more sensitive response to changes in ambient brightness, enabling dynamic exposure time and gain weight adjustment, thereby achieving the target brightness more quickly.

[0085] Please refer to FIG2 , for example, an error threshold is preset to 40, and an average gradient threshold is preset to 10;

[0086] First, determine whether the error is greater than 40;

[0087] If so, increase the exposure time weight;

[0088] If not, further determining whether the error is less than or equal to -40;

[0089] If yes, increase the gain weight, if no, calculate the average gradient of the current frame over the past five frames;

[0090] Secondly, determine whether the average gradient is greater than 10;

[0091] If so, increase the exposure time weight;

[0092] If not, further determine whether the average gradient is greater than -10;

[0093] If so, increase the gain weight;

[0094] If not, the exposure time and gain adjustment are calculated. Dynamic weight adjustment, which updates the exposure time and gain weights in real time, comprehensively accounts for dynamic lighting changes. Dynamic step size adjustment is used to quickly achieve image brightness equilibrium. This allows for stable physiological signal extraction under varying lighting conditions, effectively reducing the impact of ambient light changes, facial expressions, and body movements on physiological signal extraction, and improving the accuracy and reliability of physiological parameter measurements.

[0095] Step S400: Calculate and obtain adjustment amounts of exposure time and gain based on the error.

[0096] Specifically, the error is divided into current frame error and historical frame error according to the acquisition time;

[0097] Calculating the sum of the current frame error and the historical frame error to obtain a cumulative error ∑(error);

[0098] The current frame error, the accumulated error, and the error change rate are linearly combined according to preset parameters to obtain the output of the PID controller, and the calculation is performed according to the following formula: Δ=Kp*error+Ki*∑(error)+Kd*(error-last_error)

[0099] Wherein, Δ is the output of the PID controller, Kp (proportional adjustment coefficient), Ki (integral adjustment coefficient) and Kd (differential adjustment coefficient), ∑(error) is the cumulative error, and last_error is the historical error of the previous frame.

[0100] Specifically, the step of calculating the exposure time and the gain adjustment amount includes:

[0101] The exposure time exp_old, gain gain_old, and maximum exposure time exp_max of the previous frame are obtained. A function representing the relationship between gain and brightness, g(x), and a function representing the relationship between exposure time and brightness, h(x), are experimentally obtained. These functions are experimentally fitted into a mapping function, fun_exp2gain. Based on the exposure time exp_old and gain gain_old of the previous frame, the maximum exposure time exp_max, the output Δ of the PID controller, and the mapping function fun_exp2gain, the exposure time and gain adjustment are calculated.

[0102] Furthermore, the function g(x) of the relationship between gain and brightness, the function h(x) of the relationship between exposure time and brightness, and the mapping function fun_exp2gain can be obtained through the following method:

[0103] (1) Select the brightness range and interval:

[0104] Select a suitable brightness range, such as 0 - 255.

[0105] Select a suitable interval, such as every 10 brightness units as a recording point.

[0106] (2) Fix the exposure and record the gain:

[0107] Fix the exposure value, adjust the gain at each recording point until the desired brightness is reached, and record the gain value at each recording point.

[0108] (3) Fix the gain and record the exposure:

[0109] Fix the gain value, adjust the exposure at each recording point until the desired brightness is reached, and record the exposure value at each recording point.

[0110] (4) Data analysis and fitting:

[0111] Use polynomial fitting or machine learning methods (such as support vector machines, neural networks, etc.) to fit the data, and obtain the function g(x) describing the relationship between gain and brightness and the function h(x) describing the relationship between exposure and brightness.

[0112] (5) Verification and optimization:

[0113] Use an independent validation dataset to test the accuracy and robustness of the fitting function. Further, the fitting function can be optimized as needed. For example, by selecting different fitting methods, adjusting parameters, etc.

[0114] ((6) Implement the mapping function:

[0115] Combine the fitting functions g(x) and h(x) into a mapping function f(x) for real-time adjustment of exposure and gain.

[0116] Further, the specific steps for calculating the exposure time adjustment amount Δexp and the gain adjustment amount Δgain based on the exposure time and gain of the previous frame, the maximum exposure time, the output of the PID controller, and the mapping function fun_exp2gain are as follows:

[0117] If exp_old < exp_max and exp_old < exp_max

[0118] Then

[0119] If exp + Δ > exp_max and exp_old > exp_max

[0120] but

[0121] When Δ>0, if exp_old>Δ and exp_old <exp_max

[0122] but

[0123] When Δ<0, if fun_exp2gain(Δ)<0 and gain_old>0

[0124] but

[0125] This algorithm converts exposure time changes into corresponding gain changes, ensuring that changes in exposure time and gain have equal impact on image brightness. This method can effectively mitigate or even eliminate the shock that occurs when adjusting gain after adjusting the exposure time to the maximum.

[0126] Step S500: If the error is greater than the error threshold, then calculate and obtain the updated exposure time and gain.

[0127] Specifically, obtaining a current exposure time, and determining whether the current exposure time is less than the maximum exposure time;

[0128] If not, then adjust the threshold judgment;

[0129] If so, calculate and obtain an updated exposure time and gain, wherein the updated exposure time is the sum of the exposure time of the previous frame and the exposure time adjustment amount;

[0130] The updated gain is the sum of the previous frame gain and the gain adjustment amount.

[0131] The adjustment threshold determination includes:

[0132] Presetting the adjustment threshold;

[0133] Calculating the absolute value of the error based on the updated exposure time and gain, and determining whether the absolute value of the error is greater than the adjustment threshold;

[0134] If so, applying the updated exposure time and gain adjustment to the image frame;

[0135] If not, return to step S100.

[0136] Furthermore, the physiological signals of the adjusted image frames are extracted and the physiological parameters are further calculated.

[0137] Specifically, image processing technology is used to extract physiological signals and further calculate physiological parameters such as heart rate, blood pressure, blood oxygen, and respiratory rate.

[0138] It should be noted that the automatic exposure and gain control method for extracting physiological signals based on this embodiment can adapt to changing ambient light conditions. Specifically, all of the aforementioned thresholds can be pre-set and adjusted based on actual needs, thereby achieving sensitive and stable dynamic weight adjustment to better achieve the target brightness in the target area.

[0139] In summary, the method of the present invention includes initializing the system, detecting the target area, and intercepting the target area; segmenting the target area based on spatiotemporal information to obtain the final target area, calculating and obtaining the average brightness of the final target area; calculating and obtaining the error based on the average brightness, and performing dynamic weight adjustment; calculating and obtaining the adjustment amount of exposure time and gain based on the error; if the error is greater than the error threshold, then calculating and obtaining the updated exposure time and gain. The present invention controls the average brightness of the target area in real time, synchronously adjusts the exposure time and analog gain, and reasonably allocates the ratio between the two to achieve physiological monitoring with a high signal-to-noise ratio. In addition, the present invention also comprehensively considers the current and past lighting conditions, adopts dynamic step size adjustment to achieve rapid equilibrium of image brightness, and can stably extract physiological signals under different lighting conditions, effectively reducing the influence of factors such as ambient light changes, facial expressions and body movements on physiological signal extraction, and improving the accuracy and reliability of physiological parameter measurement.

[0140] Based on the above-mentioned automatic exposure and gain control method for extracting physiological signals, this embodiment provides an automatic exposure and gain control device for extracting physiological signals, as shown in FIG3 , the device comprising:

[0141] The image acquisition module 61 is used to acquire continuous image frames containing human skin.

[0142] Specifically, the image acquisition module 61 may include:

[0143] a. The camera sensor unit can be either a wide-band single-channel (monochrome) or a narrow-band multi-channel (multi-wavelength), including monitoring of both the RGB band (such as 450-650nm) and the NIR band (such as 760-940nm). The camera sensor unit may also include: an infrared filter (IR-CUT), which is installed at the camera sensor end and can be popped out and retracted. It automatically pops out or retracts according to the lighting conditions. For example, the infrared filter can pop out when there is sufficient natural light to filter out the red light in the natural light; when there is insufficient natural light, the infrared filter automatically retracts to allow red light to enter, thereby achieving continuous monitoring during the day and at night. The camera sensor unit can effectively improve image quality and enhance the stability and reliability of the system.

[0144] b. A light source emitting unit, used to provide continuous and stable illumination. For example, when natural light is insufficient, an infrared light source (such as an 850nm SMD LED) can be used to provide continuous and stable illumination. If the ambient light is sufficient, the light source emitting unit can be automatically shut down. Alternatively, the use of different light sources, such as blue light and ultraviolet light, can also be considered to provide more information for physiological signal detection.

[0145] The target detection module 62 is used to detect the target area.

[0146] Specifically, a target region is detected from the continuous image frames containing human skin. The target region may be the face, chest, abdomen, or soles of the feet of human skin tissue, and the target region is cut out. If the target region is not cut out, the global average brightness is calculated as the average brightness. If the target region is cut out, the target region is segmented based on spatiotemporal information.

[0147] The target region segmentation and brightness calculation module 63 is configured to segment the target region based on the spatiotemporal information to obtain a final target region, and calculate and obtain an average brightness of the final target region.

[0148] Specifically, a target region segmentation method based on dynamic change characteristics is used to segment the final target region and calculate the average brightness for subsequent calculations.

[0149] The exposure-analog gain control module 64 is used to calculate and obtain an error based on the average brightness, perform dynamic weight adjustment, and calculate and obtain the adjustment amount of exposure time and gain based on the error. If the error is greater than the error threshold, then calculate and obtain the updated exposure time and gain to adjust the image frame.

[0150] Specifically, the exposure-analog gain control module 64 can obtain parameters such as the current error, cumulative error, and error change rate based on the average brightness of the target area, perform linear combination calculations on them, and obtain updated exposure time and gain. Furthermore, the image signal processing unit (ISP) can be called to apply the calculated updated exposure time and gain to the next frame image.

[0151] Physiological signal extraction module 65 is used to extract physiological signals from the adjusted image frames. Specifically, image processing techniques can be used to extract physiological signals and further calculate physiological parameters such as heart rate, blood pressure, blood oxygen, and respiration. Furthermore, the video or image frames acquired in real time by physiological signal extraction module 65, as well as the monitoring results obtained, can be displayed on hardware such as a user interface.

[0152] Specifically, the target detection module 62, the target area segmentation and brightness calculation module 63, the exposure-analog gain control module 64 and the physiological signal extraction module 65 are virtual modules, which include a set of software programs that implement the automatic exposure and gain control method for extracting physiological signals, and are used for real-time analysis, adjustment, and processing of the continuous image frames containing human skin obtained from the image acquisition module 61.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An automatic exposure and gain control method for extracting physiological signals, characterized in that: The method comprises: Detect target area; Segmenting the target area based on the spatiotemporal information to obtain a final target area, and calculating and obtaining an average brightness of the final target area; Calculate and obtain an error based on the average brightness; Calculating and obtaining exposure time and gain adjustment amounts based on the error; If the error is greater than the error threshold, then the updated exposure time and gain are calculated and obtained.

2. The automatic exposure and gain control method for extracting physiological signals according to claim 1, characterized in that: The step of detecting the target area comprises: intercepting the target area; If the target area is not captured, the global average brightness is calculated as the average brightness; If the target area is captured, segmenting the target area based on the spatiotemporal information; The steps of segmenting the target area based on the spatiotemporal information to obtain the final target area, and calculating and obtaining the average brightness of the final target area include: Extracting a number of image frames at consecutive times from the target area, and obtaining each pixel value and spatial coordinate of the image frame as a feature of the image frame; Inputting the features into a clustering algorithm to obtain a series of superpixels; Calculating the overlap rate of the superpixel and the target area, extracting the superpixels whose overlap rate is greater than the overlap threshold, and calculating and arranging the signal-to-noise ratios of the superpixels whose overlap rate is greater than the overlap threshold in order of size; Extracting a preset proportion of super pixels with the highest signal-to-noise ratio and determining them as the final target area; Based on the final target area, an average brightness of the target area is calculated.

3. The automatic exposure and gain control method for extracting physiological signals according to claim 1, characterized in that: The step of calculating and obtaining the error based on the average brightness comprises: The difference between the average brightness and the preset target brightness is calculated as an error.

4. The automatic exposure and gain control method for extracting physiological signals according to claim 1, characterized in that: The step of calculating and obtaining the adjustment amount of exposure time and gain based on the error also includes: Performing dynamic weight adjustment based on the error, the dynamic weight adjustment comprising: When the error is greater than a preset error threshold, increasing the exposure time weight; When the error is less than or equal to the error threshold and less than or equal to the negative value of the error threshold, increasing the gain weight; When the error is less than or equal to the error threshold and greater than a negative value of the error threshold, calculating an average gradient of nearly N frames of the current frame; When the average gradient is greater than a preset average gradient threshold, increasing the exposure time weight; When the average gradient is less than or equal to a preset average gradient threshold, determining whether the average gradient is greater than a negative value of the average gradient threshold; When the average gradient is greater than the negative value of the average gradient threshold, the gain weight is increased until the average gradient is less than or equal to the negative value of the average gradient threshold.

5. The automatic exposure and gain control method for extracting physiological signals according to claim 1, characterized in that: The step of calculating and obtaining the adjustment amount of exposure time and gain based on the error comprises: The error is divided into a current frame error and a historical frame error according to the acquisition time; Calculating the sum of the current frame error and the historical frame error to obtain a cumulative error; The current frame error, the accumulated error, and the error change rate are linearly combined according to preset parameters to calculate the output of the PID controller; Get the exposure time and gain of the previous frame and the maximum exposure time; The function of the relationship between gain and brightness and the function of the relationship between exposure time and brightness are obtained through experiments; Fitting the function of the relationship between gain and brightness and the function of the relationship between exposure time and brightness into a mapping function through experiments; The exposure time and gain adjustment amount are obtained by calculation based on the exposure time and gain of the previous frame, the maximum exposure time, the output of the PID controller and the mapping function.

6. The automatic exposure and gain control method for extracting physiological signals according to claim 5, characterized in that: The output of the PID controller is obtained by calculation according to the following formula: Δ = Kp*error+Ki*∑(error)+Kd*(error-last_error) Wherein, Δ is the output of the PID controller, Kp, Ki and Kd are the parameters of the PID controller, ∑(error) is the accumulated error, and last_error is the historical error of the previous frame.

7. The automatic exposure and gain control method for extracting physiological signals according to claim 6, characterized in that: The step of calculating and obtaining the updated exposure time and gain comprises: Obtaining a current exposure time, and determining whether the current exposure time is less than the maximum exposure time; If yes, then calculate and obtain an updated exposure time and gain, wherein the updated exposure time is the sum of the exposure time of the previous frame and the exposure time adjustment amount; The updated gain is the sum of the previous frame gain and the gain adjustment amount.

8. The automatic exposure and gain control method for extracting physiological signals according to claim 7, characterized in that: The method further comprises: When the current exposure time is greater than or equal to the maximum exposure time, calculating the absolute value of the error based on the updated exposure time and gain; If the absolute value of the error is greater than a preset adjustment threshold, applying the updated exposure time and gain adjustment amount to the image frame to obtain an adjusted image frame; If the absolute value of the error is less than or equal to the adjustment threshold, the process returns to the step of detecting the target area.

9. The automatic exposure and gain control method for extracting physiological signals according to claim 8, characterized in that: The method further comprises: A physiological signal of the adjusted image frame is extracted, and a physiological parameter is calculated based on the physiological signal.

10. An automatic exposure and gain control device for extracting physiological signals, characterized in that: The device comprises: An image acquisition module, used for acquiring continuous image frames containing human skin; Target detection module, used to detect target area; A target area segmentation and brightness calculation module is used to segment the target area based on the spatiotemporal information to obtain the final target area, and calculate and obtain the average brightness of the final target area; An exposure-analog gain control module, used to calculate and obtain an error based on the average brightness, perform dynamic weight adjustment, calculate and obtain an adjustment amount of exposure time and gain based on the error, and if the error is greater than an error threshold, calculate and obtain an updated exposure time and gain, and adjust the image frame; The physiological signal extraction module is used to extract the physiological signal in the adjusted image frame.

Images