Automatic exposure and gain control method and device for extracting physiological signals
By detecting and segmenting the target area, dynamically adjusting the exposure time and gain, the problem of difficulty in adjusting the brightness of the target area in the prior art is solved, and stable physiological signal extraction under different lighting conditions is achieved, and the accuracy of physiological parameter measurement is improved. and reliability.
Patent Information
- Application Number
- CN202311221793.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-20
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2043-09-20
AI Technical Summary
In the prior art, the brightness adjustment of the target area is difficult and the quality of the target area cannot be dynamically adjusted. Especially under the influence of factors such as light changes and facial expressions, the accuracy and reliability of physiological signal extraction are affected.
By detecting the target area, segmenting based on space-time information, calculating the average brightness and error, dynamically adjusting the exposure time and gain, and real-time adjustment of exposure time and gain is achieved by using PID controller and mapping function to ensure the brightness balance of the target area.
It realizes stable extraction of physiological signals under different lighting conditions, reduces the impact of ambient light changes and facial expressions on physiological signal extraction, and improves the accuracy and reliability of physiological parameter measurements.
Smart Images

Figure CN117459833B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to an automatic exposure and gain control method and device for extracting physiological signals. Background Art
[0002] As my country gradually enters the stage of low fertility and aging society, the health problems of newborns and the elderly will become one of the social problems that need to be solved urgently. In order to achieve health monitoring and disease prevention for these two groups, it is urgent to monitor physiological information such as heart rate, blood pressure, blood oxygen saturation, and respiratory rate accurately and continuously. As a non-contact physiological parameter measurement method, camera-based remote pulse wave signal monitoring (Remote Photoplethysmography, RPPG) technology has received widespread attention in the fields of medical care, health monitoring, psychological assessment, and human-computer interaction. In the process of using a camera to extract physiological signals (such as pulse wave signals, respiratory signals, or cardiography signals, etc.), the control of exposure parameters (such as exposure time and gain) plays a vital role in ensuring image quality and accurate signal extraction.
[0003] At present, the exposure control methods for physiological signal extraction can be divided into two categories: (i) using fixed exposure parameters. This method relies on pre-set empirical exposure parameters, which are no longer adjusted once set. However, this method does not take into account the impact of ambient light changes on image quality. Therefore, the accuracy and reliability of physiological signal monitoring may be affected by factors such as changes in light intensity, camera movement, changes in facial expressions, and body movements. (ii) using the automatic exposure algorithm built into the camera image processing unit (ISP). This method attempts to cope with changes in ambient light through automatic exposure algorithms. However, existing automatic exposure algorithms mainly focus on the balance of global brightness from the perspective of photographic aesthetics. The extraction of physiological signals usually only focuses on the brightness balance of the target area (such as the face, chest and abdomen, soles of the feet, etc.). This makes it impossible for existing automatic exposure algorithms to achieve the optimal quality of the image in the target area. Under special lighting conditions (such as backlight), the brightness of the target area may be too bright or too dark, resulting in the inability to accurately extract physiological signals. In addition, since existing automatic exposure algorithms usually use a fixed adjustment step size, under conditions of rapidly changing lighting (such as physiological monitoring of drivers), the convergence speed of the algorithm may be too slow to achieve real-time stability, which may cause serious interference to the extraction and analysis of physiological signals.
[0004] Therefore, the existing technology still needs to be improved and enhanced. Summary of the invention
[0005] The technical problem to be solved by the present application is that it is difficult to adjust the brightness of the target area in the prior art, and the quality of the target area cannot be dynamically adjusted. In view of the shortcomings of the prior art, an automatic exposure and gain control method and device for extracting physiological signals are provided.
[0006] In order to solve the above technical problems, a first aspect of an embodiment of the present application provides an automatic exposure and gain control method for extracting physiological signals, and the automatic exposure and gain control method for extracting physiological signals includes:
[0007] An automatic exposure and gain control method for extracting physiological signals, the automatic exposure and gain control method for extracting physiological signals comprising:
[0008] Detect target area;
[0009] Segmenting the target area based on the spatiotemporal information to obtain a final target area, and calculating and obtaining an average brightness of the final target area;
[0010] Calculate and obtain an error based on the average brightness;
[0011] Calculating and obtaining exposure time and gain adjustment amounts based on the error;
[0012] If the error is greater than the error threshold, then the updated exposure time and gain are calculated and obtained.
[0013] Optionally, in the automatic exposure and gain control method for extracting physiological signals, the step of detecting the target area includes:
[0014] intercepting the target area;
[0015] If the target area is not captured, the global average brightness is calculated as the average brightness;
[0016] If the target area is captured, segmenting the target area based on the spatiotemporal information;
[0017] The segmenting the target area based on the spatiotemporal information to obtain the final target area, and calculating and obtaining the average brightness of the final target area comprises:
[0018] Extracting a number of image frames at consecutive times from the target area, and obtaining each pixel value and spatial coordinate of the image frame as a feature of the image frame;
[0019] Inputting the features into a clustering algorithm to obtain a series of superpixels;
[0020] Calculating the overlap rate of the superpixel and the target area, extracting the superpixels whose overlap rate is greater than the overlap threshold, and calculating and arranging the signal-to-noise ratios of the superpixels whose overlap rate is greater than the overlap threshold in order of size;
[0021] Extracting a preset proportion of super pixels with the highest signal-to-noise ratio and determining them as the final target area;
[0022] Based on the final target area, an average brightness of the target area is calculated.
[0023] Optionally, the automatic exposure and gain control method for extracting physiological signals includes the following steps: presetting a target brightness, and calculating the difference between the average brightness and the target brightness as an error.
[0024] Optionally, before the step of calculating and obtaining the adjustment amount of exposure time and gain based on the error, the step further includes:
[0025] Performing dynamic weight adjustment based on the error;
[0026] When the error is greater than a preset error threshold, increasing the exposure time weight;
[0027] When the error is less than or equal to the error threshold and less than or equal to the negative value of the error threshold, increasing the gain weight;
[0028] When the error is less than or equal to the error threshold and greater than a negative value of the error threshold, calculating an average gradient of nearly N frames of the current frame;
[0029] When the average gradient is greater than a preset average gradient threshold, increasing the exposure time weight;
[0030] When the average gradient is less than or equal to a preset average gradient threshold, determining whether the average gradient is greater than a negative value of the average gradient threshold;
[0031] When the average gradient is greater than the negative value of the average gradient threshold, the gain weight is increased until the average gradient is less than or equal to the negative value of the average gradient threshold.
[0032] Optionally, the step of calculating the exposure time and the gain adjustment amount includes:
[0033] The error is divided into a current frame error and a historical frame error according to the acquisition time;
[0034] Calculating the sum of the current frame error and the historical frame error to obtain a cumulative error;
[0035] The current frame error, the accumulated error, and the error change rate are linearly combined according to preset parameters to calculate the output of the PID controller;
[0036] Get the exposure time and gain of the previous frame and the maximum exposure time;
[0037] The function of the relationship between gain and brightness and the function of the relationship between exposure time and brightness are obtained through experiments;
[0038] Fitting the function of the relationship between gain and brightness and the function of the relationship between exposure time and brightness into a mapping function through experiments;
[0039] The exposure time and gain adjustment amount are obtained by calculation based on the exposure time and gain of the previous frame, the maximum exposure time, the output of the PID controller and the mapping function.
[0040] Optionally, the calculation obtains the output of the PID controller and performs the operation according to the following formula:
[0041] Δ=Kp*error+Ki*∑(error)+Kd*(error-last_error)
[0042] Wherein, Δ is the output of the PID controller, Kp, Ki and Kd are the parameters of the PID controller, ∑(error) is the accumulated error, and last_error is the historical error of the previous frame.
[0043] Optionally, the step of calculating and obtaining the updated exposure time and gain includes:
[0044] Obtaining a current exposure time, and determining whether the current exposure time is less than the maximum exposure time;
[0045] If yes, then calculate and obtain an updated exposure time and gain, wherein the updated exposure time is the sum of the exposure time of the previous frame and the exposure time adjustment amount;
[0046] The updated gain is the sum of the previous frame gain and the gain adjustment amount.
[0047] Optionally, the method further comprises:
[0048] When the current exposure time is greater than or equal to the maximum exposure time, calculating the absolute value of the error based on the updated exposure time and gain;
[0049] If the absolute value of the error is greater than a preset adjustment threshold, applying the updated exposure time and gain adjustment amount to the image frame to obtain an adjusted image frame;
[0050] If the absolute value of the error is less than or equal to the adjustment threshold, the process returns to the step of detecting the target area.
[0051] Optionally, the method further comprises:
[0052] A physiological signal of the adjusted image frame is extracted, and a physiological parameter is calculated based on the physiological signal.
[0053] In addition, to achieve the above object, the present invention further provides a device, wherein the device comprises:
[0054] An image acquisition module, used for acquiring continuous image frames containing human skin;
[0055] Target detection module, used to detect target area;
[0056] A target area segmentation and brightness calculation module is used to segment the target area based on the spatiotemporal information to obtain the final target area, and calculate and obtain the average brightness of the final target area;
[0057] An exposure-analog gain control module, used to calculate and obtain an error based on the average brightness, perform dynamic weight adjustment, calculate and obtain an adjustment amount of exposure time and gain based on the error, and if the error is greater than an error threshold, calculate and obtain an updated exposure time and gain, and adjust the image frame;
[0058] The physiological signal extraction module is used to extract the physiological signal in the adjusted image frame.
[0059] As can be seen from the above, in the scheme of the present invention, an automatic exposure and gain control method and device for extracting physiological signals are disclosed, including: initializing the system, detecting the target area, and intercepting the target area; segmenting the target area based on spatiotemporal information to obtain the final target area, calculating and obtaining the average brightness of the final target area; calculating and obtaining the error based on the average brightness, and performing dynamic weight adjustment; calculating and obtaining the adjustment amount of exposure time and gain based on the error; if the error is greater than the error threshold, then calculating and obtaining the updated exposure time and gain. The present invention controls the average brightness of the target area in real time, synchronously adjusts the exposure time and analog gain, and reasonably allocates the ratio between the two to achieve physiological monitoring with a high signal-to-noise ratio. In addition, the present invention also comprehensively considers the current and past lighting conditions, and adopts dynamic step size adjustment to achieve rapid balance of image brightness. Physiological signals can be stably extracted under different lighting conditions, effectively reducing the influence of factors such as ambient light changes, facial expressions and body movements on physiological signal extraction, and improving the accuracy and reliability of physiological parameter measurement. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0061] Figure 1 This is a flow chart of the automatic exposure and gain control method for extracting physiological signals provided in this application.
[0062] Figure 2 A flow chart for dynamic weight adjustment provided for this application.
[0063] Figure 3 This is a structural principle diagram of the automatic exposure and gain control device for extracting physiological signals provided in this application. DETAILED DESCRIPTION
[0064] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0065] The automatic exposure and gain control method for extracting physiological signals according to the preferred embodiment of the present invention is as follows: Figure 1 As shown, the automatic exposure and gain control method for extracting physiological signals includes the following steps:
[0066] Step S100: Detect the target area.
[0067] Specifically, before detecting the target area, the system is initialized, and the initialization system may include: setting PID parameters (Proportional, Integral, Differential, proportional, integral, differential), presetting target brightness and initial exposure time and gain weights, etc. Among them, the weight can set an initial ratio, such as 50 (exposure time): 50 (gain), to provide a balanced starting point for subsequent further adjustment; the initial ratio can also adopt other ratios as needed, such as 30 (gain): 70 (exposure). The detection of the target area can be performed using target detection technologies such as face detection technology and deep learning. The intercepted target area may include a region of interest (Region of Interest, RoI) and a non-interested region. For example, when the target area to be intercepted is the skin area of the face part, the target area may be a box provided by the face detection algorithm, and the target area includes a skin area and a non-skin area (such as background). Here, the region of interest (Region of Interest, RoI) is the skin of the face, and the non-interested region in the target area is the non-skin area in the area framed by the face. The process of detecting the target area and intercepting the target area is a known implementation technology in the prior art and will not be described in detail here.
[0068] Step S200: segment the target area based on the spatiotemporal information to obtain a final target area, and calculate and obtain the average brightness of the final target area.
[0069] Specifically, the target area is intercepted using existing target detection technology. This embodiment uses face detection technology to detect and intercept the target area. If the target area is not intercepted, it is determined to be overexposed or underexposed, that is, the difference between the current average brightness and the preset target brightness is too large, then the global average brightness is calculated as the average brightness, which is used to further perform dynamic weight adjustment of exposure time and gain. If the target area is intercepted, the intercepted area is preliminarily determined to be a preliminary target area, and the target area is segmented based on spatiotemporal information.
[0070] After determining the preliminary target area, the target area is segmented using a target area segmentation method with dynamic change characteristics, and the region of interest (RoI) and the non-interest region are segmented for further processing. The segmentation method can be based on time information and spatial information to segment the target area. If the information of the current frame and the previous and next frames is considered, the time information is the value of the pixel at the same position in different frames, and the spatial information is the value of the pixel at different positions in the same frame, that is, in the same time. For example, for each pixel, its pixel intensity is the (R, G, B) value, the spatial information is the (x, y) coordinate, and the time information can be the tth frame, or the time sliding window. If the 10 frames before and after are considered, the sliding window length is set to 20, and the current frame is the tth frame in the sliding window.
[0071] Furthermore, a plurality of image frames in a continuous time are extracted from the target area, and each pixel value and spatial coordinate of the image frame are obtained as the feature of the image frame. The feature is input into a clustering algorithm to obtain a series of superpixels. The overlap rate of the superpixels and the target area is calculated, and superpixels with an overlap rate greater than an overlap threshold are extracted. The signal-to-noise ratio of the superpixels with an overlap rate greater than the overlap threshold is calculated and arranged in order of size; the superpixels with the highest preset signal-to-noise ratio are extracted and determined as the final target area.
[0072] For example, the overlap rate of the superpixel and the target area is calculated, and the superpixels with an overlap rate greater than 0.8 can be extracted, and the signal-to-noise ratios of the superpixels with an overlap rate greater than 0.8 are calculated and arranged in order of size. The top 50% of the superpixels with the highest signal-to-noise ratios are extracted and determined as the final target area. Based on the final target area, the average brightness of the final target area can be calculated.
[0073] Step S300: Calculate and obtain an error based on the average brightness.
[0074] See also Figure 2 Specifically, a target brightness is preset, and the difference between the average brightness and the target brightness is calculated as an error.
[0075] The error can be calculated using the following formula:
[0076] error=tar get_brightness-current_brightness
[0077] Among them, error is the error, target_brightness is the preset target average brightness, and current_brightness is the average brightness of the target area in the current frame.
[0078] Specifically, before the step of calculating and obtaining the adjustment amount of exposure time and gain based on the error, the method further includes: performing dynamic weight adjustment, and the step of dynamic weight adjustment includes:
[0079] Preset an error threshold and an average gradient threshold;
[0080] First, determine whether the error is greater than the error threshold;
[0081] If yes, increase the exposure time weight;
[0082] If not, further determine whether the error is less than or equal to the negative value of the error threshold: if so, increase the gain weight; if not, calculate the average gradient of the current frame in the past N frames;
[0083] Secondly, determining whether the average gradient is greater than the average gradient threshold;
[0084] If yes, increase the exposure time weight;
[0085] If not, it is further determined whether the average gradient is greater than the negative value of the average gradient threshold: if so, the gain weight is increased; if not, the exposure time and gain adjustment amount are calculated. This step can analyze the error change trend of several consecutive frames. If the error continues to increase, the weight of the exposure time is increased; if the error continues to decrease, the weight of the gain is increased. In this way, it can respond more sensitively to changes in ambient brightness, realize dynamic exposure time and gain weight adjustment, and thus reach the target brightness faster.
[0086] See also Figure 2 , for example, an error threshold is preset to 40 and an average gradient threshold is preset to 10;
[0087] First, determine whether the error is greater than 40;
[0088] If yes, increase the exposure time weight;
[0089] If not, further determining whether the error is less than or equal to -40;
[0090] If yes, increase the gain weight, if no, calculate the average gradient of the last 5 frames of the current frame;
[0091] Secondly, determine whether the average gradient is greater than 10;
[0092] If yes, increase the exposure time weight;
[0093] If not, further determining whether the average gradient is greater than -10;
[0094] If so, increase the gain weight;
[0095] If not, the exposure time and gain adjustment are calculated. Through dynamic weight adjustment, the weights of exposure time and gain are updated in real time, dynamic lighting changes can be comprehensively considered, and dynamic step adjustment is adopted to achieve rapid balance of image brightness. Physiological signals can be stably extracted under changing lighting conditions, effectively reducing the impact of ambient light changes, facial expressions and body movements on physiological signal extraction, and improving the accuracy and reliability of physiological parameter measurement.
[0096] Step S400: Calculate and obtain the adjustment amount of exposure time and gain based on the error.
[0097] Specifically, the error is divided into a current frame error and a historical frame error according to the acquisition time;
[0098] Calculating the sum of the current frame error and the historical frame error to obtain a cumulative error ∑(error);
[0099] The current frame error, the accumulated error, and the error change rate are linearly combined according to preset parameters to obtain the output of the PID controller, and the calculation is performed according to the following formula:
[0100] Δ=Kp*error+Ki*∑(error)+Kd*(error-last_error)
[0101] Among them, Δ is the output of the PID controller, Kp (proportional adjustment coefficient), Ki (integral adjustment coefficient) and Kd (differential adjustment coefficient), ∑(error) is the accumulated error, and last_error is the historical error of the previous frame.
[0102] Specifically, the step of calculating the exposure time and the gain adjustment amount includes:
[0103] The exposure time exp_old of the previous frame, the gain gain_old of the previous frame, and the maximum exposure time exp_max are obtained. The function g(x) of the relationship between gain and brightness and the function h(x) of the relationship between exposure time and brightness are obtained through experiments, and the function g(x) of the relationship between gain and brightness and the function h(x) of the relationship between exposure time and brightness are fitted into a mapping function fun_exp2gain through experiments. Based on the exposure time exp_old and gain gain_old of the previous frame, the maximum exposure time exp_max, the output Δ of the PID controller and the mapping function fun_exp2gain, the exposure time and gain adjustment are calculated.
[0104] Furthermore, the function g(x) of the relationship between gain and brightness, the function h(x) of the relationship between exposure time and brightness, and the mapping function fun_exp2gain can be obtained by the following method:
[0105] (1) Select the brightness range and interval:
[0106] Select an appropriate brightness range, such as 0-255.
[0107] Choose a suitable interval, such as one recording point for every 10 brightness units.
[0108] (2) Fixed exposure, recording gain:
[0109] Fix the exposure value, adjust the gain at each recording point until the desired brightness is reached, and record the gain value at each recording point.
[0110] (3) Fixed gain, record exposure:
[0111] Fix the gain value, adjust the exposure at each recording point until the desired brightness is achieved, and record the exposure value at each recording point.
[0112] (4) Data analysis and fitting:
[0113] The data is fitted using polynomial fitting or machine learning methods (such as support vector machines, neural networks, etc.) to obtain a function g(x) describing the relationship between gain and brightness and a function h(x) describing the relationship between exposure and brightness.
[0114] (5) Verification and optimization:
[0115] An independent validation data set is used to test the accuracy and robustness of the fitting function. Furthermore, the fitting function can be optimized as needed, for example, by selecting different fitting methods, adjusting parameters, etc.
[0116] (6) Implement the mapping function:
[0117] The fitting functions g(x) and h(x) are combined into a mapping function f(x) for real-time adjustment of exposure and gain.
[0118] Further, the specific steps of calculating the exposure time adjustment amount Δexp and the gain adjustment amount Δgain based on the exposure time and gain of the previous frame, the maximum exposure time, the output of the PID controller and the mapping function fun_exp2gain are:
[0119] If exp_old<exp_max and exp_old<exp_max
[0120] but
[0121] If exp+Δ>exp_max and exp_old>exp_max
[0122] but
[0123] When Δ>0, if exp_old>Δ and exp_old<exp_max
[0124] but
[0125] When Δ<0, if fun_exp2gain(Δ)<0 and gain_old>0
[0126] but
[0127] The algorithm converts the exposure time change into the corresponding gain change to ensure that the changes in exposure time and gain have equal impact on the image brightness. This method can effectively reduce or even eliminate the impact phenomenon caused by the transition process of adjusting the gain after adjusting the exposure time to the maximum.
[0128] Step S500: If the error is greater than the error threshold, then calculate and obtain the updated exposure time and gain.
[0129] Specifically, obtaining a current exposure time, and determining whether the current exposure time is less than the maximum exposure time;
[0130] If not, then make an adjustment threshold judgment;
[0131] If so, calculate and obtain an updated exposure time and gain, wherein the updated exposure time is the sum of the exposure time of the previous frame and the exposure time adjustment amount;
[0132] The updated gain is the sum of the previous frame gain and the gain adjustment amount.
[0133] The adjustment threshold determination includes:
[0134] Presetting the adjustment threshold;
[0135] Calculating the absolute value of the error based on the updated exposure time and gain, and determining whether the absolute value of the error is greater than the adjustment threshold;
[0136] If so, applying the updated exposure time and gain adjustment amount to the image frame;
[0137] If not, return to step S100.
[0138] Furthermore, the physiological signal of the adjusted image frame is extracted, and the physiological parameter is further calculated.
[0139] Specifically, image processing technology is used to extract physiological signals and further calculate physiological parameters such as heart rate, blood pressure, blood oxygen, and respiratory rate.
[0140] It should be noted that the automatic exposure and gain control method for extracting physiological signals based on this embodiment can adapt to changing ambient light. Specifically, all the above thresholds can be pre-set and adjusted according to actual needs, thereby achieving sensitive and stable dynamic weight adjustment, so as to better enable the target area to reach the target brightness.
[0141] In summary, the method of the present invention includes initializing the system, detecting the target area, and intercepting the target area; segmenting the target area based on spatiotemporal information to obtain the final target area, calculating and obtaining the average brightness of the final target area; calculating and obtaining the error based on the average brightness, and performing dynamic weight adjustment; calculating and obtaining the adjustment amount of exposure time and gain based on the error; if the error is greater than the error threshold, then calculating and obtaining the updated exposure time and gain. The present invention controls the average brightness of the target area in real time, synchronously adjusts the exposure time and analog gain, and reasonably allocates the ratio between the two to achieve physiological monitoring with a high signal-to-noise ratio. In addition, the present invention also comprehensively considers the current and past lighting conditions, and adopts dynamic step size adjustment to achieve rapid balance of image brightness. Physiological signals can be stably extracted under different lighting conditions, effectively reducing the influence of factors such as ambient light changes, facial expressions and body movements on the extraction of physiological signals, and improving the accuracy and reliability of physiological parameter measurement.
[0142] Based on the above-mentioned automatic exposure and gain control method for extracting physiological signals, this embodiment provides an automatic exposure and gain control device for extracting physiological signals, such as Figure 3 As shown, the device comprises:
[0143] The image acquisition module 61 is used to acquire continuous image frames containing human skin.
[0144] Specifically, the image acquisition module 61 may include:
[0145] a. The camera sensor unit can be a wide-band single-channel (monochrome) or a narrow-band multi-channel (multi-wavelength), including monitoring of both RGB bands (such as 450-650nm) and NIR bands (such as 760-940nm). The camera sensor unit may also include: an infrared filter (IR-CUT), which is installed at the camera sensor end and can be popped up or retracted automatically according to the lighting conditions. For example, the infrared filter can be popped up when there is sufficient natural light to filter out the red light in the natural light; the infrared filter can be automatically retracted when there is insufficient natural light to allow red light to enter, so as to achieve continuous monitoring from day to night. The camera sensor unit can effectively improve the image quality and enhance the stability and reliability of the system.
[0146] b. A light source emitting unit, which is used to provide continuous and stable illumination. For example, when natural light is insufficient, an infrared light source can be used to supplement the light (such as a chip LED with a wavelength of 850nm) to provide continuous and stable illumination. If the ambient light is sufficient, the light source emitting unit can be automatically turned off. On the other hand, it is also possible to consider using different types of light sources, such as blue light, ultraviolet light, etc., which can provide more information for physiological signal detection.
[0147] The target detection module 62 is used to detect the target area.
[0148] Specifically, a target area is detected from the continuous image frames containing human skin, and the target area may be the face, chest, abdomen, soles of the feet, etc. of human skin tissue, and the target area is intercepted. If the target area is not intercepted, the global average brightness is calculated as the average brightness; if the target area is intercepted, the target area is segmented based on spatiotemporal information.
[0149] The target region segmentation and brightness calculation module 63 is used to segment the target region based on the spatiotemporal information to obtain the final target region, and calculate and obtain the average brightness of the final target region.
[0150] Specifically, a target region segmentation method based on dynamic change characteristics is used to segment the final target region and calculate the average brightness for subsequent calculations.
[0151] The exposure-analog gain control module 64 is used to calculate and obtain the error based on the average brightness, perform dynamic weight adjustment, and calculate and obtain the adjustment amount of exposure time and gain based on the error. If the error is greater than the error threshold, then calculate and obtain the updated exposure time and gain to adjust the image frame.
[0152] Specifically, the exposure-analog gain control module 64 can obtain parameters such as the current error, the accumulated error, and the error change rate according to the average brightness of the target area, and perform linear combination calculations on them to obtain updated exposure time and gain. Furthermore, the image signal processing unit (Image signal processor, ISP) can be called to apply the calculated updated exposure time and gain to the next frame image.
[0153] The physiological signal extraction module 65 is used to extract the physiological signal in the adjusted image frame. Specifically, the physiological signal can be extracted by using image processing technology, and further physiological parameters such as heart rate, blood pressure, blood oxygen, and respiration can be calculated. Furthermore, the video or image frame obtained in real time by the physiological signal extraction module 65, as well as the obtained monitoring results, can be displayed on hardware such as a UI interface.
[0154] Specifically, the target detection module 62, the target area segmentation and brightness calculation module 63, the exposure-analog gain control module 64 and the physiological signal extraction module 65 are virtual modules, which include a set of software programs for implementing the automatic exposure and gain control method for extracting physiological signals, and are used for real-time analysis, adjustment, and processing of the continuous image frames containing human skin obtained from the image acquisition module 61.
[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. An automatic exposure and gain control method for extracting physiological signals, characterized in that: The method comprises: Detect target area; Segmenting the target area based on the spatiotemporal information to obtain a final target area, and calculating and obtaining an average brightness of the final target area; Calculate and obtain an error based on the average brightness; Calculating and obtaining exposure time and gain adjustment amounts based on the error; If the error is greater than the error threshold, then, calculating and obtaining the updated exposure time and gain; Wherein, before the step of calculating and obtaining the adjustment amount of exposure time and gain based on the error, the step further includes: Performing dynamic weight adjustment based on the error, the dynamic weight adjustment comprising: When the error is greater than a preset error threshold, increasing the exposure time weight; When the error is less than or equal to the error threshold and less than or equal to the negative value of the error threshold, increasing the gain weight; When the error is less than or equal to the error threshold and greater than a negative value of the error threshold, calculating an average gradient of nearly N frames of the current frame; When the average gradient is greater than a preset average gradient threshold, increasing the exposure time weight; When the average gradient is less than or equal to a preset average gradient threshold, determining whether the average gradient is greater than a negative value of the average gradient threshold; When the average gradient is greater than the negative value of the average gradient threshold, the gain weight is increased until the average gradient is less than or equal to the negative value of the average gradient threshold.
2. The automatic exposure and gain control method for extracting physiological signals according to claim 1, characterized in that: The step of detecting the target area comprises: intercepting the target area; If the target area is not captured, the global average brightness is calculated as the average brightness; If the target area is captured, segmenting the target area based on the spatiotemporal information; The steps of segmenting the target area based on the spatiotemporal information to obtain the final target area, and calculating and obtaining the average brightness of the final target area include: Extracting a number of image frames at consecutive times from the target area, and obtaining each pixel value and spatial coordinate of the image frame as a feature of the image frame; Inputting the features into a clustering algorithm to obtain a series of superpixels; Calculating the overlap rate of the superpixel and the target area, extracting the superpixels whose overlap rate is greater than the overlap threshold, and calculating and arranging the signal-to-noise ratios of the superpixels whose overlap rate is greater than the overlap threshold in order of size; Extracting a preset proportion of super pixels with the highest signal-to-noise ratio and determining them as the final target area; Based on the final target area, an average brightness of the target area is calculated.
3. The automatic exposure and gain control method for extracting physiological signals according to claim 1, characterized in that: The step of calculating and obtaining the error based on the average brightness comprises: The difference between the average brightness and the preset target brightness is calculated as an error.
4. The automatic exposure and gain control method for extracting physiological signals according to claim 1, characterized in that: The step of calculating and obtaining the adjustment amount of exposure time and gain based on the error comprises: The error is divided into a current frame error and a historical frame error according to the acquisition time; Calculating the sum of the current frame error and the historical frame error to obtain a cumulative error; The current frame error, the accumulated error and the error change rate are linearly combined according to preset parameters to calculate the output of the PID controller; Get the exposure time and gain of the previous frame and the maximum exposure time; The function of the relationship between gain and brightness and the function of the relationship between exposure time and brightness are obtained through experiments; Fitting the function of the relationship between gain and brightness and the function of the relationship between exposure time and brightness into a mapping function through experiments; The exposure time and gain adjustment amount are obtained by calculation based on the exposure time and gain of the previous frame, the maximum exposure time, the output of the PID controller and the mapping function.
5. The automatic exposure and gain control method for extracting physiological signals according to claim 4, characterized in that: The output of the PID controller is obtained by calculation according to the following formula: Δ=Kp*error+Ki*∑(error)+Kd*(error-last_error) Wherein, Δ is the output of the PID controller, Kp, Ki and Kd are parameters of the PID controller, ∑(error) is the accumulated error, and last_error is the historical frame error of the previous frame.
6. The automatic exposure and gain control method for extracting physiological signals according to claim 5, characterized in that: The step of calculating and obtaining the updated exposure time and gain comprises: Obtaining a current exposure time, and determining whether the current exposure time is less than the maximum exposure time; If yes, then calculate and obtain an updated exposure time and gain, wherein the updated exposure time is the sum of the exposure time of the previous frame and the exposure time adjustment amount; The updated gain is the sum of the previous frame gain and the gain adjustment amount.
7. The automatic exposure and gain control method for extracting physiological signals according to claim 6, characterized in that: The method further comprises: When the current exposure time is greater than or equal to the maximum exposure time, calculating an absolute value of the error based on the updated exposure time and gain; If the absolute value of the error is greater than a preset adjustment threshold, applying the updated exposure time and gain adjustment amount to the image frame to obtain an adjusted image frame; If the absolute value of the error is less than or equal to the adjustment threshold, the process returns to the step of detecting the target area.
8. The automatic exposure and gain control method for extracting physiological signals according to claim 7, characterized in that: The method further comprises: A physiological signal of the adjusted image frame is extracted, and a physiological parameter is calculated based on the physiological signal.
9. An automatic exposure and gain control device for extracting physiological signals, characterized in that: The device comprises: An image acquisition module, used for acquiring continuous image frames containing human skin; Target detection module, used to detect target area; A target area segmentation and brightness calculation module is used to segment the target area based on the spatiotemporal information to obtain the final target area, and calculate and obtain the average brightness of the final target area; An exposure-analog gain control module, used to calculate and obtain an error based on the average brightness, perform dynamic weight adjustment, calculate and obtain an adjustment amount of exposure time and gain based on the error, and if the error is greater than an error threshold, calculate and obtain an updated exposure time and gain, and adjust the image frame; A physiological signal extraction module, used for extracting physiological signals from the adjusted image frame; Wherein, before the step of calculating and obtaining the adjustment amount of exposure time and gain based on the error, the exposure-analog gain control module further includes: Performing dynamic weight adjustment based on the error, the dynamic weight adjustment comprising: When the error is greater than a preset error threshold, increasing the exposure time weight; When the error is less than or equal to the error threshold and less than or equal to the negative value of the error threshold, increasing the gain weight; When the error is less than or equal to the error threshold and greater than a negative value of the error threshold, calculating an average gradient of nearly N frames of the current frame; When the average gradient is greater than a preset average gradient threshold, increasing the exposure time weight; When the average gradient is less than or equal to a preset average gradient threshold, determining whether the average gradient is greater than a negative value of the average gradient threshold; When the average gradient is greater than the negative value of the average gradient threshold, the gain weight is increased until the average gradient is less than or equal to the negative value of the average gradient threshold.
Citation Information
Patent Citations
Human body physiological parameter monitoring method based on human face recognition, applied to workstation
CN110037667A
Physiological parameter robustness detection method based on infrared video
CN115147769A
Target detection method and system
CN115909117A
Information service system, electronic device, control method, and control program
JP2014053693A
Biological information measurement device, biological information measurement method, and biological information measurement system and program
JP2018011641A
