A method, device, electronic device and storage medium for detecting a physiological state

Abstract

The present disclosure provides a physiological state detection method, apparatus, electronic device, and storage medium. The method includes: obtaining a video stream collected by a camera device; extracting a face image of a target object from multiple frames of images in the video stream; extracting at least one region of interest in each frame of the face image, where the region of interest includes at least one connected face smooth sub-region; respectively determining a first contribution degree and a second contribution degree of each region of interest to the physiological state detection result according to the area and pixel brightness information of each region of interest; fusing the image information of at least one region of interest based on the first contribution degree and the second contribution degree; and extracting physiological state information based on the fused image information to obtain a physiological state detection result of the target object. The present disclosure realizes physiological state detection based on an image processing method, can perform real-time measurement anytime and anywhere, and has better practicability.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and in particular, to a physiological state detection method, apparatus, electronic device, and storage medium. Background Art

[0002] Accurate physiological state data is the basis for analyzing human variability. Therefore, the detection of physiological states is of great significance and is thus widely applied to various application scenarios.

[0003] Taking the safe driving scenario as an example, effective physiological state detection can help understand the physiological states of vehicle occupants, thereby providing auxiliary decision-making for safe driving. In related technologies, physiological state detection mainly relies on dedicated detection devices, such as blood pressure monitors, heart rate monitors, blood oxygen monitors, etc. In addition, wearable devices such as smart watches and smart bands integrated with relevant sensing components can also be used to measure physiological states.

[0004] It can be seen that the above detection solutions require contact measurement with dedicated instruments, which brings inconvenience to detection and thus cannot well meet the needs of scenarios such as safe driving. Summary of the Invention

[0005] The embodiments of the present disclosure at least provide a physiological state detection method, apparatus, electronic device, and storage medium.

[0006] In a first aspect, the embodiments of the present disclosure provide a physiological state detection method, including:

[0007] Obtaining a video stream collected by a camera device;

[0008] Extracting a face image of a target object from multiple frames of images in the video stream;

[0009] Extracting at least one region of interest in each frame of the face image, where the region of interest includes at least one connected smooth face sub-region;

[0010] Determining a first contribution degree of each region of interest to the physiological state detection result according to the area of each region of interest;

[0011] Determining a second contribution degree of each region of interest to the physiological state detection result according to the pixel brightness information of each region of interest;

[0012] Based on the first contribution degree and the second contribution degree, fusing the image information of at least one region of interest;

[0013] Extracting physiological state information based on the fused image information to obtain a physiological state detection result of the target object.

[0014] In a possible implementation, the first contribution degree includes an area weight, and the second contribution degree includes a brightness weight;

[0015] Fusing the image information of at least one region of interest based on the first contribution degree and the second contribution degree includes:

[0016] Based on the product of the area weight and the brightness weight respectively corresponding to the at least one region of interest, determining the total weight value respectively corresponding to the at least one region of interest;

[0017] Based on the corresponding total weight value, performing weighted fusion on the image information of the at least one region of interest to determine the fused image information.

[0018] In a possible implementation, the area weight is determined as follows:

[0019] Performing a summation operation on the areas of the respective regions of interest to obtain an area sum value;

[0020] For each of the regions of interest, calculating the ratio of the area of the region of interest to the area sum value to obtain the area proportion of the region of interest, and using it as the area weight corresponding to the region of interest.

[0021] In a possible implementation, when the pixel brightness information includes an average brightness value, the brightness weight is determined as follows:

[0022] Performing a summation operation on the average brightness values of the respective regions of interest to obtain a brightness sum value;

[0023] For each of the regions of interest, calculating the ratio of the average brightness value of the region of interest to the brightness sum value to obtain the brightness proportion of the region of interest, and using it as the brightness weight corresponding to the region of interest.

[0024] In a possible implementation, determining the first contribution degree of each region of interest to the physiological state detection result according to the area of each region of interest includes:

[0025] When the area of the region of interest is less than a preset area threshold, determining the first contribution degree of the region of interest to be 0; and / or

[0026] Determining the second contribution degree of each region of interest to the physiological state detection result according to the pixel brightness information of each region of interest includes:

[0027] Determining the average brightness value of each region of interest according to the pixel brightness information of each region of interest;

[0028] When the average luminance value is less than the first preset luminance threshold, determining that the second contribution degree of the region of interest is 0, or when the average luminance value is greater than the second preset luminance threshold, determining that the second contribution degree of the region of interest is 0.

[0029] In a possible implementation manner, the fusing the image information of at least one region of interest based on the first contribution degree and the second contribution degree includes:

[0030] For each region of interest, performing time-domain processing on the luminance values of the region of interest in the multiple frames of images to determine a time-domain luminance signal corresponding to the region of interest;

[0031] Fusing the time-domain luminance signals corresponding to the multiple regions of interest based on the first contribution degree and the second contribution degree to obtain a fused time-domain luminance signal;

[0032] The extracting physiological state information based on the fused image information to obtain a physiological state detection result of the target object includes:

[0033] Performing frequency-domain processing on the fused time-domain luminance signal, and determining a physiological state value of the target object based on a peak value of the frequency-domain signal obtained by the frequency-domain processing.

[0034] In a possible implementation manner, when the fused image information includes luminance values corresponding to multiple different color channels, the extracting physiological state information based on the fused image information to obtain a physiological state detection result of the target object includes:

[0035] Performing time-domain processing on the luminance values corresponding to the three color channels in the fused image information to determine a time-domain luminance signal corresponding to each of the color channels;

[0036] Performing principal component analysis on the time-domain luminance signals corresponding to the multiple different color channels to obtain a time-domain signal characterizing the physiological state of the target object;

[0037] Performing frequency-domain processing on the time-domain signal characterizing the physiological state of the target object to obtain a frequency-domain signal characterizing the physiological state of the target object;

[0038] Determining a physiological state value of the target object based on a peak value of the frequency-domain signal.

[0039] In a possible implementation manner, the method further includes:

[0040] For each frame of the face image, performing face feature point extraction;

[0041] Based on the extracted facial feature points, at least one connected facial smoothing sub-region in each frame of the facial image is determined.

[0042] In a possible implementation manner, the first contribution value is proportional to the area of the corresponding region of interest; the second contribution is proportional to the average pixel brightness value of the corresponding region of interest and / or the second contribution is inversely proportional to the variance of the pixel brightness values of the corresponding region of interest.

[0043] In a second aspect, an embodiment of the present disclosure further provides a physiological state detection device, including:

[0044] An acquisition module, configured to acquire a video stream collected by a camera device;

[0045] A first extraction module, configured to extract a facial image of a target object from multiple frames of images in the video stream;

[0046] A second extraction module, configured to extract at least one region of interest in each frame of the facial image, where the region of interest includes at least one connected facial smoothing sub-region;

[0047] A determination module, configured to determine a first contribution of each region of interest to the physiological state detection result according to the area of each region of interest; and determine a second contribution of each region of interest to the physiological state detection result according to the pixel brightness information of each region of interest;

[0048] A fusion module, configured to fuse the image information of at least one region of interest based on the first contribution and the second contribution;

[0049] A detection module, configured to extract physiological state information based on the fused image information to obtain a physiological state detection result of the target object.

[0050] In a third aspect, an embodiment of the present disclosure further provides an electronic device, including: a processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the physiological state detection method according to any one of the first aspect and its various implementation manners are executed.

[0051] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of the physiological state detection method according to any one of the first aspect and its various implementation manners are executed.

[0052] The physiological state detection method, device, electronic device, and storage medium provided by the embodiments of the present disclosure can, when a video stream is obtained, first extract the face image of the target object from the video stream, and then extract at least one region of interest in the face image. In this way, when determining the first contribution degree and the second contribution degree of each region of interest to the physiological state detection result according to the area and pixel brightness information of each region of interest respectively, the image information of at least one region of interest can be fused, and finally, physiological state information is extracted based on the fused image information to obtain the physiological state detection result. Compared with the problem of inconvenient measurement caused by the need to use special instruments for contact measurement in the related art, the present disclosure realizes physiological state detection based on an image processing method, can perform real-time measurement anytime and anywhere, has better practicability, and in the process of performing physiological state detection, can combine the region area and region brightness to realize the fusion of the image information of the region of interest, which can further improve the measurement accuracy.

[0053] To make the above objects, features, and advantages of the present disclosure more obvious and understandable, the following specifically enumerates preferred embodiments and, in conjunction with the accompanying drawings, makes a detailed description as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required to be used in the embodiments. The accompanying drawings are incorporated into the specification and constitute a part of this specification. These drawings show embodiments that conform to the present disclosure and, together with the specification, are used to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0055] Figure 1 Shows a flowchart of a physiological state detection method provided by an embodiment of the present disclosure;

[0056] Figure 2 Shows a flowchart of a specific method for selecting ROIs in the physiological state detection method provided by an embodiment of the present disclosure;

[0057] Figure 3 Shows a schematic diagram of a physiological state detection device provided by an embodiment of the present disclosure;

[0058] Figure 4 Shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Apparently, the described embodiments are only some, rather than all, of the embodiments of the present disclosure. The components of the embodiments of the present disclosure generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Therefore, the detailed description of the embodiments of the present disclosure provided in the figures is not intended to limit the scope of the claimed present disclosure, but merely represents selected embodiments of the present disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure without creative efforts fall within the scope of the present disclosure.

[0060] It should be noted that like reference numerals and letters denote like items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0061] As used herein, the term "and / or" merely describes an association relationship and indicates that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" as used herein means any one of a plurality or any combination of at least two of a plurality. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set composed of A, B, and C.

[0062] It has been found through research that in the related art, physiological state detection mainly relies on dedicated detection devices, such as sphygmomanometers, heart rate monitors, and blood oxygen monitors. In addition, wearable devices such as smartwatches and smart bracelets integrated with relevant sensing components can also be used to measure physiological states.

[0063] It can be seen that the above detection scheme requires contact measurement with dedicated instruments, which brings inconvenience to detection and thus cannot well meet the needs of scenarios such as safe driving.

[0064] To solve the above problems, a non-contact detection scheme, namely remote Photoplethysmographic (rPPG), is provided in the related art. This method only requires a mobile phone terminal with a camera that people currently widely use to complete the detection, without additional hardware costs and is very convenient to use. However, the current bottleneck of the rPPG method is that the detection accuracy is inferior to some dedicated detection devices and is also easily affected by external light. In addition, for physiological feature detection using the rPPG method, the detected object needs to remain stationary for a period of time, and it can only be used for active detection.

[0065] Based on the above research, the present disclosure provides at least one physiological state detection scheme weighted based on regional area and regional brightness, so as to extract PPG signals in a weighted manner. The PPG signals extracted on this basis contain more effective pixel-like points, which can effectively improve the detection accuracy.

[0066] To facilitate the understanding of this embodiment, first, a physiological state detection method disclosed in the embodiments of the present disclosure will be introduced in detail. The execution subject of the physiological state detection method provided in the embodiments of the present disclosure is generally an electronic device with certain computing capabilities. Such an electronic device includes, for example: a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the physiological state detection method may be implemented by a processor calling computer-readable instructions stored in a memory.

[0067] See Figure 1 As shown in the figure, it is a flowchart of the physiological state detection method provided in the embodiments of the present disclosure. The method includes steps S101 to S107, where:

[0068] S101: Obtain a video stream collected by a camera device;

[0069] S102: Extract a face image of a target object from multiple frames of images in the video stream;

[0070] S103: Extract at least one region of interest in each frame of the face image. The region of interest includes at least one connected smooth sub-region of the face;

[0071] S104: Determine a first contribution degree of each region of interest to the physiological state detection result according to the area of each region of interest;

[0072] S105: Determine a second contribution degree of each region of interest to the physiological state detection result according to the pixel brightness information of each region of interest;

[0073] S106: Based on the first contribution degree and the second contribution degree, fuse the image information of at least one region of interest;

[0074] S107: Extract physiological state information based on the fused image information to obtain a physiological state detection result of the target object.

[0075] To facilitate the understanding of the physiological state detection method provided by the embodiments of the present disclosure, the application scenario of this method will be briefly introduced first. The physiological state detection method in the embodiments of the present disclosure can be applied to the automotive field. That is, the embodiments of the present disclosure can achieve the detection of the physiological state of a human object in the vehicle cabin environment. In the case of obtaining the physiological state detection result of the target object in the vehicle cabin environment, it is possible to timely understand whether the target object has an abnormal physical condition, and can provide timely reminders or assistance when the physical state is abnormal, which provides a more sufficient realistic possibility for safe driving.

[0076] In addition, the embodiments of the present disclosure can also be applied to other related fields that require physiological state detection, such as medical treatment and home life, without specific limitations here. Considering the wide application in the automotive field, the automotive field will be used as an example for illustration next.

[0077] Among them, the video stream in the embodiments of the present disclosure can be collected by a camera device (such as a fixed camera in the vehicle cabin), can also be collected by the built-in camera of the user terminal, or can be collected by other means, without specific limitations here.

[0078] In order to achieve the detection of the physiological state of a specific target object, the installation position of the camera can be preset based on the specific target object. For example, in order to achieve the detection of the physiological state of the driver in the vehicle, the camera here can be installed at a position where the shooting range covers the driving area, such as the inner side of the A-pillar of the vehicle, on the console, or at the steering wheel position; another example is that in order to achieve the detection of the physiological state of the occupants with multiple riding attributes including the driver and passengers in the vehicle, the camera here can be installed at positions such as the interior rearview mirror, the top trim, and the reading lamp where the shooting range can cover multiple seat areas in the vehicle cabin.

[0079] In practical applications, the in-vehicle image acquisition device included in the Driver Monitoring System (DMS) can also be used to collect the video stream of the driving area, or the in-vehicle image acquisition device included in the Occupant Monitoring System (OMS) can be used to collect the video stream of the riding area.

[0080] Considering that the skin color and brightness changes caused by the blood flow in the facial blood vessels can reflect physiological states such as heartbeat and breathing, here, face detection can be first performed on multiple frames of images in the video stream to extract multiple frames of face images of the target object in the vehicle cabin, and then the extraction of physiological state information can be achieved for the face images.

[0081] The target object can be an object with specific riding attributes, such as the driver or the passenger in the co-driver seat; or, the target object can be an object whose identity is registered in advance using facial information, such as the vehicle owner registered through an application; or, the target object can also be any occupant in the vehicle. At least one occupant can be located by performing facial detection on the video stream in the cabin, and one or more detected occupants are used as the target object.

[0082] During the face detection process, there may be a situation where the faces of multiple objects appear in a single frame of image. In some scenarios, it is possible to select an occupant in a certain riding position for physiological state detection, that is, using the occupant in that riding position as the target object. To implement the physiological state detection of the target object in the vehicle cabin, here, based on the face detection results of multiple frames of images and the specified riding position, multiple frames of face images of the target object can be determined from the detected face images, where the specified riding position is used to indicate the position of the target object to be measured.

[0083] The relative position of the camera for collecting the video stream in the vehicle cabin is fixed. According to the position of the camera, the images it captures can be divided according to the seat area. For example, for a 5-seater private car, it can be divided into: the image area corresponding to the driver's seat, the image area corresponding to the co-driver seat, the image area corresponding to the left rear seat, the image area corresponding to the right rear seat, and the image area corresponding to the middle rear seat. According to the position of the faces of each occupant object in the vehicle in the image and the coordinate range of each image area, it is possible to determine the image area into which the faces of each occupant object fall, and further determine the occupant object in the specified riding position as the target object.

[0084] In practical applications, the OMS generally takes pictures of the entire interior of the vehicle and may capture multiple people. It is possible to manually select the "front riding position" and "rear riding position" to specify the target object to be measured. At this time, the embodiments of the present disclosure can measure the faces in the corresponding areas of the image. When the DMS is shooting the main driving area and the object it captures only includes the driver, there is no need to specify the object.

[0085] It should be noted that physiological states such as heart rate, respiratory rate, blood oxygen, and blood pressure often require monitoring for a certain period of time before they can be evaluated. Therefore, in the embodiments of the present disclosure, the image change information corresponding to multiple frames of face images in a video stream lasting for a period of time is used to extract physiological state information, so that the extracted physiological state detection results are more in line with the needs of the actual scenario.

[0086] In the process of analyzing image change information based on a face image, there is a problem that detection cannot continue due to the loss of the Region of Interest (ROI) caused by factors such as face rotation, occlusion, or external light. Embodiments of the present disclosure can perform detection using multiple regions of interest in the face image. In this way, even if one region of interest is lost, detection can continue when other regions of interest perform well, making the usage scenarios more extensive.

[0087] In addition, considering that there is an angle between the human face and the optical axis of the camera, there are differences in the pixel area occupied by different regions of interest in the camera imaging and the corresponding pixel brightness. Therefore, here, the pixel area and the corresponding pixel brightness of the region of interest in the image can be weighted to obtain a signal quantity closer to the actual situation and improve the detection accuracy.

[0088] Among them, the image information within the region of interest can largely characterize the blood flow change situation. In practical applications, the region of interest can be a connected smooth sub-region of the face. This smooth sub-region of the face has a more uniform reflectivity to a certain extent, so that it can capture more effective skin color and brightness changes caused by the flow of facial blood vessels, and then more accurate physiological state detection can be achieved.

[0089] In the case of determining the region of interest of the face image, the physiological state detection method provided by the embodiments of the present disclosure can first determine the first contribution degree of each region of interest to the physiological state detection result and the second contribution degree of each region of interest to the physiological state detection result, and then fuse the image information of at least one region of interest based on the first contribution degree and the second contribution degree. Finally, physiological state information is extracted based on the fused image information. The physiological state detection result extracted here can be a detection result including at least one of heart rate, respiratory rate, blood oxygen, blood pressure, etc.

[0090] Here, the fusion process can be weighted fusion based on the first contribution degree and the second contribution degree. That is, for a region of interest, the higher the corresponding first contribution degree and the second contribution degree, the stronger the representation ability of the fused image information corresponding to the region of interest.

[0091] Alternatively, the above fusion process can also be performed based on the comparison result of the first contribution degree and the second contribution degree. Specifically, if the first contribution degree of a region of interest is much greater than the second contribution degree, the extraction result of the physiological state information corresponding to the region of interest can be determined based on the brightness information of the region of interest, ignoring the influence of the area; if the first contribution degree of a region of interest is much less than the second contribution degree, the extraction result of the physiological state information corresponding to the region of interest can be determined only based on the area of the region of interest, ignoring the influence of the brightness.

[0092] Among them, the first contribution degree can be determined based on the area of the corresponding region of interest, and the value of the first contribution degree is proportional to the area of the corresponding region of interest. That is, the larger the area of the region of interest, to a certain extent, it will provide a higher first contribution degree for physiological state detection. On the contrary, the smaller the area of the region of interest, the lower the first contribution degree provided for physiological state detection. This is mainly because as the area of the region increases, the amount of effective information that can be extracted will also increase, and the increase in the amount of effective information can improve the accuracy of physiological state detection to a certain extent.

[0093] In addition, the second contribution degree can be determined based on the pixel brightness information of the corresponding region of interest. The second contribution degree is proportional to the average pixel brightness value of the corresponding region of interest. That is, the region of interest with stronger pixel brightness will provide a higher second contribution degree for physiological state detection to a certain extent. On the contrary, the region of interest with weaker pixel brightness will provide a lower second contribution degree for physiological state detection. This is mainly because as the pixel brightness increases, the corresponding image quality will also increase, and better image quality can also improve the accuracy of physiological state detection to a certain extent. In addition, the second contribution degree is inversely proportional to the variance of the pixel brightness value of the corresponding region of interest. In this case, the corresponding second contribution degree can be determined based on the variance value, which will not be elaborated here.

[0094] It should be noted that in the process of determining the second contribution degree based on pixel brightness information, the possible extreme situations of overexposure or underexposure need to be considered. In these extreme situations, it may affect the accuracy of physiological state detection. Based on this, in practical applications, the regions of interest with overexposure or underexposure can be filtered out based on pixel brightness first, and then the second contribution degree of the regions of interest that meet the brightness requirements can be determined to ensure the accuracy of the final physiological state detection result.

[0095] Considering the key role of the determination of the region of interest in realizing physiological state detection, the determination process of the region of interest will be specifically described next. In some alternative implementation manners, the region of interest can be determined through the following Step 1 and Step 2:

[0096] Step 1: Extract facial feature points for each frame of facial image;

[0097] Step 2: Based on the extracted facial feature points, determine at least one connected facial smooth sub-region in each frame of facial image.

[0098] Here, first, facial feature points can be extracted from the facial image. Subsequently, based on the extracted facial feature points, the facial smooth sub-regions in the facial image can be determined and used as the regions of interest.

[0099] Among them, the process of extracting the facial feature points can be implemented by using a face key point detection algorithm. For example, the facial feature points of a standard facial image can be preset in advance. Here, the standard face can be a facial image of a face facing the camera including the facial features. In this way, during the process of extracting the facial feature points of the facial image of the target object extracted from each frame of image, each facial feature point can be determined based on the comparison between the extracted facial image of the target object and the standard facial image. For example, they can be relevant feature points with obvious facial characteristics such as eyebrow feature points, nose bridge feature points, nose tip feature points, cheek feature points, and mouth corner feature points.

[0100] In the embodiments of the present disclosure, one or more facial smooth sub-regions in the facial image can be determined based on the coordinate information of the determined facial feature points.

[0101] The facial smooth sub-region here can be a rectangular region, or other regions with a connected shape. The embodiments of the present disclosure do not make specific limitations on this. Next, a rectangular region will be used as an example for illustration.

[0102] In practical applications, the above facial smooth sub-regions can be a forehead smooth region determined based on the eyebrow feature points, a left upper cheek smooth region and a right upper cheek smooth region determined based on the cheek feature points, the nose bridge feature points, and the nose tip feature points, and a left lower cheek smooth region and a right lower cheek smooth region determined based on the cheek feature points, the nose tip feature points, and the mouth corner feature points.

[0103] In the case of no area occlusion, the above five regions can be extracted simultaneously in one frame of facial image. In the case of area occlusion, the regions that can actually be extracted from one frame of facial image can be determined according to the actual situation.

[0104] Such as Figure 2 shown, is a schematic diagram of the facial feature points that can be extracted for the facial image captured by the camera, with a total of 106 feature points. Here, based on the coordinate information of the facial feature points, 5 facial smooth sub-regions can be screened. For details, please refer to Figure 2, where Region 1 can be the rectangular ROI of Region 1 constructed by two feature points on both sides of the eyebrows; Region 2 is a cheek area on the left, and the rectangular ROI of Region 2 can be constructed by the position of the feature points on the left edge of the face, the feature points of the bridge of the nose, and the feature points of the left eye; Region 3 is a cheek area on the right, and the rectangular ROI of Region 3 can be constructed by the position of the feature points on the right edge of the face, the feature points of the bridge of the nose, and the feature points of the right eye. Region 4 is another cheek area on the left, and the rectangular ROI of Region 4 can be constructed by the position of the feature points on the left edge of the face, the feature points of the left nostril, and the feature points of the left corner of the mouth. Region 5 is another cheek area on the right, and the rectangular ROI of Region 5 can be constructed by the position of the feature points on the right edge of the face, the feature points of the right nostril, and the feature points of the right corner of the mouth.

[0105] When extracting the regions of interest from each frame of facial image, on the one hand, the first contribution degree corresponding to each region of interest can be determined based on the area of each region of interest, and on the other hand, the second contribution degree corresponding to each region of interest can be determined based on the pixel brightness information of each region of interest.

[0106] Among them, when the area of the region of interest is smaller than the preset area threshold, the first contribution degree of the region of interest is determined to be 0. That is, for a relatively small region of interest, its image representation ability is limited, and the corresponding first contribution degree can be not considered here. In addition, when determining the average brightness value of each region of interest according to the pixel brightness information of each region of interest, when the average brightness value is smaller than the first preset brightness threshold, the second contribution degree of the region of interest is determined to be 0, or when the average brightness value is greater than the second preset brightness threshold, the second contribution degree of the region of interest is determined to be 0. That is, for a region of interest that may be overexposed or underexposed, the corresponding second contribution degree can be not considered here.

[0107] Based on this, when the area of a region of interest is within the normal area and the region of interest is at a normal brightness, the corresponding area weight and brightness weight can be determined respectively.

[0108] Among them, the area weight can be implemented according to the following steps:

[0109] Step 1: Perform a summation operation on the areas of each region of interest to obtain the area sum value;

[0110] Step 2: For each region of interest, calculate the ratio of the area of the region of interest to the area sum value to obtain the area proportion of the region of interest, which is used as the area weight corresponding to the region of interest.

[0111] Here, after summing up the areas of each region of interest, an area sum value can be obtained. For a region of interest with a larger proportion of its area in the area sum value, a larger area weight can be determined. Conversely, for a region of interest with a smaller proportion of its area in the area sum value, a smaller area weight can be determined.

[0112] In addition, the luminance weight can be implemented according to the following steps:

[0113] Step 1: Sum up the average luminance values of each region of interest to obtain a luminance sum value;

[0114] Step 2: For each region of interest, calculate the ratio of the average luminance value of the region of interest to the luminance sum value to obtain the luminance proportion of the region of interest, which is used as the luminance weight corresponding to the region of interest.

[0115] Here, after summing up the average luminance values of each region of interest, a luminance sum value can be obtained. For a region of interest with a larger proportion of its average luminance value in the luminance sum value, a larger luminance weight can be determined. Conversely, for a region of interest with a smaller proportion of its luminance in the luminance sum value, a smaller luminance weight can be determined.

[0116] The physiological state detection method provided by the embodiments of the present disclosure can fuse the image information of each region of interest based on the area weight and the luminance weight for subsequent physiological state detection. Among them, the specific fusion process can include the following steps:

[0117] Step 1: Determine the total weight value corresponding to each of at least one region of interest based on the product of the area weight and the luminance weight respectively corresponding to the at least one region of interest;

[0118] Step 2: Perform weighted fusion on the image information of the at least one region of interest based on the corresponding total weight value to determine the fused image information.

[0119] Here, the total weight value corresponding to each region of interest can be determined first. Then, in the case of assigning the corresponding total weight value to the image information of each region of interest, the fused image information can be determined through weighted summation.

[0120] For the region of interest with relatively large area weight and luminance weight, the corresponding total weight value is also larger, which can provide more information support for the fused image information to a certain extent. For the region of interest with relatively large area weight and relatively small luminance weight, or the region of interest with relatively small area weight and relatively large luminance weight, the corresponding total weight value may be larger or smaller, which needs to be determined based on the specific image analysis results. The image information obtained through weighted fusion can maximize the extraction of effective pixel points on the face, which provides more data support for subsequent physiological state detection, thus facilitating the improvement of detection accuracy.

[0121] In the case where the first contribution degree and the second contribution degree of each region of interest to the physiological state detection result are determined, the physiological state detection method provided by the embodiments of the present disclosure can first determine the time-domain luminance signal corresponding to each region of interest through time-domain processing, and then fuse the time-domain luminance signals based on the first contribution degree and the second contribution degree of each region of interest, so as to facilitate the extraction of physiological state information based on the fused time-domain luminance signal.

[0122] To further improve the accuracy of physiological state detection, here, the fused time-domain luminance signal can be subjected to frequency-domain processing. More useful information can be analyzed based on the frequency-domain signal obtained through frequency-domain processing. For example, the amplitude distribution and energy distribution of each frequency component can be determined, so as to obtain the frequency values of the main amplitude and energy distributions. Here, the physiological state value of the target object can be determined based on the peak value of the frequency-domain signal.

[0123] In the embodiments of the present disclosure, in the case where the image information obtained through fusion is determined, here, the time-domain luminance signal corresponding to each color channel can be determined based on the luminance values corresponding to the three color channels of the image information obtained through fusion. After performing principal component analysis on each color channel, the physiological state detection of the target object can be realized. Specifically, it can be realized through the following steps:

[0124] Step 1: Perform time-domain processing on the luminance values corresponding to the three color channels in the image information obtained through fusion to determine the time-domain luminance signal corresponding to each color channel;

[0125] Step 2: Perform principal component analysis on the time-domain luminance signals corresponding to multiple different color channels to obtain a time-domain signal characterizing the physiological state of the target object;

[0126] Step 3: Perform frequency-domain processing on the time-domain signal characterizing the physiological state of the target object to obtain a frequency-domain signal characterizing the physiological state of the target object;

[0127] Step 4: Determine the physiological state value of the target object based on the peak value of the frequency-domain signal.

[0128] Considering that the physiological state directly affects the blood flow change of the target object, and the blood flow change can be characterized based on the brightness change of the image. Therefore, here, the image information to be fused can first be determined as the time-domain brightness signals of each of the red, green, and blue color channels, forming an RGB three-dimensional signal. Then, principal component analysis is performed on the time-domain brightness signals of the three different color channels, and the one-dimensional signal obtained after extracting the principal components (dimensionality reduction) is used as the time-domain signal characterizing the physiological state of the target object. This time-domain signal can be determined by the time-domain brightness signal of one of the above color channels (for example, the green channel), and the selected channel can be the one that can best characterize the blood flow change. In addition, it can also be determined by other principal component analysis methods, and no specific limitations are made here.

[0129] To facilitate more accurate principal component analysis, before performing principal component analysis on the three-dimensional time-domain brightness signal, processing such as regularization and Detrend filtering for denoising can be carried out. In addition, after principal component analysis, a moving average filtering denoising process can also be performed on the obtained time-domain signal, so as to further improve the accuracy of the time-domain signal and the accuracy of subsequent physiological state detection.

[0130] To facilitate further improving the accuracy of physiological state detection, here, the time-domain signal can be processed in the frequency domain to analyze more useful information. Here, the physiological state value of the target object can be determined based on the peak value of the frequency-domain signal.

[0131] Taking heart rate detection as an example, here, the peak value pmax of the frequency-domain signal can be determined. The original heart rate measurement value can be obtained through the summation result of pmax and the heart rate reference value. Among them, pmax represents the heart rate change amount, and the heart rate reference value can be determined by the lower limit of the heart rate estimation range based on experience. The influence of factors such as video frame rate and frequency-domain signal length can also be considered to adjust the heart rate reference value.

[0132] After determining the heart rate, relevant physiological indexes such as blood oxygen saturation and heart rate variability can be measured. For blood oxygen saturation, here, the time-domain signals of HbO2 and Hb can be detected using red light (600 - 800nm) and near-infrared light region (800 - 1000nm) respectively, and then the corresponding ratio is calculated to obtain the blood oxygen saturation. For heart rate variability, after extracting the time-domain signal, by calculating the distance between every two adjacent wave peaks and combining with the frame rate to obtain several interval times, and then taking the standard deviation (Standard Deviation of NN Intervals, SDNN) of these interval times, the heart rate variability is obtained.

[0133] The method for detecting the breathing rate is similar to that for detecting the heart rate. The main difference lies in that the range where the breathing rate is located is different from the range where the heart rate is located, and the corresponding reference values are set differently. Based on the same method described above, the breathing rate can be detected.

[0134] What is implemented in the embodiments of the present disclosure is the physiological state detection of multiple frames of images. That is, the image change information corresponding to the multiple frames of images can characterize the change of the physiological state. In practical applications, the detection result of the physiological state determined by the video stream can be updated as the video stream is continuously collected.

[0135] Here, in the case of obtaining one or more frames of images included in the new video stream, face detection can be performed on the images in the new video stream to extract the face images of the target objects in the vehicle cabin. Then, at least one region of interest in the face images is determined. When the first contribution degree and the second contribution degree of each region of interest to the physiological state detection result are determined, based on the first contribution degree and the second contribution degree, the image information of at least one region of interest is fused, and the physiological state detection result is updated based on the fused image information. If the preset detection duration is not reached, the update is performed again based on the obtained new video stream until the preset detection duration is reached, and the updated physiological state detection result is obtained.

[0136] Here, still taking the heart rate detection as an example for illustration. In the case where the preset detection duration is determined to be 30 s, the video stream can be continuously obtained within 30 s. In the case where the heart rate measurement value is calculated based on multiple frames of images of the starting video stream (for example, the video stream within the starting 5 s), it is still within 30 s. At this time, as the image frames are collected, the number of image frames increases. For each additional frame or every additional n frames, a new heart rate measurement value can be calculated, and then smoothed by moving average. After reaching 30 s, the measurement ends, and the final measurement result is obtained.

[0137] In the vehicle cabin environment, in order to help the target object perform more rapid physiological state measurement, here, during a physiological state detection process, a detection process reminder signal for reminding the target object of the required detection duration can be generated according to the duration of the obtained video stream and the preset detection duration. For example, when the duration of the obtained video stream (that is, the detection time that the physiological state detection of the current target object has continued) reaches 25 s and the preset detection duration is 30 s, a voice or screen prompt such as "Please keep still, and the detection will be completed in 5 seconds" can be issued; or, when the physiological state detection duration of the current target object reaches 30 s, a voice or screen prompt of "The measurement has been completed" is issued.

[0138] After the physiological state detection is implemented, the embodiments of the present disclosure can also display the physiological state detection result to provide better vehicle cabin services for the target object through the displayed physiological state detection.

[0139] In the disclosed embodiments, on the one hand, the physiological state detection results of the target object can be transmitted to a display screen in the cabin for display on the display screen. In this way, cabin personnel can monitor their own physiological states in real time, and can seek medical treatment in time or take other necessary measures when their own physiological states are abnormal. On the other hand, the physiological state detection results of the target object can also be transmitted to the server of the physiological state detection application, so that when the target object requests to obtain the detection results through the physiological state detection application, the physiological state detection results can be sent to the terminal device used by the target object through the server.

[0140] That is, the physiological state detection results of the target object can be recorded on the server side, and the physiological state detection results can also be statistically analyzed on the server side. For example, the physiological state statistical results of a month or a week in history can be determined. In this way, when the target object initiates a physiological state detection application request, the physiological state detection results, statistical results, etc. can be sent to the target object's terminal device to achieve a more comprehensive physiological state assessment.

[0141] Among them, the above-mentioned physiological state detection application can be a specific application program (Application, APP) used for physiological state detection. The APP can respond to the acquisition request of the detection results related to the target object, and then realize the result presentation on the APP, which is more practical.

[0142] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0143] Based on the same inventive concept, a physiological state detection device corresponding to the physiological state detection method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned physiological state detection method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0144] Reference Figure 3 , which is a schematic diagram of a physiological state detection device provided by an embodiment of the present disclosure, the device includes: an acquisition module 301, a first extraction module 302, a second extraction module 303, a determination module 304, a fusion module 305 and a detection module 306; wherein,

[0145] An acquisition module 301 is used to acquire a video stream captured by a camera device;

[0146] The first extraction module 302 is configured to extract the facial images of the target object from multiple frames of images in the video stream;

[0147] The second extraction module 303 is configured to extract at least one region of interest in each frame of facial image, and the region of interest includes at least one connected facial smoothing sub-region;

[0148] The determination module 304 is configured to determine the first contribution degree of each region of interest to the physiological state detection result according to the area of each region of interest; and determine the second contribution degree of each region of interest to the physiological state detection result according to the pixel brightness information of each region of interest;

[0149] The fusion module 305 is configured to fuse the image information of at least one region of interest based on the first contribution degree and the second contribution degree;

[0150] The detection module 306 is configured to extract physiological state information based on the fused image information to obtain the physiological state detection result of the target object.

[0151] The physiological state detection device provided by the embodiments of the present disclosure, when obtaining a video stream, can first extract the facial images of the target object from the video stream, and then extract at least one region of interest in the facial images. In this way, when determining the first contribution degree and the second contribution degree of each region of interest to the physiological state detection result according to the area and pixel brightness information of each region of interest respectively, the image information of at least one region of interest can be fused, and finally, physiological state information is extracted based on the fused image information to obtain the physiological state detection result. Compared with the problem of inconvenient measurement caused by contact measurement with a dedicated instrument in the related art, the present disclosure realizes physiological state detection based on an image processing method, can perform real-time measurement anytime and anywhere, has better practicability, and in the process of performing physiological state detection, can combine the region area and region brightness to realize the fusion of the image information of the region of interest, which can further improve the measurement accuracy.

[0152] In a possible implementation manner, the first contribution degree includes an area weight, and the second contribution degree includes a brightness weight; the fusion module 305 is configured to fuse the image information of at least one region of interest based on the first contribution degree and the second contribution degree according to the following steps:

[0153] Determine the total weight value corresponding to each of the at least one region of interest based on the product of the area weight and the brightness weight respectively corresponding to the at least one region of interest;

[0154] Perform weighted fusion on the image information of the at least one region of interest based on the corresponding total weight value to determine the fused image information.

[0155] In a possible implementation, the fusion module 305 is configured to determine the area weight in the following manner:

[0156] Perform a summation operation on the areas of the regions of interest to obtain a sum value of the areas;

[0157] For each region of interest, calculate the ratio of the area of the region of interest to the sum value of the areas to obtain the area proportion of the region of interest, which is used as the area weight corresponding to the region of interest.

[0158] In a possible implementation, when the pixel brightness information includes an average brightness value, the fusion module 305 is configured to determine the brightness weight in the following manner:

[0159] Perform a summation operation on the average brightness values of the regions of interest to obtain a sum value of the brightness;

[0160] For each region of interest, calculate the ratio of the average brightness value of the region of interest to the sum value of the brightness to obtain the brightness proportion of the region of interest, which is used as the brightness weight corresponding to the region of interest.

[0161] In a possible implementation, the determination module 304 is configured to determine the first contribution degree of each region of interest to the physiological state detection result according to the following steps based on the areas of the regions of interest:

[0162] When the area of the region of interest is less than a preset area threshold, determine that the first contribution degree of the region of interest is 0; and / or

[0163] The determination module 304 is configured to determine the second contribution degree of each region of interest to the physiological state detection result according to the following steps based on the pixel brightness information of the regions of interest:

[0164] Determine the average brightness value of each region of interest according to the pixel brightness information of the regions of interest;

[0165] When the average brightness value is less than a first preset brightness threshold, determine that the second contribution degree of the region of interest is 0, or when the average brightness value is greater than a second preset brightness threshold, determine that the second contribution degree of the region of interest is 0.

[0166] In a possible implementation, the fusion module 305 is configured to fuse the image information of at least one region of interest based on the first contribution degree and the second contribution degree according to the following steps:

[0167] For each region of interest, perform time-domain processing on the brightness values of the region of interest in multiple frames of images to determine the time-domain brightness signal corresponding to the region of interest;

[0168] Fuse the time-domain luminance signals corresponding to multiple regions of interest based on the first contribution degree and the second contribution degree to obtain a fused time-domain luminance signal;

[0169] A detection module 306, configured to extract physiological state information based on the fused image information according to the following steps to obtain a physiological state detection result of the target object:

[0170] Perform frequency-domain processing on the fused time-domain luminance signal, and determine the physiological state value of the target object based on the peak value of the frequency-domain signal obtained by the frequency-domain processing.

[0171] In a possible implementation manner, when the fused image information includes luminance values corresponding to multiple different color channels, the detection module 306 is configured to extract physiological state information based on the fused image information according to the following steps to obtain a physiological state detection result of the target object:

[0172] Perform time-domain processing on the luminance values corresponding to the three color channels in the fused image information to determine the time-domain luminance signal corresponding to each color channel;

[0173] Perform principal component analysis on the time-domain luminance signals corresponding to multiple different color channels to obtain a time-domain signal characterizing the physiological state of the target object;

[0174] Perform frequency-domain processing on the time-domain signal characterizing the physiological state of the target object to obtain a frequency-domain signal characterizing the physiological state of the target object;

[0175] Determine the physiological state value of the target object based on the peak value of the frequency-domain signal.

[0176] In a possible implementation manner, the second extraction module 303 is configured to extract a face smooth sub-region according to the following steps:

[0177] For each frame of face image, extract face feature points;

[0178] Based on the extracted face feature points, determine at least one connected face smooth sub-region in each frame of face image.

[0179] In a possible implementation manner, the first contribution degree value is proportional to the area of the corresponding region of interest; the second contribution degree is proportional to the average pixel luminance value of the corresponding region of interest and / or the second contribution degree is inversely proportional to the variance of the pixel luminance values of the corresponding region of interest.

[0180] For the processing flow of each module in the device and the interaction flow between the modules, reference can be made to the relevant descriptions in the above method embodiments, which will not be elaborated here.

[0181] An embodiment of the present disclosure further provides an electronic device, such as Figure 4 shown in the schematic structural diagram of the electronic device provided by the embodiment of the present disclosure, including: a processor 401, a memory 402, and a bus 403. The memory 402 stores machine-readable instructions executable by the processor 401 (for example, Figure 3 the execution instructions corresponding to the acquisition module 301, the first extraction module 302, the second extraction module 303, the determination module 304, the fusion module 305, and the detection module 306 in the device in

[0182] Obtain the video stream collected by the imaging device;

[0183] Extract the facial image of the target object from multiple frames of images in the video stream;

[0184] Extract at least one region of interest in each frame of the facial image, and the region of interest includes at least one connected smooth facial sub-region;

[0185] Determine the first contribution degree of each region of interest to the physiological state detection result according to the area of each region of interest;

[0186] Determine the second contribution degree of each region of interest to the physiological state detection result according to the pixel brightness information of each region of interest;

[0187] Based on the first contribution degree and the second contribution degree, fuse the image information of at least one region of interest;

[0188] Extract physiological state information based on the fused image information to obtain the physiological state detection result of the target object.

[0189] An embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the physiological state detection method described in the above method embodiment. Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

[0190] An embodiment of the present disclosure further provides a computer program product, which carries program code, and the instructions included in the program code can be used to execute the steps of the physiological state detection method described in the above method embodiment. Specifically, refer to the above method embodiment, and details are not described herein again.

[0191] Among them, the above computer program product can be specifically implemented by hardware, software, or a combination thereof. In an alternative embodiment, the computer program product is specifically embodied as a computer storage medium. In another alternative embodiment, the computer program product is specifically embodied as a software product, such as a Software Development Kit (SDK), etc.

[0192] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems and devices can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein. In several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be in an electrical, mechanical, or other form.

[0193] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0194] In addition, in each embodiment of the present disclosure, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0195] When the above functions are realized in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such an understanding, the technical solution of the present disclosure, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing an electronic device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present disclosure. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0196] Finally, it should be noted that the above embodiments are only specific embodiments of the present disclosure, used to illustrate the technical solutions of the present disclosure, rather than limiting them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present disclosure can still modify the technical solutions described in the foregoing embodiments, or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes, or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should all be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A physiological state detection method, characterized in that, Including: Obtain a video stream collected by a camera device; Extract a face image of a target object from multiple frames of images in the video stream; Extract at least one region of interest in each frame of the face image, where the region of interest includes at least one connected face smoothing sub-region; Determine a first contribution degree of each region of interest to the physiological state detection result according to the area of each region of interest; Determine a second contribution degree of each region of interest to the physiological state detection result according to the pixel brightness information of each region of interest; Based on the first contribution degree and the second contribution degree, fuse the image information of at least one region of interest; Extract physiological state information based on the fused image information obtained, and obtain the physiological state detection result of the target object.

2. The method according to claim 1, wherein The first contribution degree includes an area weight, and the second contribution degree includes a brightness weight; The fusing the image information of at least one region of interest based on the first contribution degree and the second contribution degree includes: Based on the product of the area weight and the brightness weight respectively corresponding to the at least one region of interest, determine a total weight value respectively corresponding to the at least one region of interest; Based on the corresponding total weight value, perform weighted fusion on the image information of the at least one region of interest, and determine the fused image information obtained.

3. The method according to claim 2, wherein The area weight is determined in the following manner: Perform a summation operation on the areas of each region of interest to obtain an area sum value; For each region of interest, calculate the ratio of the area of the region of interest to the area sum value to obtain the area proportion of the region of interest, and use it as the area weight corresponding to the region of interest.

4. The method according to claim 2 or 3, characterized in that, When the pixel brightness information includes an average brightness value, the brightness weight is determined in the following manner: Perform a summation operation on the average brightness values of each region of interest to obtain a brightness sum value; For each region of interest, calculate the ratio of the average brightness value of the region of interest to the brightness sum value to obtain the brightness proportion of the region of interest, and use it as the brightness weight corresponding to the region of interest.

5. The method according to any one of claims 1 to 4, characterized in that, The determining the first contribution degree of each region of interest to the physiological state detection result according to the area of each region of interest includes: When the area of the region of interest is less than a preset area threshold, determine that the first contribution degree of the region of interest is 0; and / or The determining the second contribution degree of each region of interest to the physiological state detection result according to the pixel brightness information of each region of interest includes: Determine the average brightness value of each region of interest according to the pixel brightness information of each region of interest; When the average brightness value is less than a first preset brightness threshold, determine that the second contribution degree of the region of interest is 0, or when the average brightness value is greater than a second preset brightness threshold, determine that the second contribution degree of the region of interest is 0.

6. The method according to any one of claims 1 to 5, characterized in that The fusing the image information of at least one region of interest based on the first contribution degree and the second contribution degree includes: For each region of interest, perform time-domain processing on the luminance values of the region of interest in the multi-frame images to determine the time-domain luminance signal corresponding to the region of interest; Based on the first contribution degree and the second contribution degree, fuse the time-domain luminance signals corresponding to multiple regions of interest to obtain a fused time-domain luminance signal; Performing physiological state information extraction based on the image information obtained by the fusion to obtain the physiological state detection result of the target object, including: Perform frequency-domain processing on the fused time-domain luminance signal, and determine the physiological state value of the target object based on the peak value of the frequency-domain signal obtained by the frequency-domain processing.

7. The method according to any one of claims 1 to 5, characterized in that When the image information obtained by the fusion includes luminance values corresponding to multiple different color channels, performing physiological state information extraction based on the image information obtained by the fusion to obtain the physiological state detection result of the target object, including: Perform time-domain processing on the luminance values corresponding to three color channels in the image information obtained by the fusion to determine the time-domain luminance signal corresponding to each color channel; Perform principal component analysis on the time-domain luminance signals corresponding to multiple different color channels to obtain a time-domain signal characterizing the physiological state of the target object; Perform frequency-domain processing on the time-domain signal characterizing the physiological state of the target object to obtain a frequency-domain signal characterizing the physiological state of the target object; Determine the physiological state value of the target object based on the peak value of the frequency-domain signal.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: For each frame of the face image, perform face feature point extraction; Based on the extracted face feature points, determine at least one connected face smoothing sub-region in each frame of the face image.

9. The method according to any one of claims 1 to 8, characterized in that, The first contribution degree value is proportional to the area of the corresponding region of interest; the second contribution degree is proportional to the average pixel luminance value of the corresponding region of interest and / or the second contribution degree is inversely proportional to the variance of the pixel luminance values of the corresponding region of interest.

10. A physiological state detection device, characterized in that, Including: An acquisition module, configured to acquire a video stream collected by a camera device; A first extraction module, configured to extract a face image of a target object from multiple frames of images of the video stream; A second extraction module, configured to extract at least one region of interest in each frame of the face image, where the region of interest includes at least one connected face smoothing sub-region; A determination module, configured to determine the first contribution degree of each region of interest to the physiological state detection result according to the area of each region of interest; Determine the second contribution degree of each region of interest to the physiological state detection result according to the pixel luminance information of each region of interest; A fusion module, configured to fuse the image information of at least one region of interest based on the first contribution degree and the second contribution degree; A detection module, configured to perform physiological state information extraction based on the image information obtained by the fusion to obtain the physiological state detection result of the target object.

11. An electronic device, characterized in that, Including: A processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps of the physiological state detection method according to any one of claims 1 to 9 are executed.

12. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the physiological state detection method according to any one of claims 1 to 9 are executed.

Images