Vehicle safety monitoring method, device, equipment and program product in rest mode

Abstract

The invention discloses a vehicle safety monitoring method and device in a rest mode, equipment and a program product, and the method comprises the steps: obtaining temperature time sequence data and air composition time sequence data in a target vehicle when the rest mode of the target vehicle is started and the existence of a rest person in the target vehicle is detected, and obtaining the face image time sequence data of the rest person; according to the temperature time sequence data and the air component time sequence data, the cabin environment comfort degree of the target vehicle is recognized; the physiological state health degree of the rest person is recognized according to the face image time sequence data; judging whether the rest person has a safety risk or not according to the cabin environment comfort and the physiological state health degree; and when the rest person has the safety risk, early warning is performed on the rest person and an emergency plan is started. The method guarantees the safety of people having a rest in the vehicle in the rest mode, improves the safety of the vehicle, and can be applied to the technical field of vehicle monitoring.

Description

Technical Field

[0001] This invention relates to the field of vehicle monitoring technology, and in particular to a method, device, equipment and program product for vehicle safety monitoring in a rest mode. Background Technology

[0002] As car functions become increasingly sophisticated, more and more vehicles are equipped with a nap mode to meet users' needs for short rests inside the car. This mode typically adjusts the seat angle, turns off some non-essential electrical equipment, and turns on the air conditioning to maintain a comfortable temperature inside the car, providing users with a comfortable resting environment.

[0003] However, the existing vehicle's nap mode has significant safety hazards: when the nap mode automatically terminates after the preset time, or when the air conditioning system abnormally stops working due to malfunction, insufficient power, or other reasons, the system simply terminates the nap-related functions without issuing any alarms or warnings to the passengers. At this time, the vehicle is in a relatively enclosed environment. With the breathing of the passengers, the oxygen concentration inside the vehicle gradually decreases, and if the outside temperature is high, the interior temperature rises rapidly. Because the passengers are in a sleep or light sleep state, they are unlikely to notice the environmental changes and are highly susceptible to dangerous situations such as hypoxia and heatstroke, which can even endanger their lives in severe cases.

[0004] The above problems urgently need to be addressed. Summary of the Invention

[0005] The purpose of this invention is to at least partially solve one of the technical problems existing in the prior art.

[0006] Therefore, one objective of this invention is to provide a vehicle safety monitoring method in a rest mode, which ensures the safety of people resting in the vehicle during the rest mode and improves vehicle safety.

[0007] Another objective of this invention is to provide a vehicle safety monitoring device in a rest mode.

[0008] To achieve the above-mentioned technical objectives, the technical solutions adopted in the embodiments of the present invention include: On one hand, embodiments of the present invention provide a vehicle safety monitoring method in a rest mode, comprising the following steps: When the target vehicle starts the nap mode and a person is detected resting inside the target vehicle, the time-series data of temperature and air composition inside the target vehicle are acquired, and the time-series data of the person resting is acquired. The cabin comfort of the target vehicle is identified based on the time-series temperature data and the time-series air composition data. The physiological health status of the resting person is identified based on the temporal data of the facial images; Based on the cabin environment comfort and the physiological health status, determine whether there is a safety risk to the resting personnel; If there is a safety risk to the resting personnel, an early warning will be issued to the resting personnel and an emergency plan will be activated.

[0009] Furthermore, in one embodiment of the present invention, the step of acquiring time-series data of temperature and air composition inside the target vehicle, and acquiring time-series data of facial images of the resting person, specifically includes: The temperature time-series data is acquired by multiple temperature sensors installed at different locations inside the target vehicle. The air composition time-series data is acquired by a multi-gas sensor module installed in the target vehicle; The facial image time-series data is acquired by an onboard camera installed in the target vehicle. When the rest mode is not interrupted, the temperature sensor, the multi-gas sensor module, and the vehicle camera maintain intermittent acquisition mode. When the rest mode is interrupted, the temperature sensor, the multi-gas sensor module, and the vehicle camera switch to real-time acquisition mode.

[0010] Furthermore, in one embodiment of the present invention, the step of identifying the cabin environment comfort of the target vehicle based on the temperature time-series data and the air composition time-series data specifically includes: Predict the target cabin temperature at the next moment based on the temperature time series data, and predict the target air composition at the next moment based on the air composition time series data; The target cabin temperature is compared with the preset comfort temperature range to obtain a temperature fit score. The target air composition is compared with a preset air composition ratio to obtain an air compatibility score. The cabin environment comfort level is determined based on the temperature compatibility score and the air compatibility score.

[0011] Furthermore, in one embodiment of the present invention, the step of identifying the physiological health status of the resting person based on the facial image time-series data specifically includes: Extract the regional image time-series data of the forehead region and / or cheek region based on the facial image time-series data; The red channel pixel values, green channel pixel values, and blue channel pixel values ​​of the time-series data of the regional image are sampled in the time domain to obtain multi-channel light intensity signals; The multi-channel light intensity signal is input into a pre-trained heart rate recognition model to obtain the real-time heart rate of the resting person; Micro-motion data of several key facial points are extracted based on the temporal data of the facial images. Time-domain analysis was performed on the micro-motion data to obtain the respiratory frequency band signal; The real-time respiratory rate of the resting person is determined based on the respiratory frequency band signal; The real-time heart rate is compared with a preset normal heart rate range to obtain a heart rate health score. The real-time respiratory rate is compared with a preset normal respiratory rate range to obtain a respiratory rate health score. The physiological health status is determined based on the heart rate health score and the respiratory rate health score.

[0012] Furthermore, in one embodiment of the present invention, the heart rate recognition model is trained through the following steps: Acquire temporal samples of the tester's facial images and determine the corresponding heart rate label based on the tester's measured heart rate; Based on the facial image temporal samples, extract the regional image temporal samples of the forehead region and / or cheek region; The red channel pixel values, green channel pixel values, and blue channel pixel values ​​of the temporal sample of the regional image are sampled in the temporal domain to obtain multi-channel light intensity signal samples. The multi-channel light intensity signal samples are input into a pre-constructed multi-branch convolutional neural network to obtain the predicted heart rate. The loss value is determined based on the predicted heart rate and the heart rate label; The parameters of the multi-branch convolutional neural network are updated using the backpropagation algorithm based on the loss value to obtain the trained heart rate recognition model.

[0013] Furthermore, in one embodiment of the present invention, the step of determining whether there is a safety risk to the resting personnel based on the cabin environment comfort and the physiological health status specifically includes: When the cabin environment comfort level is greater than or equal to a preset first threshold and the physiological health level is greater than or equal to a preset second threshold, it is determined that the resting personnel do not pose a safety risk. When the cabin environment comfort level is less than the first threshold and the physiological health level is greater than or equal to the second threshold, it is determined that the resting person has a slight safety risk. When the cabin environment comfort level is greater than or equal to the first threshold and the physiological health level is less than the second threshold, it is determined that the resting person has a moderate safety risk. If the cabin environment comfort level is less than the first threshold and the physiological health level is less than the second threshold, it is determined that the resting person has a serious safety risk.

[0014] Furthermore, in one embodiment of the present invention, the step of issuing an early warning to the resting personnel and activating the emergency plan specifically includes: The system matches the corresponding voice prompt content according to the safety risk level, plays the voice prompt content through the voice prompt module, and detects whether the resting person responds. If the resting personnel do not respond within a preset time period, the air conditioning and / or windows of the target vehicle will be adjusted according to the comfort level of the cabin environment, and a distress message will be sent to the cloud platform according to the physiological health status.

[0015] On the other hand, embodiments of the present invention provide a vehicle safety monitoring device in a rest mode, comprising: The data acquisition module is used to acquire time-series data of temperature and air composition inside the target vehicle when the target vehicle starts the nap mode and a person resting is detected inside the target vehicle, and to acquire time-series data of facial images of the person resting. A cabin environment recognition module is used to identify the cabin environment comfort of the target vehicle based on the temperature time series data and the air composition time series data. A physiological state recognition module is used to identify the physiological health status of the resting person based on the time-series data of the facial images; The safety risk assessment module is used to determine whether there is a safety risk to the resting personnel based on the comfort level of the cabin environment and the health level of their physiological state. The early warning module is used to issue an early warning to the resting personnel and activate the emergency plan when there is a safety risk.

[0016] On the other hand, embodiments of the present invention provide an electronic device, including: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the above-described vehicle safety monitoring method in a rest mode.

[0017] On the other hand, embodiments of the present invention also provide a computer-readable storage medium storing a processor-executable computer program that, when executed by a processor, implements the above-described vehicle safety monitoring method in a rest mode.

[0018] On the other hand, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the above-described vehicle safety monitoring method in a rest mode.

[0019] The advantages and beneficial effects of the present invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention: In this embodiment of the invention, when a target vehicle activates its nap mode and a person is detected resting inside, the system acquires time-series data on temperature and air composition within the vehicle, as well as time-series data on the resting person's facial image. Based on the temperature and air composition data, the system identifies the comfort level of the vehicle's cabin environment; based on the facial image data, it identifies the physiological health of the resting person. Finally, based on both cabin comfort and physiological health, the system determines whether the resting person poses a safety risk. If a safety risk is identified, an alert is issued and an emergency plan is activated. This embodiment of the invention identifies cabin comfort based on time-series temperature and air composition data, and identifies physiological health based on time-series facial image data. This allows for accurate assessment of whether a resting person poses a safety risk from both environmental and physiological perspectives. Targeted alerts and emergency responses are then implemented when a safety risk is detected, ensuring the safety of passengers resting in the vehicle during nap mode and improving overall vehicle safety. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments of the present invention are described below. It should be understood that the drawings described below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating the steps of a vehicle safety monitoring method in a rest mode, as provided in an embodiment of the present invention; Figure 2 This is a structural block diagram of a vehicle safety monitoring device in a rest mode provided by an embodiment of the present invention; Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of this invention; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this invention as detailed in the appended claims.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0024] The vehicle safety monitoring method in the nap mode provided in this invention can be applied to a terminal, a server, or software running on a terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application that implements the vehicle safety monitoring method in the nap mode, but is not limited to the above forms.

[0025] This invention can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0026] It should be noted that in various specific embodiments of the present invention, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user parking space location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of the present invention require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to a confirmation page. Only after obtaining the user's separate permission or consent is the necessary user-related data for the normal operation of the embodiments of the present invention acquired.

[0027] Reference Figure 1 This invention provides a vehicle safety monitoring method in a rest mode, specifically including the following steps: S101. When the target vehicle starts the nap mode and a person is detected resting inside the target vehicle, acquire the time-series data of temperature and air composition inside the target vehicle, and acquire the time-series data of the person's facial image. S102. Identify the cabin comfort of the target vehicle based on temperature time series data and air composition time series data; S103. Identify the physiological health status of resting personnel based on facial image time-series data; S104. Determine whether there is a safety risk to resting personnel based on the comfort of the cabin environment and their physiological health. S105. When there is a safety risk to resting personnel, issue an early warning to them and activate the emergency plan.

[0028] This invention identifies cabin comfort based on time-series data of temperature and air composition inside the vehicle, and identifies physiological health based on time-series data of facial images of resting passengers. This allows for accurate assessment of whether there is a safety risk to resting passengers from both the cabin environment and physiological state dimensions. Targeted warnings and emergency responses are provided when a safety risk is detected, ensuring the safety of passengers resting in the vehicle during nap mode and improving vehicle safety.

[0029] As a further optional implementation, time-series data of temperature and air composition inside the target vehicle are acquired, as well as time-series data of facial images of resting persons, specifically including: S1011. Temperature time-series data is acquired by multiple temperature sensors installed at different locations inside the target vehicle; S1012. Obtain air composition time-series data by installing a multi-gas sensor module in the target vehicle; S1013. Acquire facial image timing data through an in-vehicle camera installed in the target vehicle; When the rest mode is not interrupted, the temperature sensor, multi-gas sensor module and vehicle camera maintain intermittent acquisition mode. When the rest mode is interrupted, the temperature sensor, multi-gas sensor module and vehicle camera switch to real-time acquisition mode.

[0030] Specifically, the vehicle status signal is first obtained through the vehicle's CAN bus to confirm that the vehicle has started the rest mode (e.g., the gear is in P, the handbrake is engaged, and the multimedia system is switched to rest mode). At the same time, the in-vehicle infrared camera / visual sensor is activated, and the presence of a living person in the vehicle is identified through human detection algorithms (such as YOLO and Faster R-CNN) to rule out the possibility of the vehicle being accidentally triggered when empty.

[0031] In some optional embodiments, a dual verification structure of "seat pressure sensor + in-vehicle camera" can be used to detect whether there are resting people in the vehicle after the vehicle nap mode is turned on. The pressure sensor is set under the seat cushion of each seat in the vehicle to detect whether there is a pressure signal in the seat. The in-vehicle camera is set at the rearview mirror position to collect panoramic images of the vehicle interior and perform human contour recognition. When the pressure sensor detects a valid pressure signal and the camera recognizes the human contour, it is determined that there are resting people in the vehicle.

[0032] Temperature data is collected from different areas of the cabin (head, feet, and seats) by in-vehicle temperature sensors to generate time-series temperature data; O2 concentration, CO2 concentration, and VOC (volatile organic compound) content in the air are collected in real time by a multi-gas sensor module to generate time-series air composition data; and a high-definition in-vehicle vision camera is activated to capture facial images of resting passengers to generate time-series facial image data. The temperature sensor, multi-gas sensor module, and vehicle camera in this embodiment of the invention have two operating modes: intermittent acquisition mode (during nap mode operation) and real-time acquisition mode (switched on when nap mode is interrupted). This embodiment also includes a mode status monitoring module for monitoring the operating status of the vehicle's nap mode and determining whether the nap mode is interrupted. Interruption states include interruption due to a preset timeout and passive interruption caused by an air conditioning system malfunction. The mode status monitoring module communicates with the vehicle control unit (VCU) and air conditioning controller via the CAN bus. Only when the nap mode operating status is "terminated" or the air conditioning system operating status is "abnormally stopped" is the nap mode determined to be interrupted, and the temperature sensor, multi-gas sensor module, and vehicle camera are triggered to switch from intermittent acquisition mode to real-time acquisition mode.

[0033] As a further optional implementation, the cabin comfort of the target vehicle is identified based on time-series temperature data and time-series air composition data, specifically including: S1021. Predict the target cabin temperature at the next moment based on the temperature time series data, and predict the target air composition at the next moment based on the air composition time series data. S1022. Compare the target cabin temperature with the preset comfort temperature range to obtain a temperature fit score. S1023. Compare the target air composition with the preset air composition ratio to obtain an air compatibility score. S1024. Determine cabin environment comfort based on temperature adaptability score and air adaptability score.

[0034] Specifically, the target cabin temperature for the next moment is predicted using an exponentially weighted moving average algorithm based on temperature time-series data, and the target air composition for the next moment is predicted using an exponentially weighted moving average algorithm based on air composition time-series data. The target cabin temperature is compared with the temperature range suitable for human rest (22-26℃), and a temperature fit score is calculated based on the degree of temperature deviation. The target air composition is compared with the proportion of air components suitable for human rest, and an air fit score is calculated based on the degree of deviation of the proportion of each component. Finally, the cabin environment comfort is determined by weighted summation.

[0035] As a further optional implementation, the physiological health status of resting individuals is identified based on temporal data of facial images, specifically including: S1031. Extract the regional image time series data of the forehead region and / or cheek region based on the facial image time series data; S1032. Perform time-domain sampling on the red channel pixel values, green channel pixel values, and blue channel pixel values ​​of the regional image time-series data to obtain multi-channel light intensity signals; S1033. Input the multi-channel light intensity signal into the pre-trained heart rate recognition model to obtain the real-time heart rate of the resting person; S1034. Extract micro-motion data of several key facial points based on the temporal data of facial images. S1035. Perform time-domain analysis on the micro-motion data to obtain the respiratory frequency signal; S1036. Determine the real-time respiratory rate of resting personnel based on respiratory frequency band signals; S1037. Compare the real-time heart rate with the preset normal heart rate range to obtain a heart rate health score. S1038. Compare the real-time respiratory rate with the preset normal respiratory rate range to obtain a respiratory rate health score. S1039. Determine the physiological health status based on the heart rate health score and the respiratory rate health score.

[0036] Specifically, heartbeats cause periodic changes in subcutaneous blood volume in the face, which in turn cause slight fluctuations in skin color (the green channel is most sensitive to blood absorption). In this embodiment of the invention, regional image time-series data of the forehead and / or cheek areas are extracted from facial image time-series data. Then, the pixel values ​​of the red, green, and blue channels are sampled in the time domain to obtain multi-channel light intensity signals. A pre-trained heart rate recognition model is then used to identify the real-time heart rate of the person resting. During breathing, the rise and fall of the chest cavity causes slight periodic displacements in facial muscles (such as the nostrils and cheeks). By tracking the movement trajectories of these feature points, the breathing rhythm can be reconstructed. In this embodiment of the invention, micro-motion data of several key facial points are extracted from facial image time-series data. The micro-motion data is analyzed in the time domain to obtain respiratory frequency band signals. The real-time respiratory frequency of the person resting is determined based on the respiratory frequency band signals.

[0037] Specifically, algorithms such as MTCNN and RetinaFace are used to locate facial regions. Facial pose correction is achieved through key point detection (e.g., 68 facial key points) to eliminate the influence of angle shifts. Regions with rich blood supply and minimal motion interference, such as the forehead and cheeks, are selected as Regions of Interest (ROIs) to avoid interference from dynamic regions such as the eye area and mouth. Histogram equalization and gamma correction are used to eliminate the influence of ambient light changes on skin color and enhance signal stability. The red, green, and blue channel pixel values ​​of the ROIs are temporally sampled to obtain multi-channel light intensity signals. These multi-channel light intensity signals are then input into a pre-trained... A well-developed heart rate recognition model obtains the real-time heart rate of resting individuals; it tracks the sub-pixel displacement of key facial points (such as the sides of the nose and the jawline), recording their vertical movement trajectory; it performs time-domain analysis on the displacement data, using a bandpass filter to extract the respiratory frequency signal; it uses a peak detection algorithm to count the number of breaths per unit time, obtaining the respiratory rate; it compares the real-time heart rate with a preset normal heart rate range to obtain a heart rate health score; it compares the real-time respiratory rate with a preset normal respiratory rate range to obtain a respiratory rate health score; and it determines the physiological health status by weighted summation of the heart rate health score and the respiratory rate health score.

[0038] As an optional implementation, the heart rate recognition model is trained through the following steps: S201. Obtain time-series samples of the tester's facial images and determine the corresponding heart rate label based on the tester's measured heart rate. S202. Extract time-series regional image samples of the forehead region and / or cheek region based on the time-series facial image samples. S203. Perform temporal sampling on the red channel pixel values, green channel pixel values, and blue channel pixel values ​​of the temporal sample of the regional image to obtain multi-channel light intensity signal samples. S204. Input the multi-channel light intensity signal samples into a pre-constructed multi-branch convolutional neural network to obtain the predicted heart rate; S205. Determine the loss value based on the predicted heart rate and heart rate label; S206. Update the parameters of the multi-branch convolutional neural network based on the loss value using the backpropagation algorithm to obtain the trained heart rate recognition model.

[0039] Specifically, the process involves acquiring temporal samples of facial images of test subjects and determining corresponding heart rate labels based on their measured heart rates; extracting temporal samples of regional images of the forehead and / or cheek regions from these facial image temporal samples; performing temporal sampling on the red, green, and blue channel pixel values ​​of these regional image temporal samples to obtain multi-channel light intensity signal samples; inputting these multi-channel light intensity signal samples into a pre-constructed multi-branch convolutional neural network, extracting light intensity features of the red, green, and blue channels through different branches, fusing them through a feature fusion layer, and then mapping them through a fully connected layer to obtain the predicted heart rate; determining the loss value based on the predicted heart rate and heart rate label; updating the parameters of the multi-branch convolutional neural network based on the loss value using a backpropagation algorithm to complete one iteration of training; stopping training when the number of iterations reaches a preset threshold or the loss value falls below a preset threshold, thus obtaining the trained heart rate recognition model.

[0040] As a further optional implementation, the safety risk to resting personnel can be assessed based on cabin comfort and physiological health, specifically including: S1041. When the cabin environment comfort level is greater than or equal to the preset first threshold and the physiological health level is greater than or equal to the preset second threshold, it is determined that there is no safety risk to the resting personnel. S1042. When the cabin environment comfort level is less than the first threshold and the physiological health level is greater than or equal to the second threshold, it is determined that there is a slight safety risk to the resting personnel. S1043. When the cabin environment comfort level is greater than or equal to the first threshold and the physiological health level is less than the second threshold, it is determined that the resting personnel have a moderate safety risk. S1044. When the cabin environment comfort level is less than the first threshold and the physiological health level is less than the second threshold, it is determined that the resting personnel are at serious safety risk.

[0041] Specifically, safety thresholds for cabin environment comfort and physiological health are pre-set. When both are greater than or equal to the corresponding safety thresholds, it indicates that there is no safety risk to the resting personnel. When only cabin environment comfort is lower than the corresponding safety threshold, it indicates that the cabin environment is abnormal and the resting personnel face a slight safety risk. When only physiological health is lower than the corresponding safety threshold, it indicates that the resting personnel's physiological state is abnormal and the resting personnel face a moderate safety risk. When both are lower than the corresponding safety thresholds, it indicates that the resting personnel face a severe safety risk.

[0042] As an optional implementation method, an early warning system is issued to resting personnel and an emergency response plan is activated, which specifically includes: S1051. Match the corresponding voice prompt content according to the safety risk level, play the voice prompt content through the voice prompt module, and detect whether the resting personnel respond. S1052. If the resting personnel do not respond within the preset time period, adjust the air conditioning and / or windows of the target vehicle according to the comfort of the cabin environment, and send a request for help to the cloud platform according to the physiological health status.

[0043] Specifically, the vehicle's audio system is used to pre-store exclusive voice reminder content for different safety risk levels. When there is a safety risk to resting personnel, the corresponding voice reminder content will be continuously played at a medium volume (60-70 decibels). After the voice prompt module has been playing the warning information for a preset time (5 minutes), if no response from the person in the vehicle is detected (including pressing any physical button on the vehicle, touching the central control screen, or pulling the door handle), the emergency plan will be executed.

[0044] The emergency plan includes cabin environment adjustment and remote assistance. When the cabin environment comfort does not meet the corresponding safety threshold, the air conditioning and / or windows of the target vehicle will be adjusted according to the cabin environment comfort. When the physiological health status does not meet the corresponding safety threshold, an assistance message will be sent to the cloud platform through the 4G / 5G network. The assistance message includes the real-time physiological status of the resting personnel and the real-time location information of the vehicle.

[0045] The method steps of the embodiments of the present invention have been described above. It can be understood that the embodiments of the present invention identify cabin environment comfort based on time-series data of temperature and air composition within the vehicle, and identify physiological health based on time-series data of facial images of resting occupants. This allows for accurate assessment of whether there are safety risks associated with resting occupants from both the cabin environment and physiological state dimensions. Furthermore, targeted warnings and emergency responses are provided when safety risks are present, ensuring the safety of occupants resting in the vehicle during nap mode and improving vehicle safety.

[0046] Reference Figure 2This invention provides a vehicle safety monitoring device in a rest mode, comprising: The data acquisition module is used to acquire time-series data of temperature and air composition inside the target vehicle when the target vehicle starts the nap mode and the presence of resting persons inside the target vehicle is detected, as well as time-series data of facial images of the resting persons. The cabin environment recognition module is used to identify the cabin environment comfort of the target vehicle based on temperature time series data and air composition time series data. The physiological state recognition module is used to identify the physiological health status of resting personnel based on the time-series data of facial images. The safety risk assessment module is used to determine whether there is a safety risk to resting personnel based on the comfort of the cabin environment and their physiological health. The early warning module is used to issue warnings to resting personnel and activate emergency plans when there is a safety risk.

[0047] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0048] Reference Figure 3 This invention provides an electronic device, comprising: At least one processor; At least one memory for storing at least one program; When the above-mentioned at least one program is executed by the above-mentioned at least one processor, the above-mentioned at least one processor implements the above-mentioned vehicle safety monitoring method in a rest mode.

[0049] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0050] This invention also provides a computer-readable storage medium storing a processor-executable computer program that, when executed by a processor, implements the above-described vehicle safety monitoring method in a nap mode.

[0051] This invention provides a computer-readable storage medium that can execute a vehicle safety monitoring method in a rest mode provided in the method embodiment of this invention. It can execute any combination of the implementation steps of the method embodiment and has the corresponding functions and beneficial effects of the method.

[0052] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described vehicle safety monitoring method in a nap mode.

[0053] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0054] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0055] The embodiments described in this invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0056] The terms "first," "second," "third," "fourth," etc. (if present) in the specification and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0057] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the aforementioned blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0058] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the aforementioned functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0059] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0060] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0061] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the aforementioned program can be printed, because the aforementioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0062] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0063] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0064] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0065] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A method for vehicle safety monitoring in a rest mode, characterized in that, Includes the following steps: When the target vehicle starts the nap mode and a person is detected resting inside the target vehicle, the time-series data of temperature and air composition inside the target vehicle are acquired, and the time-series data of the person resting is acquired. The cabin comfort of the target vehicle is identified based on the time-series temperature data and the time-series air composition data. The physiological health status of the resting person is identified based on the temporal data of the facial images; Based on the cabin environment comfort and the physiological health status, determine whether there is a safety risk to the resting personnel; If there is a safety risk to the resting personnel, an early warning will be issued to the resting personnel and an emergency plan will be activated.

2. The vehicle safety monitoring method in a rest mode according to claim 1, characterized in that, The acquisition of time-series data on temperature and air composition inside the target vehicle, and the acquisition of time-series data on facial images of the resting person, specifically includes: The temperature time-series data is acquired by multiple temperature sensors installed at different locations inside the target vehicle. The air composition time-series data is acquired by a multi-gas sensor module installed in the target vehicle; The facial image time-series data is acquired by an onboard camera installed in the target vehicle. When the rest mode is not interrupted, the temperature sensor, the multi-gas sensor module, and the vehicle camera maintain intermittent acquisition mode. When the rest mode is interrupted, the temperature sensor, the multi-gas sensor module, and the vehicle camera switch to real-time acquisition mode.

3. The vehicle safety monitoring method in a rest mode according to claim 1, characterized in that, The step of identifying the cabin comfort of the target vehicle based on the temperature time-series data and the air composition time-series data specifically includes: Predict the target cabin temperature at the next moment based on the temperature time series data, and predict the target air composition at the next moment based on the air composition time series data; The target cabin temperature is compared with the preset comfort temperature range to obtain a temperature fit score. The target air composition is compared with a preset air composition ratio to obtain an air compatibility score. The cabin environment comfort level is determined based on the temperature compatibility score and the air compatibility score.

4. The vehicle safety monitoring method in a rest mode according to claim 1, characterized in that, The step of identifying the physiological health status of the resting person based on the temporal data of the facial images specifically includes: Extract the regional image time-series data of the forehead region and / or cheek region based on the facial image time-series data; The red channel pixel values, green channel pixel values, and blue channel pixel values ​​of the time-series data of the regional image are sampled in the time domain to obtain multi-channel light intensity signals; The multi-channel light intensity signal is input into a pre-trained heart rate recognition model to obtain the real-time heart rate of the resting person; Micro-motion data of several key facial points are extracted based on the temporal data of the facial images. Time-domain analysis was performed on the micro-motion data to obtain the respiratory frequency band signal; The real-time respiratory rate of the resting person is determined based on the respiratory frequency band signal; The real-time heart rate is compared with a preset normal heart rate range to obtain a heart rate health score. The real-time respiratory rate is compared with a preset normal respiratory rate range to obtain a respiratory rate health score. The physiological health status is determined based on the heart rate health score and the respiratory rate health score.

5. A vehicle safety monitoring method in a rest mode according to claim 4, characterized in that, The heart rate recognition model is trained through the following steps: Acquire temporal samples of the tester's facial images and determine the corresponding heart rate label based on the tester's measured heart rate; Based on the facial image temporal samples, extract the regional image temporal samples of the forehead region and / or cheek region; The red channel pixel values, green channel pixel values, and blue channel pixel values ​​of the temporal sample of the regional image are sampled in the temporal domain to obtain multi-channel light intensity signal samples. The multi-channel light intensity signal samples are input into a pre-constructed multi-branch convolutional neural network to obtain the predicted heart rate. The loss value is determined based on the predicted heart rate and the heart rate label; The parameters of the multi-branch convolutional neural network are updated using the backpropagation algorithm based on the loss value to obtain the trained heart rate recognition model.

6. A vehicle safety monitoring method in a rest mode according to claim 1, characterized in that, The determination of whether there is a safety risk to the resting personnel based on the cabin environment comfort and the physiological health status specifically includes: When the cabin environment comfort level is greater than or equal to a preset first threshold and the physiological health level is greater than or equal to a preset second threshold, it is determined that the resting personnel do not pose a safety risk. When the cabin environment comfort level is less than the first threshold and the physiological health level is greater than or equal to the second threshold, it is determined that the resting person has a slight safety risk. When the cabin environment comfort level is greater than or equal to the first threshold and the physiological health level is less than the second threshold, it is determined that the resting person has a moderate safety risk. If the cabin environment comfort level is less than the first threshold and the physiological health level is less than the second threshold, it is determined that the resting person has a serious safety risk.

7. A vehicle safety monitoring method in a rest mode according to any one of claims 1 to 6, characterized in that, The specific steps of issuing an early warning and activating the emergency plan for the resting personnel include: The system matches the corresponding voice prompt content according to the safety risk level, plays the voice prompt content through the voice prompt module, and detects whether the resting person responds. If the resting personnel do not respond within a preset time period, the air conditioning and / or windows of the target vehicle will be adjusted according to the comfort level of the cabin environment, and a distress message will be sent to the cloud platform according to the physiological health status.

8. A vehicle safety monitoring device in a rest mode, characterized in that, include: The data acquisition module is used to acquire time-series data of temperature and air composition inside the target vehicle when the target vehicle starts the nap mode and a person resting is detected inside the target vehicle, and to acquire time-series data of facial images of the person resting. A cabin environment recognition module is used to identify the cabin environment comfort of the target vehicle based on the temperature time series data and the air composition time series data. A physiological state recognition module is used to identify the physiological health status of the resting person based on the time-series data of the facial images; The safety risk assessment module is used to determine whether there is a safety risk to the resting personnel based on the comfort level of the cabin environment and the health level of their physiological state. The early warning module is used to issue an early warning to the resting personnel and activate the emergency plan when there is a safety risk.

9. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a vehicle safety monitoring method in a rest mode as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements a vehicle safety monitoring method in a rest mode as described in any one of claims 1 to 7.

Images