Vehicle personnel health state identification method and device, electronic equipment and storage medium
By identifying the lip color, chest movement and eye-opening status of the person on the car, combining vehicle status information, judging the health risk level and performing auxiliary operations, the problem of large error in judging the health status of smart vehicles is solved, and the accuracy of health assistance is improved and timely.
Patent Information
- Application Number
- CN202411072591.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-11-12
AI Technical Summary
When judging the health status of the existing smart vehicles, there is a large error, which leads to the inability to promptly remind and rescue, affecting the health and driving safety of the personnel in the vehicle.
By obtaining personnel images, identifying lip color, chest movement and eye-closing status, combining vehicle status information, determine the health risk level and perform corresponding health assisted operations.
It improves the accuracy of judging the health status of the people on the car and the accuracy and pertinence of health auxiliary operations, ensuring timely responses when health problems arise.
Smart Images

Figure CN118942077A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of intelligent vehicle technology, and in particular to a method, device, electronic device and storage medium for identifying the health status of people on board a vehicle. Background Art
[0002] As cars become more and more popular, more and more people will travel by car. In some driving scenarios, people in the car may suffer from hypoxia and physical discomfort due to the high altitude and low oxygen content. Or they may suffer from nausea, vomiting and other physical discomfort due to motion sickness.
[0003] Current smart vehicles can use in-car cameras to capture images of people, identify the images, and determine whether the person is feeling unwell, and then execute response strategies such as outputting warning information. However, judging health status based solely on images can result in large errors. For example, factors such as unclear images and large differences in images of different people can lead to errors in judgment, and the people on the vehicle cannot receive timely reminders and assistance, affecting the health of the people on the vehicle and the driving safety of the vehicle. Summary of the invention
[0004] In view of this, in order to solve some or all of the above-mentioned technical problems, the embodiments of the present application provide a method, device, electronic device and computer-readable storage medium for identifying the health status of people on board.
[0005] In a first aspect, an embodiment of the present application provides a method for identifying the health status of a person on a vehicle, the method comprising: obtaining a person image taken of a target person on the vehicle; performing person status identification on the person image to obtain person health status information; based on the person health status information, determining whether the target person is currently in an unhealthy state; if the target person is in an unhealthy state, obtaining vehicle status information of the vehicle; based on the health status information and the vehicle status information, determining the current health risk level of the target person; and executing pre-set health assistance operations corresponding to the health risk level.
[0006] In one possible implementation, person status recognition is performed on a person image to obtain person health status information, including: determining a lip area from the person image, performing lip color recognition on the lip area, and obtaining lip color status information; and / or determining a chest area from the person image, and performing thorax movement recognition on the chest area, and obtaining thorax movement status information; and / or determining an eye area from the person image, performing eye opening and closing status recognition on the eye area, and obtaining eye opening and closing status information; and determining at least one of the lip color status information, thorax movement status information, and eye opening and closing status information as the person health status information.
[0007] In one possible implementation, based on the health status information of the person, determining whether the target person is currently in an unhealthy state includes: determining a corresponding first health score based on the lip color status information; and / or determining a corresponding second health score based on the chest movement status information; and / or determining a duration of eye closure based on the eye opening and closing status information; determining whether the health status information of the person meets preset health conditions based on at least one of the first health score, the second health score and the duration of eye closure, and if not, determining that the target person is currently in an unhealthy state.
[0008] In one possible implementation, obtaining vehicle status information of a vehicle includes: determining altitude information of a current location of the vehicle, and determining the altitude information as vehicle status information; determining a current health risk level of a target person based on health status information and vehicle status information, including: determining an altitude level corresponding to the altitude information; and determining the health risk level based on the altitude level and the health status information of the person.
[0009] In one possible implementation, executing a pre-set health assistance operation corresponding to the health risk level includes: outputting health assistance prompt information; receiving feedback information generated by feedback on the health assistance prompt information; and executing a pre-set health assistance operation corresponding to the health risk level based on the feedback information.
[0010] In one possible implementation, based on the feedback information, a pre-set health assistance operation corresponding to the health risk level is executed, including: if the feedback information indicates that the target person needs help, determine the target help terminal, and send help information to the target help terminal; if the feedback information indicates that the target person does not need help and the health risk level is the first level, stop outputting health assistance prompt information; if the feedback information indicates that the target person does not need help and the health risk level is not the first level, periodically output health assistance prompt information according to the preset prompt period corresponding to the health risk level.
[0011] In one possible implementation, health auxiliary prompt information is periodically output according to a preset prompt period corresponding to the health risk level, including: if the health risk level is the second level, health auxiliary prompt information is periodically output according to a preset first prompt period; if the health risk level is the third level, health auxiliary prompt information is periodically output according to a preset second prompt period, wherein the second prompt period is smaller than the first prompt period.
[0012] In one possible implementation, after outputting the health assistance prompt information, the method also includes: if no feedback information is received within a first preset time period, determining the duration of eye closure from the health status information of the person, wherein the duration of eye closure is obtained in advance by identifying the open and closed eyes state of the eye area in the person's image; if the duration of eye closure is greater than or equal to a second preset time period, determining the target help terminal, and sending a help message to the target help terminal.
[0013] In a second aspect, an embodiment of the present application provides a device for identifying the health status of people on a vehicle, the device comprising: a first acquisition module, used to acquire a person image obtained by photographing a target person on a vehicle; an identification module, used to perform person status identification on the person image to obtain the person's health status information; a first determination module, used to determine whether the target person is currently in an unhealthy state based on the person's health status information; a second acquisition module, used to acquire vehicle status information of the vehicle if the target person is in an unhealthy state; a second determination module, used to determine the current health risk level of the target person based on the health status information and the vehicle status information; and an execution module, used to execute pre-set health-assist operations corresponding to the health risk level.
[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory for storing a computer program; a processor for executing the computer program stored in the memory, and when the computer program is executed, it implements the method of any embodiment of the method for identifying the health status of a person on board a vehicle of the first aspect of the present application.
[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, a method of any embodiment of the method for identifying the health status of a person on board a vehicle as described in the first aspect above is implemented.
[0016] In a fifth aspect, an embodiment of the present application provides a computer program, which includes a computer-readable code. When the computer-readable code runs on a device, the processor in the device implements a method as described in any embodiment of the method for identifying the health status of a person on board a vehicle according to the first aspect above.
[0017] The embodiments of the present application provide a method, device, electronic device, and computer-readable storage medium for identifying the health status of people on board. By acquiring a person image taken of a target person on the vehicle, the person image is identified to obtain the person's health status information. When the person's health status information indicates an unhealthy state, the vehicle status information of the vehicle is acquired. Based on the health status information and the vehicle status information, the current health risk level of the target person is determined, and health assistance operations corresponding to the health risk level are performed. The embodiments of the present application combine images and vehicle status to judge the health status of people on board. The judgment of health status is more accurate, and the accuracy and pertinence of the health assistance operations performed are higher, thereby helping to make timely and accurate responses when health problems occur to people on board. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0020] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.
[0021] Figure 1 A flowchart of a method for identifying the health status of a person on board provided in an embodiment of the present application;
[0022] Figure 2 A flowchart of another method for identifying the health status of a person on board provided in an embodiment of the present application;
[0023] Figure 3 A flowchart of another method for identifying the health status of a person on board provided in an embodiment of the present application;
[0024] Figure 4 A flowchart of another method for identifying the health status of a person on board provided in an embodiment of the present application;
[0025] Figure 5 A flowchart of another method for identifying the health status of a person on board provided in an embodiment of the present application;
[0026] Figure 6 A schematic diagram of the structure of a device for identifying the health status of a person on board provided in an embodiment of the present application;
[0027] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0028] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. It should be noted that unless otherwise specifically stated, the relative arrangement of the components and steps, the numerical expressions and the numerical values described in these embodiments do not limit the scope of the present application.
[0029] Those skilled in the art will understand that the terms "first" and "second" in the embodiments of the present application are only used to distinguish between different steps, devices, modules and other objects, and do not represent any specific technical meanings, nor do they indicate the logical order between them.
[0030] It should also be understood that in this embodiment, “plurality” may refer to two or more than two, and “at least one” may refer to one, two or more than two.
[0031] It should also be understood that any component, data or structure mentioned in the embodiments of the present application can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.
[0032] In addition, the term "and / or" in this application is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this application generally indicates that the associated objects before and after are in an "or" relationship.
[0033] It should also be understood that the description of the various embodiments in this application focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.
[0034] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present application, its application, or uses.
[0035] Technologies, circuits, and devices known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the above-mentioned technologies, circuits, and devices should be considered as part of the specification.
[0036] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.
[0037] It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other. To facilitate the understanding of the embodiments of the present application, the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0038] In order to solve the technical problem of low accuracy in identifying the health status of occupants in the prior art, the present application provides a method for identifying the health status of occupants in a vehicle, which can improve the accuracy of identifying the health status of occupants in the vehicle and the accuracy of responding to the health status.
[0039] Figure 1 A flow chart of a method for identifying the health status of a person on a vehicle provided in an embodiment of the present application. The method can be applied to a vehicle and can be executed by a controller on the vehicle. In addition, the method can also be executed by other electronic devices, such as smart phones, laptops, desktop computers, portable computers, servers, etc. These devices can be connected to the vehicle for communication, receive images and other information uploaded by the vehicle, and after executing the method, send instructions to the vehicle to perform health assistance operations.
[0040] In addition, the execution subject of the present method may be hardware or software. When the execution subject is hardware, the execution subject may be one or more of the electronic devices. For example, a single electronic device may execute the present method, or a plurality of electronic devices may cooperate with each other to execute the present method. When the execution subject is software, the present method may be implemented as a plurality of software or software modules, or as a single software or software module. No specific limitation is made herein.
[0041] like Figure 1 As shown, the method specifically includes:
[0042] Step 101, obtaining a person image captured of a target person in a vehicle.
[0043] In this embodiment, the target person may be a person in any seat in the vehicle. For example, the driver is the target person, and a camera is provided in front of the driver's seat to photograph the driver, and the image obtained is the above-mentioned person image. For another example, a camera may also be provided in front of the back seat, and the camera photographs the person in the back seat to obtain the above-mentioned person image.
[0044] Step 102: Perform personnel status recognition on the personnel image to obtain personnel health status information.
[0045] In this embodiment, various parts of a person in a person image can be identified according to an image recognition method. For example, facial recognition can be performed on a person image to identify the facial color features, expression features, etc. of the target person, and based on these features, the health status information of the person is generated. For another example, the chest rise and fall movement can be identified on a person image to determine features such as the chest rise and fall amplitude and frequency, and based on these features, the health status of the target person is determined to generate the health status information of the person. Generally, the health status information of a person can be represented by quantitative data, that is, the health status of the target person can be determined through the health status information of the person.
[0046] The above-mentioned method for recognizing an image can be implemented by using relevant image recognition technology in the art. For example, a neural network model trained based on a machine learning method can be used for image recognition.
[0047] Step 103: Based on the personnel health status information, determine whether the target person is currently in an unhealthy state.
[0048] In this embodiment, the health status information of the person may include quantified data, through which the data may be compared with the data in the health status, so as to determine whether the target person is in an unhealthy state.
[0049] For example, the personnel health status information may include a score indicating the health status of the target person. If the score is less than or equal to a preset score threshold, it is determined that the target person is currently in an unhealthy state.
[0050] Step 104: If the target person is in an unhealthy state, obtain the vehicle status information of the vehicle.
[0051] In this embodiment, the vehicle status information may include various types of status of the vehicle, such as the current altitude of the vehicle, the temperature inside the vehicle, the acceleration of the vehicle, the turning angle, etc.
[0052] Step 105, determining the current health risk level of the target person based on the health status information and the vehicle status information.
[0053] In this embodiment, multiple health risk levels can be preset, and each health risk level corresponds to a certain range of personnel health status information and vehicle status information. Personnel health status information and vehicle status information can be combined to represent personnel health status at different levels.
[0054] As an example, the magnitude of the vehicle's acceleration will affect the health status of the personnel, that is, the greater the acceleration, the greater the probability of motion sickness in the vehicle. Therefore, the acceleration can be used as a parameter for dividing the health risk level, and the data included in the personnel health status information can be calculated such as weighted summation to obtain the health score, and then the corresponding health risk level can be determined according to the health score.
[0055] For another example, the steering wheel angle and vehicle speed will also affect the health status of the person, that is, the greater the steering wheel angle and vehicle speed, the greater the probability of motion sickness in the car. Therefore, the steering wheel angle and vehicle speed can also be used as parameters for dividing the health risk level, and the data included in the health status information of the person can be calculated such as weighted summation to obtain the health score, and then the corresponding health risk level can be determined according to the health score.
[0056] Step 106, executing a preset health assistance operation corresponding to the health risk level.
[0057] In this embodiment, the health assistance operation may include various operation modes such as voice prompts and telephone assistance. Different health risk levels may correspond to different health assistance operation modes. For example, if the health risk level indicates that the health status of the target person is better, only the voice prompt information may be output. If the health risk level indicates that the health status of the target person is poor, the voice prompt information may be output first, and the target user feedback needs help, then the help information may continue to be sent to the emergency contact or vehicle service center. If the health risk level indicates that the health status of the target person is worse, the help information may be sent directly to the emergency contact or vehicle service center.
[0058] The method for identifying the health status of people on board provided in the embodiment of the present application obtains a person image taken of a target person on the vehicle, identifies the person status of the person image, obtains the person health status information, obtains the vehicle status information of the vehicle when the person health status information indicates an unhealthy state, determines the current health risk level of the target person based on the health status information and the vehicle status information, and performs health assistance operations corresponding to the health risk level. The embodiment of the present application realizes the combination of images and vehicle status to judge the health status of people on board, the judgment of health status is more accurate, and the accuracy and pertinence of the health assistance operations performed are higher, which helps to make timely and accurate responses when health problems occur to people on board.
[0059] In some optional implementations of this embodiment, in step 102, the person status may be identified on the person image in at least one of the following ways to obtain the person health status information:
[0060] Method 1: determine the lip area from the person image, perform lip color recognition on the lip area, and obtain lip color status information.
[0061] Usually, when the human body is in poor health, the color of the lips tends to change significantly. For example, when the oxygen content in the outside world is low, the human body will be hypoxic, causing the skin and mucous membranes to usually turn purple, and this phenomenon is mainly concentrated in the lips.
[0062] The electronic device can collect facial information from the image of the person and locate the lip area, and extract the color change of the occupant's lips by executing a preset algorithm. The specific process is: extract the picture captured by the camera, first locate the lips, and determine the coordinates of the lip position. Then execute the algorithm in the lip area (for example, an edge detection algorithm can be used to determine the lip edge, and a color space change algorithm can be used to extract the color of the pixels within the lip edge) to extract color information. The extracted color information can be used as the lip color status information.
[0063] Method 2: determine the chest area from the person image, and perform chest cage motion recognition on the chest area to obtain chest cage motion state information.
[0064] Specifically, the electronic device can use a target detection algorithm to locate the chest area in the image of the person. Then, the chest motion monitoring algorithm is used to detect changes in the target person's breathing state. Specifically, the chest position corresponding to the continuous multiple-frame personnel images obtained in the same way as this step over a period of time can be obtained, and the chest fluctuation amplitude and fluctuation frequency in the continuous multiple-frame personnel images can be analyzed to infer the breathing state. By executing the chest motion monitoring algorithm, the outline of the chest can be identified, and its position changes between different frames can be tracked. The obtained chest fluctuation amplitude and fluctuation frequency can be used as chest motion state information.
[0065] Method three, determining the eye area from the person image, performing eye opening and closing state recognition on the eye area, and obtaining eye opening and closing state information.
[0066] Specifically, the electronic device can use the target detection algorithm to locate the eye area in the image of the person and perform eye opening and closing detection on the eye area. The specific process is to extract eye features from the eye area, such as the outline, shape, grayscale distribution, etc. of the eye, and determine whether the eyes are open by analyzing these feature points. For example, the distance between the upper eyelid and the lower eyelid can be determined by extracting the eye key points, and the eye opening and closing status information of the target person can be generated based on the distance.
[0067] Finally, at least one of the lip color status information, chest movement status information, and eye opening and closing status information is determined as the personnel health status information.
[0068] It should be noted that the specific information used as the personnel health status information can be preset or selected according to the actual situation of image recognition. For example, if any of the lip area, chest area and eye area fails to be recognized due to poor image quality, the recognized status information corresponding to the area can be deleted, and the status information corresponding to other areas can be used as the personnel health status information.
[0069] Optionally, the vehicle's on-board system can collect images of the person, and collect lip color status information, chest movement status information, and eye opening and closing status information, and upload this information to the server. The server can use the collected information as training samples to continuously optimize the above-mentioned lip color detection algorithm, chest movement monitoring algorithm, and eye opening and closing detection algorithm, and continuously improve the accuracy of determining the health status information of the person.
[0070] This embodiment extracts lip color status information, chest movement status information, and eye opening and closing status information from the person's image, and generates person health status information based on this information, thereby realizing multi-faceted health status identification of the target person. The generated person health status information can represent the person's health status from multiple dimensions, which helps to improve the accuracy of the person's health status judgment.
[0071] In some optional implementations of this embodiment, based on the lip color state information, the chest movement state information, and the eye opening and closing state information in the above embodiment, the above step 103 may include:
[0072] Based on the lip color status information, determine the corresponding first health score; and / or, based on the thoracic movement status information, determine the corresponding second health score; and / or, based on the eye opening and closing status information, determine the duration of eye closure.
[0073] Specifically, the size of the first health score can reflect the health of the target person. Usually, it can be set that the larger the first health score is, the healthier the target person is. The first health score can be obtained by statistically analyzing the lip color changes of multiple consecutive frames of images of the person. For example, the degree to which the lip color changes from normal to purple can be determined by comparing the lip color status information of multiple consecutive frames of images of the person. The change range between the color value of the lips in the purple state and the color value in the normal state is converted into a health score within a preset range (e.g., 0-100%). The parameters used to convert the color change range into a health score can be pre-calibrated, that is, the lip color change range of the person in a healthy state and an unhealthy state is analyzed, and the conversion coefficient is set according to the color change range and the actual health status.
[0074] The size of the above-mentioned second health score can reflect the health of the target person. Usually, it can be set that the larger the second health score is, the healthier the target person is. The second health score can be obtained by statistically analyzing the changes in the chest state of multiple consecutive frames of personnel images. Changes in the chest state can reflect changes in the respiratory state, and then reflect the health status of the person in the respiratory dimension, especially the health status under hypoxic conditions. For example, the degree of change of the chest fluctuation amplitude and the fluctuation frequency can be determined by comparing the chest fluctuation amplitude and the fluctuation frequency of multiple consecutive frames of personnel images, and the degree of change can be converted into a health score within a preset range (for example, 0-100%). The parameters used to convert the degree of change of the chest movement state into a health score can be pre-calibrated, that is, the degree of change of the chest movement state of the person in a healthy state and an unhealthy state is analyzed, and the conversion coefficient is set according to the degree of change and the actual health state.
[0075] The above-mentioned eye-closing duration can be obtained by collecting statistics on the changes in the eye-opening and eye-closing states of multiple consecutive frames of images of the person. That is, the eye-opening and eye-closing state information of multiple consecutive frames of images of the person is recorded, and if the eye-closing state is identified from the multiple consecutive frames of images of the person, the duration of the multiple consecutive frames of images of the person is used as the eye-closing duration.
[0076] Finally, based on at least one of the first health score, the second health score and the duration of eye closure, determine whether the health status information of the person meets the preset health conditions. If not, determine that the target person is currently in an unhealthy state.
[0077] Specifically, the first health score, the second health score and the duration of closing eyes can correspond to different health conditions respectively, or the corresponding health conditions can be set in any combination. For example, when the first health score is less than or equal to the first threshold, and / or when the second health score is less than or equal to the second threshold, and / or when the duration of closing eyes is greater than or equal to the preset duration threshold, it is determined that the health status information of the person does not meet the preset health conditions.
[0078] It should be noted that the types of information combinations corresponding to the above health conditions can be set arbitrarily. For example, when the sum of the first health score and the second health score is less than or equal to the third threshold, it can be determined that the health status information of the person does not meet the preset health condition.
[0079] This embodiment uses at least one of the first health score, the second health score and the duration of eyes closing to determine whether the target person is in an unhealthy state, thereby using data from multiple dimensions to comprehensively determine whether the target person is healthy, thereby improving the accuracy of health status judgment.
[0080] In some optional implementations of this embodiment, such as Figure 2 As shown, step 104 includes:
[0081] Step 1041, determine the altitude information of the current location of the vehicle, and determine the altitude information as the vehicle status information.
[0082] Specifically, the altitude information can be obtained in various ways, for example, it can be measured by an altitude measuring instrument on the vehicle, or the current position of the vehicle can be obtained by satellite positioning, and the pre-recorded altitude information corresponding to the position can be obtained.
[0083] Step 105 includes:
[0084] Step 1051, determining the altitude level corresponding to the altitude information.
[0085] Among them, different altitude levels correspond to different altitude ranges. For example, the altitude can be divided into four levels, namely below 2000 meters, 2000-3000 meters, 3000-4000 meters, and above 4000 meters, respectively recorded as h1, h2, h3, and h4.
[0086] Step 1052, determining the health risk level according to the altitude level and the personnel health status information.
[0087] Specifically, due to different altitudes, the corresponding oxygen content in the air is different, that is, the higher the altitude, the lower the oxygen content in the air, and the oxygen content in the air will affect the health status of the personnel. Therefore, when the personnel health status information indicates that the target personnel has a health risk, the oxygen content in the air can be further estimated according to the altitude level, and then the health risk level can be determined. That is, the higher the altitude level, the higher the health risk level, and the worse the health status of the target personnel.
[0088] As an example, the altitude levels can be divided to obtain health risk levels. For example, if the altitude level is h1 or h2, the health risk level is the first level, that is, the severity of hypoxia is relatively mild; if the altitude level is h3, the health risk level is the second level, that is, the severity of hypoxia is slightly higher; if the altitude level is h4, the health risk level is the third level, that is, the severity of hypoxia is the highest.
[0089] For another example, the altitude level can also be used as a parameter for dividing the health risk level, and calculations such as weighted summation can be performed with the data included in the personnel health status information to obtain the health risk value, and then the health risk level corresponding to the health risk value can be determined based on the correspondence between the pre-calibrated health risk value and the health risk level.
[0090] This embodiment obtains the altitude information of the current location of the vehicle, determines the health risk level according to the altitude information, and uses the altitude as a basis for judging the health status, thereby enabling judgment of the hypoxia status of the person. In high-altitude scenarios, it can accurately judge whether the target person is feeling unwell due to hypoxia, thereby enabling timely and accurate auxiliary operations on the hypoxia status, reducing the damage of hypoxia to the human body, and improving the driving safety of the vehicle in high-altitude scenarios.
[0091] In some optional implementations of this embodiment, such as Figure 3 As shown, step 106 includes:
[0092] Step 1061, output health assistance prompt information.
[0093] Among them, the health assistance prompt information can be output in various ways, including playing voice prompt information, displaying prompt information on the screen, etc.
[0094] Step 1062, receiving feedback information generated in response to the health assistance prompt information.
[0095] The feedback information may also be generated in various ways, for example, the feedback information may be voice feedback information, that is, the person on the vehicle makes a voice and the electronic device recognizes the voice. Alternatively, the feedback information may be information generated by the person on the vehicle clicking an option on the screen.
[0096] Step 1063: Based on the feedback information, execute a pre-set health assistance operation corresponding to the health risk level.
[0097] Among them, health assistance operations are related to health risk levels and feedback information, and the correspondence between the three can be preset. That is, when the feedback information indicates that the target person does not need help, a health assistance operation indicating that no help is needed can be performed; when the feedback information indicates that the target person needs help, the corresponding health assistance operation can be performed according to the health risk level. For example, when the health risk level is high, a health assistance operation to ask an emergency contact or customer service for help is performed, and when the health risk level is low, health assistance prompt information is continuously output. Alternatively, health assistance prompt information can be output in different ways according to different health risk levels (for example, when the risk level is low, a pop-up window displays prompt information, and when the risk level is high, voice prompt information is played).
[0098] This embodiment outputs health assistance prompt information, receives feedback information, and performs health assistance operations based on the feedback information and health risk levels. It realizes the execution of health assistance operations based on interaction with people on the vehicle, improves the targeted nature of health assistance operations, and helps to provide assistance services to target personnel more accurately.
[0099] In some optional implementations of this embodiment, such as Figure 4 As shown, the above step 1063 includes:
[0100] Step 10631: If the feedback information indicates that the target person needs help, determine the target help terminal and send help information to the target help terminal.
[0101] The type of the target help-seeking terminal can be manually set in advance or set by default. For example, the target help-seeking terminal can be a pre-set emergency contact. If the target person needs help, a help-seeking message is sent to the terminal used by the emergency contact. Alternatively, the target help-seeking terminal can be a vehicle customer service center. If the target person needs help, a help-seeking message is sent to the vehicle customer service center, and the vehicle customer service center dials a medical rescue call on behalf of the target person and notifies the rescue. The help-seeking message can also include other auxiliary information, such as the current location of the vehicle, the current health status information of the user, etc.
[0102] Step 10632: If the feedback information indicates that the target person does not need help and the health risk level is the first level, stop outputting health assistance prompt information.
[0103] The first level may be the lowest level of health risk. That is, when the health risk level is the first level, the target person is considered to have mild symptoms, and health assistance prompt information may be output only once. As an example, referring to the altitude level, the health risk level may be determined to be the first level when the altitude levels are h1 and h2.
[0104] As an example, if the health risk level is the first level, the central control screen on the vehicle will display a health assistance prompt message in a pop-up window, "The system has detected that you are suffering from hypoxia. Are you feeling unwell?" At the same time, the prompt message will be played by voice and the options "Yes" or "No" will be displayed below the pop-up window. If the user clicks "No", indicating that the user does not need help, the pop-up window will be closed and no more prompts will be given.
[0105] If the user clicks "Yes" or confirms by voice, indicating that the user needs help, the prompt message "Do you need to contact an emergency contact or medical rescue service?" will continue to be displayed on the vehicle's central control screen, and the prompt message will be played by voice, and options will be displayed below the pop-up window: "Contact an emergency contact" or "Need medical rescue service". If anyone in the car clicks or voice triggers "Contact an emergency contact" or "Need medical rescue service", it means that the target person needs help. That is, if the user clicks "Contact an emergency contact", the emergency contact number preset in the car will be automatically dialed, and the current vehicle location information will be automatically sent to the emergency contact's mobile phone. If the user clicks "Need medical rescue service", the vehicle and user's current status information (vehicle location, whether the user is awake) will be uploaded to the vehicle customer service center, which will dial the medical rescue phone on their behalf and notify the rescue.
[0106] Step 10633, if the feedback information indicates that the target person does not need help and the health risk level is not the first level, health assistance prompt information is periodically output according to the preset prompt cycle corresponding to the health risk level.
[0107] For example, if the health risk level is not the first level, it means that the health status of the target person is poor. The central control screen on the vehicle displays a health assistance prompt message in a pop-up window, "The system detects that you are hypoxic. Are you feeling unwell?" At the same time, the prompt message is played by voice, and the options "Yes" or "No" are displayed below the pop-up window. If the user clicks "No", the pop-up window is closed, and then after a prompt cycle, the above health assistance prompt message is output again to ensure the safety of the target person.
[0108] This embodiment interacts with the user to output health assistance prompt information in different ways when the target person's symptoms are mild or severe, thereby achieving targeted processing based on the target person's health status, thereby ensuring that the target person can be accurately prompted and ensuring the target person's safety.
[0109] In some optional implementations of this embodiment, step 10633 includes:
[0110] If the health risk level is the second level, the health auxiliary prompt information is periodically output according to the preset first prompt period; if the health risk level is the third level, the health auxiliary prompt information is periodically output according to the preset second prompt period, wherein the second prompt period is shorter than the first prompt period.
[0111] Among them, the third level indicates that the health status of the target person is worse than the second level. As an example, referring to the above altitude levels, when the altitude level is h3, the health risk level can be determined to be the second level, and when the altitude level is h4, the health risk level can be determined to be the third level. When the health status of the target person is worse, health auxiliary prompt information is output at a higher prompt frequency, so as to ensure that the target person can get help in time and reduce health risks.
[0112] In some optional implementations of this embodiment, such as Figure 5 As shown, after the above step 1061, the method further includes:
[0113] Step 1064: If no feedback information is received within the first preset time period, determine the duration of eye closure from the personnel health status information.
[0114] The duration of eye closure is obtained in advance by identifying the open and closed state of the eye region in the person image. Specifically, the person health status information in this embodiment may include the duration of eye closure obtained by identifying the person image. The specific method for generating the duration of eye closure can refer to the above embodiment, that is, the method for determining the duration of eye closure in the above embodiment needs to be executed before the steps of this embodiment are executed, which will not be repeated here.
[0115] Step 1065: If the duration of eye closure is greater than or equal to the second preset duration, determine the target help-seeking terminal and send help-seeking information to the target help-seeking terminal.
[0116] When the duration of eye closure is greater than or equal to the second preset duration, it indicates that the health condition of the target person is extremely poor. At this time, there is no need to interact with the target person and a help message can be directly sent.
[0117] This embodiment realizes timing after outputting the health assistance prompt information, and automatically sends a help message to the target help terminal when the target person keeps his eyes closed for a long time, thereby automatically asking for help when the target person is unable to interact with the vehicle computer normally, reducing the risk of accidents to the target person.
[0118] Figure 6A schematic diagram of the structure of a device for identifying the health status of a person on a vehicle provided in an embodiment of the present application. Specifically comprising: a first acquisition module 601, used to acquire a person image taken of a target person on a vehicle; an identification module 602, used to identify the person status of the person image and obtain the person health status information; a first determination module 603, used to determine whether the target person is currently in an unhealthy state based on the person health status information; a second acquisition module 604, used to acquire the vehicle status information of the vehicle if the target person is in an unhealthy state; a second determination module 605, used to determine the current health risk level of the target person based on the health status information and the vehicle status information; an execution module 606, used to execute a pre-set health assistance operation corresponding to the health risk level.
[0119] In some optional implementations of the present embodiment, the recognition module includes: a first recognition unit, used to determine the lip area from the image of the person, perform lip color recognition on the lip area, and obtain lip color status information; and / or, a second recognition unit, used to determine the chest area from the image of the person, and perform thoracic movement recognition on the chest area to obtain thoracic movement status information; and / or, a third recognition unit, used to determine the eye area from the image of the person, perform eye opening and closing status recognition on the eye area, and obtain eye opening and closing status information; a third determination module, used to determine at least one of the lip color status information, thoracic movement status information, and eye opening and closing status information as the health status information of the person.
[0120] In some optional implementations of this embodiment, the first determination module includes: a first determination unit, used to determine the corresponding first health score based on lip color status information; and / or, a second determination unit, used to determine the corresponding second health score based on thoracic movement status information; and / or, a third determination unit, used to determine the duration of eye closure based on eye open and closed status information; a fourth determination unit, used to determine whether the health status information of the person meets the preset health conditions based on at least one of the first health score, the second health score and the duration of eye closure, and if not, determine that the target person is currently in an unhealthy state.
[0121] In some optional implementations of this embodiment, the second acquisition module includes: a fifth determination unit, used to determine the altitude information of the current location of the vehicle, and determine the altitude information as vehicle status information; the second determination module includes: a sixth determination unit, used to determine the altitude level corresponding to the altitude information; a seventh determination unit, used to determine the health risk level based on the altitude level and personal health status information.
[0122] In some optional implementations of this embodiment, the execution module includes: an output unit for outputting health assistance prompt information; a receiving unit for receiving feedback information generated in response to the health assistance prompt information; and an execution unit for executing pre-set health assistance operations corresponding to the health risk level based on the feedback information.
[0123] In some optional implementations of the present embodiment, the execution unit includes: a first determination subunit, used to determine the target help terminal and send help information to the target help terminal if the feedback information indicates that the target person needs help; a control subunit, used to stop outputting health auxiliary prompt information if the feedback information indicates that the target person does not need help and the health risk level is the first level; an output subunit, used to periodically output health auxiliary prompt information according to a preset prompt period corresponding to the health risk level if the feedback information indicates that the target person does not need help and the health risk level is not the first level.
[0124] In some optional implementations of this embodiment, the output subunit is further used to: if the health risk level is the second level, periodically output health auxiliary prompt information according to a preset first prompt period; if the health risk level is the third level, periodically output health auxiliary prompt information according to a preset second prompt period, wherein the second prompt period is smaller than the first prompt period.
[0125] In some optional implementations of this embodiment, the device also includes: a fourth determination module, used to determine the duration of eye closure from the health status information of the person if no feedback information is received within the first preset time period, wherein the duration of eye closure is obtained in advance by identifying the open and closed eyes state of the eye area in the person's image; a sending module, used to determine the target help terminal if the duration of eye closure is greater than or equal to the second preset time period, and send a help message to the target help terminal.
[0126] The device for identifying the health status of a person on board provided in this embodiment may be as follows: Figure 6 The on-board occupant health status identification device shown in the figure can execute all steps of the above-mentioned on-board occupant health status identification methods, thereby achieving the technical effects of the above-mentioned on-board occupant health status identification methods. For details, please refer to the above-mentioned relevant description. For the sake of brevity, it will not be elaborated here.
[0127] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, Figure 7The electronic device 700 shown includes: at least one processor 701, a memory 702, at least one network interface 704 and other user interfaces 703. The various components in the electronic device 700 are coupled together via a bus system 705. It is understood that the bus system 705 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 705 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 705 is not described in detail. Figure 7 Various buses are labeled as bus system 705.
[0128] The user interface 703 may include a display, a keyboard, or a pointing device (eg, a mouse, a trackball, a touch pad, or a touch screen).
[0129] It can be understood that the memory 702 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct RAM bus random access memory (DRRAM). The memory 702 described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0130] In some implementations, the memory 702 stores the following elements, executable units or data structures, or a subset thereof, or an extended set thereof: an operating system 7021 and an application 7022 .
[0131] The operating system 7021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 7022 includes various application programs, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. The program for implementing the method of the embodiment of the present application can be included in the application 7022.
[0132] In this embodiment, by calling the program or instruction stored in the memory 702, specifically, the program or instruction stored in the application 7022, the processor 701 is used to execute the method steps provided by each method embodiment, for example, including:
[0133] Acquire a person image taken of a target person on the vehicle; perform person status recognition on the person image to obtain person health status information; determine whether the target person is currently in an unhealthy state based on the person health status information; if the target person is in an unhealthy state, acquire the vehicle status information of the vehicle; determine the current health risk level of the target person based on the health status information and the vehicle status information; and execute a pre-set health assistance operation corresponding to the health risk level.
[0134] The method disclosed in the above embodiment of the present application can be applied to the processor 701, or implemented by the processor 701. The processor 701 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 701 or the instruction in the form of software. The above processor 701 can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software units in the decoding processor can be executed. The software unit can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 702, and the processor 701 reads the information in the memory 702 and completes the steps of the above method in combination with its hardware.
[0135] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPDevice, DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the above functions of the present application, or a combination thereof.
[0136] For software implementation, the technology described above in this article can be implemented by a unit that performs the functions described above in this article. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.
[0137] The electronic device provided in this embodiment may be Figure 7 The electronic device shown in can execute all the steps of the above-mentioned methods for identifying the health status of each person on the vehicle, thereby achieving the technical effects of the above-mentioned methods for identifying the health status of each person on the vehicle. Please refer to the above related description for details. For the sake of brevity, it will not be repeated here.
[0138] The embodiment of the present application also provides a storage medium (computer-readable storage medium). The storage medium here stores one or more programs. The storage medium may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk or a solid-state drive; the memory may also include a combination of the above-mentioned types of memory.
[0139] When one or more programs in the storage medium can be executed by one or more processors, the above-mentioned method for identifying the health status of people on board the vehicle executed on the electronic device side can be implemented.
[0140] The processor is used to execute the program stored in the memory to implement the following steps of the method for identifying the health status of a person on board a vehicle executed on the electronic device side:
[0141] Acquire a person image taken of a target person on the vehicle; perform person status recognition on the person image to obtain person health status information; determine whether the target person is currently in an unhealthy state based on the person health status information; if the target person is in an unhealthy state, acquire the vehicle status information of the vehicle; determine the current health risk level of the target person based on the health status information and the vehicle status information; and execute a pre-set health assistance operation corresponding to the health risk level.
[0142] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different circuits to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0143] The steps of the circuits or algorithms described in conjunction with the embodiments disclosed herein may be implemented using hardware, software modules executed by a processor, or a combination of the two. The software modules may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0144] It should be understood that the terms used in the text are only for the purpose of describing specific example embodiments, and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used in the text may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The steps, processes, and operations described in the text are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.
[0145] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.
Claims
1. A method for identifying the health status of a person on board a vehicle, characterized in that: The method comprises: Acquire a person image captured of a target person on the vehicle; Performing personnel status recognition on the personnel image to obtain personnel health status information; Based on the personnel health status information, determining whether the target person is currently in an unhealthy state; If the target person is in an unhealthy state, obtaining vehicle status information of the vehicle; Determining a current health risk level of the target person based on the health status information and the vehicle status information; Execute a preset health assistance operation corresponding to the health risk level.
2. The method according to claim 1, characterized in that: The performing personnel status recognition on the personnel image to obtain personnel health status information includes: Determine a lip region from the person image, perform lip color recognition on the lip region, and obtain lip color status information; and / or, Determine a chest region from the person image, and perform chest cage motion recognition on the chest region to obtain chest cage motion state information; and / or, Determine an eye region from the person image, perform eye-opening and eye-closing state recognition on the eye region, and obtain eye-opening and eye-closing state information; At least one of the lip color status information, the thorax movement status information, and the eye opening and closing status information is determined as the personnel health status information.
3. The method according to claim 2, characterized in that The determining, based on the personnel health status information, whether the target person is currently in an unhealthy state includes: Based on the lip color status information, determining a corresponding first health score; and / or, Based on the thoracic movement state information, determining a corresponding second health score; and / or, Determining the duration of eye closure based on the eye opening and closing state information; Based on at least one of the first health score, the second health score and the duration of eyes closing, determine whether the health status information of the person meets the preset health conditions; if not, determine that the target person is currently in an unhealthy state.
4. The method according to claim 1, characterized in that: The obtaining of vehicle status information of the vehicle includes: Determining altitude information of a current location of the vehicle, and determining the altitude information as the vehicle state information; The determining the current health risk level of the target person based on the health status information and the vehicle status information includes: Determining an altitude level corresponding to the altitude information; The health risk level is determined according to the altitude level and the personnel health status information.
5. The method according to claim 1, characterized in that The performing of a preset health assistance operation corresponding to the health risk level includes: Output health assistance prompt information; Receiving feedback information generated in response to the health assistance prompt information; Based on the feedback information, a preset health assistance operation corresponding to the health risk level is performed.
6. The method according to claim 5, characterized in that The performing a preset health assistance operation corresponding to the health risk level based on the feedback information includes: If the feedback information indicates that the target person needs help, determine a target help-seeking terminal, and send help-seeking information to the target help-seeking terminal; If the feedback information indicates that the target person does not need help and the health risk level is the first level, stop outputting the health assistance prompt information; If the feedback information indicates that the target person does not need help and the health risk level is not the first level, the health assistance prompt information is periodically output according to a preset prompt period corresponding to the health risk level.
7. The method according to claim 6, characterized in that The periodically outputting the health auxiliary prompt information according to the preset prompt period corresponding to the health risk level includes: If the health risk level is the second level, periodically outputting the health auxiliary prompt information according to a preset first prompt period; If the health risk level is the third level, the health auxiliary prompt information is periodically output according to a preset second prompt cycle, wherein the second prompt cycle is smaller than the first prompt cycle.
8. The method according to claim 5, characterized in that After outputting the health assistance prompt information, the method further includes: If the feedback information is not received within the first preset time, determining the duration of eye closure from the health status information of the person, wherein the duration of eye closure is obtained in advance by identifying the open and closed eye state of the eye region in the image of the person; If the eye-closing duration is greater than or equal to a second preset duration, a target help-seeking terminal is determined, and help-seeking information is sent to the target help-seeking terminal.
9. A device for identifying the health status of people on board, characterized in that: The device comprises: The first acquisition module is used to acquire a person image obtained by photographing a target person on the vehicle; An identification module is used to identify the status of the person in the image and obtain the health status information of the person; A first determination module is used to determine whether the target person is currently in an unhealthy state based on the health status information of the person; A second acquisition module is used to acquire vehicle status information of the vehicle if the target person is in an unhealthy state; A second determination module, configured to determine a current health risk level of the target person based on the health status information and the vehicle status information; The execution module is used to execute a preset health assistance operation corresponding to the health risk level.
10. An electronic device, characterized in that: include: Memory for storing computer programs; A processor is used to execute the computer program stored in the memory, and when the computer program is executed, the method for identifying the health status of people on board as described in any one of claims 1 to 8 above is implemented.
