Vehicle early warning method, device, equipment, storage medium and program product

Abstract

The invention relates to a vehicle early warning method, device and equipment, a storage medium and a program product, and relates to the technical field of artificial intelligence, the method is executed by a vehicle, the vehicle comprises at least two space areas, and each space area is provided with at least two vital sign sensors. The method comprises the following steps: acquiring position attitude information of a target person in a vehicle; positioning a target space area based on the position attitude information of the target person; detecting a target person through the at least two target sensors to obtain vital sign information corresponding to the at least two target sensors; performing weighted averaging operation on the vital sign information corresponding to the at least two target sensors, and acquiring data after weighted averaging as target sensing data; inputting the target sensing data into an intelligent early warning system in the vehicle to obtain an output result of the intelligent early warning system; and executing vehicle early warning operation based on an output result of the intelligent early warning system. According to the scheme, the vehicle early warning efficiency can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular to a vehicle early warning method, device, equipment, storage medium and program product. Background Art

[0002] With the rapid development of intelligent connected vehicle technology, vehicle safety and personnel health monitoring have become important research directions.

[0003] In related technologies, the vehicle can transmit signals between the non-contact life detection module and the central processor, establish data communication with the cloud server through the central processor, and establish calls through the voice communication module. If the vehicle is parked and there are still signs of life in the car, the owner will be notified to prevent children or pets left in the car from being forgotten.

[0004] However, most of the vital signs recognition systems in the above solutions are used to detect whether there is a living organism in the car after the car is locked or the driver temporarily leaves the car. The application scenarios are relatively single and have great limitations. The monitoring effect of vital signs in the car is poor, which makes the efficiency of vehicle early warning low. Summary of the invention

[0005] The embodiments of the present application provide a vehicle early warning method, device, equipment, storage medium and program product, which can improve the efficiency of vehicle early warning. The technical solution is as follows:

[0006] In one aspect, a vehicle early warning method is provided, the method being executed by a vehicle, the vehicle comprising at least two spatial regions, each of the spatial regions being configured with at least two vital sign sensors, the method comprising:

[0007] Acquire position and posture information of a target person inside the vehicle; the position and posture information is used to indicate the position and posture of the target person;

[0008] Locating a target spatial area based on the position and posture information of the target person, wherein the target spatial area is a spatial area where the target person is located in the at least two spatial areas;

[0009] Detecting the target person through at least two target sensors to obtain vital sign information corresponding to the at least two target sensors respectively; the at least two target sensors are at least two vital sign sensors corresponding to the target space area; the vital sign information is used to indicate the vital signs of the target person; the vital sign information includes at least body temperature, blood sample saturation, heart rate and respiratory rate;

[0010] Performing a weighted average operation on the vital sign information respectively corresponding to the at least two target sensors, and acquiring weighted average data as target sensor data;

[0011] Inputting the target sensor data into the intelligent warning system in the vehicle to obtain an output result of the intelligent warning system;

[0012] Based on the output result of the intelligent warning system, a vehicle warning operation is performed.

[0013] In another aspect, a vehicle warning device is provided, the device comprising:

[0014] A position and posture information acquisition module, used to acquire the position and posture information of the target person inside the vehicle; the position and posture information is used to indicate the position and posture of the target person;

[0015] A target space region positioning module, used for positioning a target space region based on the position and posture information of the target person, wherein the target space region is a space region where the target person is located in the at least two space regions;

[0016] a vital sign information acquisition module, configured to perform detection on the target person through at least two target sensors, and obtain vital sign information corresponding to the at least two target sensors respectively; the at least two target sensors are at least two vital sign sensors corresponding to the target space area; the vital sign information is used to indicate the vital signs of the target person; the vital sign information includes at least body temperature, blood sample saturation, heart rate and respiratory rate;

[0017] A target sensing data acquisition module, used to perform a weighted average operation on the vital sign information respectively corresponding to the at least two target sensors, and acquire the weighted average data as the target sensing data;

[0018] An intelligent early warning detection module inputs the target sensor data into an intelligent early warning system in the vehicle to obtain an output result of the intelligent early warning system;

[0019] The warning operation execution module is used to execute the vehicle warning operation based on the output result of the intelligent warning system.

[0020] In some embodiments, the warning operation execution module is used to input the target sensor data into the intelligent warning system, and obtain the vital sign recognition result through the intelligent warning system; the vital sign recognition result is used to indicate the vital sign status of the target person;

[0021] Based on the vital sign recognition result, an early warning operation is performed.

[0022] In some embodiments, the warning operation execution module is used to determine whether the vital signs of the target person are abnormal according to the vital sign recognition result;

[0023] In the case where the vital signs of the target person are abnormal, determining the warning level of the vital sign recognition result;

[0024] Based on the warning level, a warning operation corresponding to the warning level is performed.

[0025] In some embodiments, the apparatus further comprises:

[0026] A weight determination module is used to perform a weighted averaging operation on the vital sign information corresponding to the at least two target sensors respectively, and before obtaining the weighted averaged data as the target sensor data, determine the weight of the vital sign information in the weighted averaging operation according to the posture of the target person and the positions of the at least two target sensors relative to the target space area.

[0027] In some embodiments, the apparatus further comprises:

[0028] A vital sign information acquisition module, used to generate a vital sign information data threshold based on historical vital sign information; the historical vital sign information is vital sign information acquired by the at least two target sensors before acquiring the vital sign information of the target person this time;

[0029] The vital sign warning information issuing module is used to issue vital sign warning information when the data of the vital sign information corresponding to the at least two target sensors respectively exceed the vital sign information data threshold; the vital sign warning information is used to indicate that the vital sign of the target person is abnormal.

[0030] In some embodiments, the apparatus further comprises:

[0031] A help information sending module is used to send a help information to the emergency service department through the vehicle's Internet of Vehicles platform when the difference between the vital signs information data corresponding to the at least two target sensors and the vital signs information data threshold reaches a specified value; the help information includes the vehicle's location information and the target person's vital signs information.

[0032] On the other hand, a computer device is provided, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the vehicle warning method as described above.

[0033] On the other hand, a computer-readable storage medium is provided, in which at least one instruction, at least one program, code set or instruction set is stored, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the vehicle warning method as described above.

[0034] In another aspect, a computer program product is provided, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the vehicle warning method provided in the above-mentioned various optional implementations.

[0035] The technical solution provided by this application may have the following beneficial effects:

[0036] In the embodiment of the present application, at least two vital signs sensors are configured in each spatial area inside the vehicle, ensuring that the vehicle can fully cover the vital signs information collection of target personnel at different positions inside the vehicle. On the basis of ensuring the comprehensiveness of the vital signs information collection, the vehicle further performs a weighted averaging operation on the vital signs information. The weighted averaging operation combines the data of at least two target sensors, reduces the error impact that may be caused by a single target sensor, and improves the reliability of the early warning system. The weighted averaged target sensor data is input into the intelligent early warning system, and the intelligent early warning system is used to quickly and accurately analyze the changing trend of the vital signs data, identify abnormal situations in a timely manner, and trigger the corresponding early warning measures of the vehicle, which can effectively avoid the lag and false alarm that may exist in traditional early warning systems, and effectively improve the efficiency of vehicle early warning.

[0037] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0039] Figure 1 is a system configuration diagram of a vehicle early warning method involved in one embodiment of the present application;

[0040] Figure 2 is a flow chart of a vehicle warning method provided by an embodiment of the present application;

[0041] Figure 3 is a flow chart of a vehicle warning method provided by an embodiment of the present application;

[0042] Figure 4This is a flow chart of a vital sign detection and early warning method provided by an embodiment of the present application;

[0043] Figure 5 is a block diagram of a vehicle warning device provided by an exemplary embodiment of the present application;

[0044] Figure 6 It is a structural diagram of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0045] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0046] Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application as recited in the appended claims.

[0047] Figure 1 FIG. 1 is a system diagram of a vehicle warning method according to an embodiment of the present application. Figure 1 As shown, the method is performed by a vehicle 100, which includes at least two spatial areas, each of which is equipped with at least two vital signs sensors. The vehicle 100 obtains the position and posture information of the target person 10 inside; locates the spatial area where the target person 10 is located in at least two spatial areas based on the position and posture information of the target person 10 as the target spatial area; performs detection on the target person 10 through at least two target sensors, and obtains vital signs information corresponding to at least two target sensors for indicating the target person 10; the vital signs information includes at least body temperature, blood sample saturation, heart rate and respiratory rate; performs weighted average operation on the vital signs information corresponding to at least two target sensors, and obtains the weighted average data as target sensing data; inputs the target sensing data into the intelligent warning system in the vehicle 100, and obtains the output result of the intelligent warning system; and performs the warning operation of the vehicle 100 based on the output result of the intelligent warning system.

[0048] Figure 2 : is a flow chart of a vehicle warning method provided by an embodiment of the present application. The vehicle warning method can be executed by a vehicle, for example, the vehicle can be the above-mentioned Figure 1 The vehicle 100 is shown. The vehicle warning method is executed by a vehicle, which includes at least two space areas, and each space area is configured with at least two vital sign sensors.

[0049] In the embodiment of the present application, the interior of the vehicle is divided into at least two spatial areas, each of which may correspond to a seat, and different spatial areas correspond to different seats. For example, the driver's seat corresponds to one spatial area, and the co-pilot seat corresponds to another spatial area.

[0050] Among them, the above-mentioned vital sign sensor is a sensor that can collect vital sign information, and the vital sign sensor includes but is not limited to a camera, an infrared sensor, and an ultrasonic sensor.

[0051] In an embodiment of the present application, each spatial area is configured with at least two vital sign sensors, and the vehicle obtains vital sign information in each spatial area through the at least two vital sign sensors in the spatial area.

[0052] The method may include step 210, step 220, step 230, step 240, step 250 and step 260, which are specifically implemented as follows:

[0053] Step 210: Acquire the position and posture information of the target person inside the vehicle; the position and posture information is used to indicate the position and posture of the target person.

[0054] Among them, the position of the above-mentioned target person refers to the specific position of the target person inside the vehicle, for example, in the driver's seat, the co-pilot seat or the back seat.

[0055] The above-mentioned posture of the target person refers to the body posture of the target person inside the vehicle, such as sitting, lying or other atypical postures.

[0056] In an embodiment of the present application, the vehicle can build a three-dimensional model of the target person through the vehicle's on-board camera and depth sensor, so as to more accurately determine his position and posture.

[0057] In some embodiments, the vehicle obtains an image or video of a target person inside the vehicle through an on-board camera, and obtains position and posture information of the target person inside the vehicle based on a deep learning model; the position and posture information is used to indicate the position and posture of the target person; the deep learning model is a machine learning model that has the ability to recognize the position and posture characteristics of life forms in images or videos based on the images or videos.

[0058] In the above embodiment, the vehicle can collect images or videos of the target person inside the vehicle from multiple angles through the built-in vehicle-mounted camera, pre-process the collected images (for example, denoising, contrast enhancement, brightness correction), and input the trained deep learning model to obtain the position and posture information output by the deep learning model as the position and posture information of the target person.

[0059] Step 220: Locate a target spatial region based on the position and posture information of the target person, where the target spatial region is a spatial region where the target person is located among the at least two spatial regions.

[0060] In an embodiment of the present application, a three-dimensional coordinate system can be set inside the vehicle, and each spatial area corresponds to a three-dimensional coordinate range. When the vehicle obtains the position information of the target person, the three-dimensional coordinate range of the target person inside the vehicle is extracted from the position information, and the three-dimensional coordinate range of the target person's location is paired with the preset three-dimensional coordinate range, and the spatial area corresponding to the three-dimensional coordinate range that is closest to or completely contains the target person's location is obtained as the target spatial area.

[0061] Step 230: Detect the target person through at least two target sensors to obtain vital sign information corresponding to the at least two target sensors respectively; the at least two target sensors are at least two vital sign sensors corresponding to the target space area; the vital sign information is used to indicate the vital signs of the target person; the vital sign information includes at least body temperature, blood sample saturation, heart rate and respiratory rate.

[0062] In an embodiment of the present application, after locating the target spatial area where the target person is located, the vehicle activates at least two vital sign sensors in the target area, performs detection on the target person, and obtains the vital sign information of the target person.

[0063] Among them, the above-mentioned target sensors include but are not limited to: a body temperature sensor (used to obtain the body temperature of the target person), a blood oxygen saturation sensor (used to obtain the blood sample saturation of the target person), a heart rate sensor (used to obtain the heart rate of the target person) and a respiratory rate sensor (used to obtain the respiratory rate of the target person).

[0064] Step 240: Perform a weighted average operation on the vital sign information corresponding to at least two target sensors, and obtain weighted average data as target sensor data.

[0065] The weighted average operation is an operation in which the vehicle assigns different weight values ​​to the data obtained by at least two target sensors and then calculates the weighted average of the data obtained by at least two target sensors. The weighted average is used to integrate the vital sign information of at least two target sensors to obtain more reliable target sensing data.

[0066] In an embodiment of the present application, after the vehicle obtains the vital signs information corresponding to at least two target sensors, the vehicle determines the corresponding weight according to the quality of the data obtained by each sensor. For example, if the data obtained by a certain sensor has a higher accuracy, a higher weight is assigned to the data corresponding to the sensor. After assigning weights to the data obtained by each sensor, the data obtained by all sensors are multiplied by their corresponding weights, and the total is calculated and finally divided by the sum of all weights to obtain the weighted average target sensor data.

[0067] Step 250: Input the target sensor data into the intelligent warning system in the vehicle to obtain the output result of the intelligent warning system.

[0068] Among them, the above-mentioned intelligent warning system is a system in the vehicle that can receive and process vital signs data.

[0069] In an embodiment of the present application, after obtaining the target sensor data, the vehicle can transmit the target sensor data to the intelligent warning system through the in-vehicle network. After receiving the target sensor data, the intelligent warning system judges the target sensor data and obtains an output result. The output result can be a prompt (such as a sound alarm, a visual warning) or an action instruction (such as a suggestion to rest, contact emergency services, or automatically slow down and stop).

[0070] Step 260: Execute vehicle warning operation based on the output result of the intelligent warning system.

[0071] Among them, the above-mentioned vehicle warning operation is a preventive and protective measure executed by the intelligent warning system based on the output results obtained after the intelligent warning system analyzes the target sensor data.

[0072] In the embodiment of the present application, at least two vital signs sensors are configured in each spatial area inside the vehicle, ensuring that the vehicle can fully cover the vital signs information collection of target personnel at different positions inside the vehicle. On the basis of ensuring the comprehensiveness of the vital signs information collection, the vehicle further performs a weighted averaging operation on the vital signs information. The weighted averaging operation combines the data of at least two target sensors, reduces the error impact that may be caused by a single target sensor, and improves the reliability of the early warning system. The weighted averaged target sensor data is input into the intelligent early warning system, and the intelligent early warning system is used to quickly and accurately analyze the changing trend of the vital signs data, identify abnormal situations in a timely manner, and trigger the corresponding early warning measures of the vehicle, which can effectively avoid the lag and false alarm that may exist in traditional early warning systems, and effectively improve the efficiency of vehicle early warning.

[0073] based on Figure 2 , please refer to Figure 3 , Figure 3 This is a flow chart of a vehicle warning method provided by an embodiment of the present application. Figure 2 Step 250 in the embodiment can be implemented as step 250a and step 250b, as follows:

[0074] Step 250a: Input the target sensor data into the intelligent early warning system, and obtain the vital sign recognition result through the intelligent early warning system; the vital sign recognition result is used to indicate the vital sign status of the target person.

[0075] In an embodiment of the present application, after receiving the target sensor data, the intelligent warning system can analyze the target sensor data through a built-in machine learning model to obtain a vital sign recognition result corresponding to the target sensor data, wherein the built-in machine learning model in the intelligent warning system is a deep neural network that has the ability to judge the vital sign status of the target person based on the target sensor data.

[0076] Step 250b: Execute an early warning operation based on the vital sign recognition result.

[0077] In an embodiment of the present application, after the vehicle obtains the vital sign recognition result through the intelligent warning system, it performs a warning operation corresponding to the vital sign recognition result.

[0078] In the embodiment of the present application, the vital signs recognition results obtained by the intelligent warning system can directly reflect the vital signs status of the target person. The vehicle performs corresponding warning operations based on the vital signs status of the target person, which can make the warning operation more targeted and effectively improve the efficiency of vehicle warning.

[0079] Based on the schemes shown in any one or more of the above-mentioned embodiments, in some embodiments, the above-mentioned step 250b can be implemented as follows: judging whether the vital signs of the target person are abnormal based on the vital signs recognition results; in the case where the vital signs of the target person are abnormal, determining the warning level of the vital signs recognition results; and based on the warning level, executing a warning operation corresponding to the warning level.

[0080] In the embodiment of the present application, the vital sign recognition result may be text or numbers indicating the degree of abnormality of the vital signs of the target person.

[0081] In some embodiments, when the vital sign recognition result is text, the vital sign recognition result can directly indicate the degree of abnormality of the target person's vital signs. For example, the text content can be "slight abnormality", "intermediate abnormality", or "serious abnormality". The vehicle determines the warning level of the vital sign recognition result based on the text. In an embodiment of the present application, each text corresponds to a warning level, and different texts correspond to different warning levels. A database is stored inside the vehicle, and the database contains text pairs of different texts and warning levels corresponding to the text. After the vehicle obtains the vital sign recognition result, it queries the database based on the text to obtain the warning level corresponding to the text as the warning level corresponding to the vital sign recognition result.

[0082] In some embodiments, when the vital sign recognition result is a number, the vital sign recognition result can be a percentage value, which is used to indicate the degree of abnormality of the vital signs of the target person. The larger the percentage value, the more serious the degree of abnormality of the vital signs of the target person, such as "20%", "50%", "60%"; in the embodiment of the present application, each number corresponds to a warning level, and different numbers correspond to different warning levels. A database is stored inside the vehicle, and the database contains digital pairs of different numbers and warning levels corresponding to the numbers. After the vehicle obtains the vital sign recognition result, the database is queried according to the number to obtain the warning level corresponding to the number as the warning level corresponding to the vital sign recognition result. Optionally, a database is stored inside the vehicle, and the database contains digital pairs of different digital ranges and corresponding warning levels. After the vehicle obtains the vital sign recognition result, the database is queried according to the number to obtain the digital range to which the number belongs, and the database is traversed to obtain the warning level corresponding to the digital range as the warning level corresponding to the vital sign recognition result.

[0083] In the embodiment of the present application, each warning level may correspond to at least one warning operation, and different warning levels correspond to different warning operations. After determining the warning level, the vehicle executes at least one warning operation corresponding to the warning level.

[0084] In the embodiment of the present application, the above scheme can perform different warning operations according to the different severity of the vital signs of the target person through a layered warning mechanism, thereby improving the pertinence and effectiveness of the warning operation, thereby effectively improving the efficiency of vehicle warning.

[0085] Based on the scheme shown in any one or more of the above embodiments, in some embodiments, before the above step 240, the vehicle determines the weight of the vital sign information in the weighted averaging operation based on the posture of the target person and the position of at least two target sensors relative to the target space area.

[0086] In an embodiment of the present application, the vehicle determines the weight of the vital sign information in the weighted averaging operation based on the posture of the target person and the distance between at least two target sensors and the target person in the target space area.

[0087] In some embodiments, the distance between the target sensor and the target person in the target space area is inversely proportional to the weight of the vital sign information in the weighted average operation. The closer the distance between the target sensor and the target person, the higher the accuracy of the data obtained by the target sensor, and the greater the weight assigned to the data obtained by the target sensor. Conversely, the farther the distance between the target sensor and the target person, the lower the accuracy of the data obtained by the target sensor, and the smaller the weight assigned to the data obtained by the target sensor.

[0088] For example, when the target person is sitting, the target sensor located near the seat back or the seat belt is closest to the target person and closer to the target person's heart, and the heart rate detected by the target sensor is set with a higher weight.

[0089] Optionally, the vehicle determines the facial orientation of the target person according to the posture of the target person, and determines the weight of the vital sign information in the weighted averaging operation based on the facial orientation of the target person and the relative direction between at least two target sensors and the target person; wherein the above-mentioned relative direction is the direction corresponding to the line between at least two target sensors and the face of the target person.

[0090] In some embodiments, the vehicle determines the weight of the vital sign information in the weighted averaging operation based on the facial orientation of the target person and the angle between at least two target sensors and the target person's face. The angle between the target sensor and the target person's face is inversely proportional to the weight of the vital sign information in the weighted averaging operation; the larger the angle between the target sensor and the target person's face, the smaller the weight assigned to the data obtained by the target sensor, and conversely, the smaller the angle between the target sensor and the target person's face, the larger the weight assigned to the data obtained by the target sensor.

[0091] For example, the target sensor A directly in front of the target person has an angle of almost 0 degrees with the target person's facial orientation, and the target sensor B in front of the left of the target person has an angle of 45 degrees with the target person's facial orientation. The weight assigned to the data obtained by target sensor A is greater than the weight assigned to the data obtained by target sensor B.

[0092] It should be noted that, in the embodiment of the present application, the vehicle dynamically adjusts the weight of the data obtained by each target sensor in real time according to the rotation direction of the target person's head.

[0093] In an embodiment of the present application, the vehicle determines the weight of the vital sign information in the weighted averaging operation based on the posture of the target person and the specific positions of at least two target sensors in the target space area, which can reduce the impact of low-quality data and improve the accuracy of the results of the weighted averaging operation.

[0094] Based on the schemes shown in any one or more of the above-mentioned embodiments, in some embodiments, the vehicle generates a vital signs information data threshold based on historical vital signs information; the historical vital signs information is the vital signs information acquired by at least two target sensors before acquiring the vital signs information of the target person this time; when the data of the vital signs information corresponding to the at least two target sensors respectively exceed the vital signs information data threshold, a vital signs warning message is issued; the vital signs warning message is used to indicate that there is an abnormality in the vital signs of the target person.

[0095] The historical vital sign information refers to the vital sign data of the target person acquired by at least two target sensors in the past specified time period. For example, the historical vital sign information may be the vital sign data of the target person acquired by at least two target sensors in the past week.

[0096] Among them, the above-mentioned vital signs information data threshold is a reference standard generated by the vehicle based on historical vital signs information, which is used to determine whether the vital signs of the currently monitored target person are abnormal.

[0097] Among them, the above-mentioned vital sign warning information is an alarm or prompt information issued by the vehicle when the vital sign information detected by at least two target sensors exceeds a set data threshold.

[0098] The vital sign warning information may be issued in the form of a visual warning (such as a warning light on the dashboard), an audible alarm (such as a voice prompt played through a speaker in the vehicle), or an automatic sending of a help message to an emergency service department.

[0099] In an embodiment of the present application, after the vehicle obtains the vital signs information of the target person through at least two target sensors, it will compare it with the pre-set vital signs information data threshold. After the vital signs information of the target person obtained by at least two target sensors exceeds the pre-set vital signs information data threshold, a vital signs warning message will be issued.

[0100] In an embodiment of the present application, the vital signs information data threshold set based on historical vital signs information can indicate the vital signs information of the target person in a normal state. When the vital signs information detected by all target sensors exceeds the vital signs information data threshold, a warning message is issued, which can effectively ensure the accuracy of vehicle warnings.

[0101] Based on the schemes shown in any one or more of the above-mentioned embodiments, in some embodiments, when the difference between the data of the vital signs information corresponding to at least two target sensors and the vital signs information data threshold reaches a specified value, the vehicle sends a help message to the emergency service department through the vehicle's Internet of Vehicles platform; the help message includes the vehicle's location information and the target person's vital signs information.

[0102] The difference between the data of the vital signs information and the data threshold of the vital signs information refers to the difference between the vital signs information currently acquired by at least two target sensors and the preset data threshold of the vital signs information. In the embodiment of the present application, when the difference between the data of the vital signs information corresponding to at least two target sensors and the data threshold of the vital signs information reaches a specified value, it indicates that the vital signs of the target person are obviously abnormal and the vehicle needs emergency intervention.

[0103] Among them, the above-mentioned Internet of Vehicles platform is a communication platform that connects the vehicle with the outside world (such as the Internet). Based on the Internet of Vehicles platform, the vehicle can send assistance information to the emergency service department through the network.

[0104] In an embodiment of the present application, the vehicle obtains the vital signs information of the target person in real time through at least two target sensors. Each time a vital sign information is obtained, the data of the vital sign information is compared with the vital sign information data threshold. When the data of the vital sign information is less than the vital sign information data threshold and the difference between the data of the vital sign information and the vital sign information data threshold satisfies a specified value, the vehicle obtains its own current location information and the latest vital signs data of the target person, generates a help message containing the current location information of the vehicle and the latest vital signs data of the target person, and sends the help message to the emergency service department via a wireless network. Optionally, when the data of the vital sign information is greater than the vital sign information data threshold and the difference between the data of the vital sign information and the vital sign information data threshold satisfies a specified value, the vehicle obtains its own current location information and the latest vital signs data of the target person, generates a help message containing the current location information of the vehicle and the latest vital signs data of the target person, and sends the help message to the emergency service department via a wireless network.

[0105] For example, the heart rate threshold is 80 beats / minute, the specified value is a heart rate difference of 20 beats / minute, the heart rate reading obtained by the target sensor is 95 beats / minute, and the difference is 15 beats / minute. The vehicle obtains the vehicle's current location information and the target person's latest vital signs data, and generates a help message "The vehicle with license plate number XXXX is located at longitude and latitude XX, XX. The driver's heart rate is 95 beats / minute. Please dispatch an ambulance immediately." and sends the help message to the emergency service department.

[0106] In some embodiments, after a vehicle sends a request for help to emergency services through the vehicle's Internet of Vehicles platform, if the difference between the vital signs information data corresponding to at least two target sensors and the vital signs information data threshold continues to increase, the vehicle continues to update the request for help and immediately sends it to the emergency services through the vehicle's Internet of Vehicles platform.

[0107] In an embodiment of the present application, when the difference between the vital signs information detected by at least two target sensors and the vital signs information data threshold reaches a specified value, it indicates that the target person may have a serious adverse physical condition. The vehicle triggers an automatic help mechanism and sends a help message containing the vehicle location and detailed vital signs data. It can perform accurate early warning and immediate assistance, thereby improving the efficiency of vehicle early warning.

[0108] Based on the above Figure 2 and Figure 3 The steps in the embodiments of the present application illustrate a non-contact vital signs detection and intelligent warning system and method in a vehicle based on multimodal sensing technology.

[0109] For example, please refer to Figure 4 , Figure 4 This is a flow chart of a vital sign detection and early warning method provided by an embodiment of the present application. The method includes three modules, namely, an information collection module 401, a data processing and analysis module 402 and an intelligent early warning system 403.

[0110] 1. Information collection module 401: In this embodiment, the vehicle uses a multimodal sensor to collect vital characteristics information of passengers in the vehicle. The body temperature, blood oxygen saturation, heart rate and respiratory rate of passengers in the vehicle can be collected respectively through infrared thermal imaging, near-infrared spectral sensors and millimeter-wave radar sensors.

[0111] Infrared thermal imaging: Use infrared thermal imaging technology to capture the facial and body heat distribution of passengers in the car through the on-board camera to assess body temperature and blood circulation status.

[0112] Near-infrared spectral sensor: used to monitor blood oxygen content and heart rate, monitor the oxygenation level of hemoglobin in the blood through the skin surface of passengers in the car, and calculate blood oxygen saturation and heart rate.

[0113] Millimeter-wave radar: uses low-power millimeter waves to monitor tiny chest movements through microwave reflection and accurately measures breathing rates through the clothing of passengers in the car.

[0114] 2. Data processing and analysis module 402: In this embodiment, the vehicle can perform feature extraction and pattern recognition on the collected images and radar signals through deep learning algorithms, and perform real-time analysis.

[0115] In this embodiment, the vehicle can use computer vision and deep learning technology to automatically identify the position and posture of passengers, and accurately match each vital sign signal to the corresponding passenger even in a dynamic vehicle environment, when the seat moves or when multiple people are riding together. The deep learning algorithm is used to perform real-time analysis on the collected multi-source data, extract features and perform pattern recognition on the collected images and radar signals, and improve the accuracy of vital sign recognition; and realize the fusion of multi-modal data, and use weighted average to integrate the data of each sensor to enhance the monitoring stability and accuracy.

[0116] 3. Intelligent early warning system 403: In this embodiment, the vehicle includes a health benchmark model for immediately starting the early warning process when abnormalities are detected among the passengers in the vehicle.

[0117] In this embodiment, a health benchmark model is established inside the vehicle for comparing real-time monitoring data. Once an abnormality is detected, the early warning process is immediately initiated, and different levels of response strategies are adopted according to the abnormal level of the vital signs of the passengers in the vehicle. For example, if the body temperature of the passengers in the vehicle is detected to be too high or too low, a warning is issued to the driver through the vehicle information system, and the temperature in the vehicle is adjusted to a temperature suitable for the current body temperature of the passengers; when a life threat is detected, such as cardiac arrest or severe hypoxia, the vehicle immediately activates the in-vehicle warning system and sends a request for help to the emergency service department through the Internet of Vehicles platform, including key information such as the vehicle's GPS (Global Positioning System) coordinates and occupant status, and in the entire data transmission process, encrypted transmission and anonymous processing technology are introduced, and end-to-end encryption technology is used to ensure data transmission security, ensuring the security and privacy protection of all physiological data during storage and transmission.

[0118] The present embodiment provides an in-vehicle vital signs recognition system and method based on multimodal sensing technology. The system integrates non-contact biosensors, advanced image processing technology, artificial intelligence algorithms, and emergency response mechanisms. It can achieve real-time and continuous monitoring of key vital indicators such as heart rate, respiratory rate, and body temperature of vehicle occupants under various driving conditions by integrating multiple non-invasive sensing technologies and intelligent algorithms, and automatically trigger rescue measures under abnormal conditions. It can monitor key vital indicators such as heart rate, respiratory rate, and blood oxygen saturation of occupants without sensing them under various driving conditions of the vehicle, and can respond quickly to abnormal situations, effectively improving the accuracy of the vehicle's vital signs detection and the efficiency of vehicle early warning.

[0119] The above is only an embodiment of the present application and should not be regarded as limiting the present application. Professionals should understand that various deformations and modifications can be made to the embodiments to adapt to different application requirements. Therefore, the scope of the present application should be limited by the claims attached to the claims.

[0120] Please refer to Figure 5 , which shows a block diagram of a vehicle warning device provided by an exemplary embodiment of the present application. The data processing device can be implemented as all or part of a computer device by hardware or a combination of hardware and software to achieve the above Figure 2 and Figure 3 All or part of the steps in the embodiments shown. Figure 5 As shown, the vehicle warning device includes:

[0121] The position and posture information acquisition module 501 is used to acquire the position and posture information of the target person inside the vehicle; the position and posture information is used to indicate the position and posture of the target person;

[0122] A target space region positioning module 502, configured to locate a target space region based on the position and posture information of the target person, wherein the target space region is a space region where the target person is located among at least two space regions;

[0123] The vital sign information acquisition module 503 is used to perform detection on the target person through at least two target sensors to obtain vital sign information corresponding to the at least two target sensors respectively; the at least two target sensors are at least two vital sign sensors corresponding to the target space area; the vital sign information is used to indicate the vital signs of the target person; the vital sign information at least includes body temperature, blood sample saturation, heart rate and respiratory rate;

[0124] The target sensor data acquisition module 504 is used to perform a weighted average operation on the vital sign information corresponding to at least two target sensors, and obtain the weighted average data as the target sensor data;

[0125] The intelligent warning detection module 505 inputs the target sensor data into the intelligent warning system in the vehicle to obtain the output result of the intelligent warning system;

[0126] The warning operation execution module 506 is used to execute the vehicle warning operation based on the output result of the intelligent warning system.

[0127] In some embodiments, the warning operation execution module 506 is used to input the target sensor data into the intelligent warning system, and obtain the vital sign recognition result through the intelligent warning system; the vital sign recognition result is used to indicate the vital sign status of the target person;

[0128] Based on the vital signs recognition results, early warning operations are performed.

[0129] In some embodiments, the early warning operation execution module 506 is used to determine whether the vital signs of the target person are abnormal based on the vital sign recognition result;

[0130] In the case of abnormal vital signs of the target person, determine the warning level of the vital sign recognition result;

[0131] Based on the warning level, a warning operation corresponding to the warning level is performed.

[0132] In some embodiments, the apparatus further comprises:

[0133] The weight determination module is used to perform a weighted averaging operation on the vital sign information corresponding to at least two target sensors respectively, and before obtaining the weighted averaged data as the target sensor data, determine the weight of the vital sign information in the weighted averaging operation according to the posture of the target person and the positions of the at least two target sensors relative to the target space area.

[0134] In some embodiments, the apparatus further comprises:

[0135] A vital sign information acquisition module, used to generate a vital sign information data threshold based on historical vital sign information; the historical vital sign information is vital sign information acquired by at least two target sensors before acquiring the vital sign information of the target person this time;

[0136] The vital signs warning information issuing module is used to issue vital signs warning information when the data of the vital signs information corresponding to at least two target sensors exceed the vital signs information data threshold; the vital signs warning information is used to indicate that the vital signs of the target person are abnormal.

[0137] In some embodiments, the apparatus further comprises:

[0138] The help information sending module is used to send a help information to the emergency service department through the vehicle's Internet of Vehicles platform when the difference between the vital signs information data corresponding to at least two target sensors and the vital signs information data threshold reaches a specified value; the help information includes the vehicle's location information and the target person's vital signs information.

[0139] Please refer to Figure 6 , Figure 66 is a schematic diagram of the structure of a computer device provided by an exemplary embodiment of the present application. The computer device 600 includes a central processing unit (CPU) 601, a system memory 604 including a random access memory (RAM) 602 and a read-only memory (ROM) 603, and a system bus 605 connecting the system memory 604 and the central processing unit 601. The computer device 600 also includes a basic input / output system (I / O system) 606 that helps transmit information between various devices in the computer, and a large-capacity storage device 607 for storing an operating system 613, application programs 614 and other program modules 615.

[0140] The basic input / output system 606 includes a display 608 for displaying information and an input device 609 such as a mouse and a keyboard for user inputting information. The display 608 and the input device 609 are connected to the central processing unit 601 through an input / output controller 610 connected to the system bus 605. The basic input / output system 606 may also include an input / output controller 610 for receiving and processing inputs from a plurality of other devices such as a keyboard, a mouse, or an electronic stylus. Similarly, the input / output controller 610 also provides output to a display screen, a printer, or other types of output devices.

[0141] The mass storage device 607 is connected to the central processing unit 601 through a mass storage controller (not shown) connected to the system bus 605. The mass storage device 607 and its associated computer readable media provide non-volatile storage for the computer device 600. That is, the mass storage device 607 may include a computer readable medium (not shown) such as a hard disk or a CD-ROM (Compact Disc Read-Only Memory) drive.

[0142] Without loss of generality, computer readable media may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer readable instructions, data structures, program modules or other data. Computer storage media include RAM (Random Access Memory), ROM (Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory or other solid-state storage technologies, CD-ROM, DVD (Digital Video Disc) or other optical storage, cassettes, magnetic tapes, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media are not limited to the above. The above-mentioned system memory 604 and mass storage device 607 can be collectively referred to as memory.

[0143] The computer device 600 can be connected to the Internet or other network devices through a network interface unit 611 connected to the system bus 605 .

[0144] The memory also includes one or more programs, one or more programs are stored in the memory, and the central processing unit 601 implements the one or more programs by executing the one or more programs. Figure 2 and Figure 3 All or part of the steps in the method shown.

[0145] In an exemplary embodiment, a chip is also provided. The chip includes a programmable logic circuit and / or program instructions. When the chip runs on a computer device, it is used to implement all or part of the steps of the method shown in the above embodiments of the present application.

[0146] In an exemplary embodiment, a computer program product is also provided, the computer program product includes computer instructions, the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor reads and executes the computer instructions from the computer-readable storage medium to implement all or part of the steps of the method shown in the above-mentioned various embodiments of the present application.

[0147] In an exemplary embodiment, a computer-readable storage medium is also provided, in which a computer program is stored. The computer program is loaded and executed by a processor to implement all or part of the steps of the method shown in the above-mentioned embodiments of the present application.

[0148] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program. The above program may be stored in a computer-readable storage medium. The above storage medium may be a read-only memory, a disk or an optical disk, etc.

[0149] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented with hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that a general or special-purpose computer can access.

[0150] The above are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A vehicle early warning method, characterized in that: The method is performed by a vehicle, wherein the vehicle includes at least two spatial areas, each of the spatial areas is configured with at least two vital sign sensors, and the method comprises: Acquire position and posture information of a target person inside the vehicle; the position and posture information is used to indicate the position and posture of the target person; Locating a target spatial area based on the position and posture information of the target person, wherein the target spatial area is a spatial area where the target person is located in the at least two spatial areas; Detecting the target person through at least two target sensors to obtain vital sign information corresponding to the at least two target sensors respectively; the at least two target sensors are at least two vital sign sensors corresponding to the target space area; the vital sign information is used to indicate the vital signs of the target person; the vital sign information includes at least body temperature, blood sample saturation, heart rate and respiratory rate; Performing a weighted average operation on the vital sign information respectively corresponding to the at least two target sensors, and acquiring weighted average data as target sensor data; Inputting the target sensor data into the intelligent warning system in the vehicle to obtain an output result of the intelligent warning system; Based on the output result of the intelligent warning system, a vehicle warning operation is performed.

2. The method according to claim 1, characterized in that: The step of inputting the target sensor data into the intelligent warning system and performing a vehicle warning operation based on an output result of the intelligent warning system includes: Inputting the target sensing data into the intelligent early warning system, and obtaining a vital sign recognition result through the intelligent early warning system; the vital sign recognition result is used to indicate the vital sign status of the target person; Based on the vital sign recognition result, an early warning operation is performed.

3. The method according to claim 2, characterized in that The performing of the early warning operation based on the vital sign recognition result includes: According to the vital sign recognition result, determining whether the vital sign of the target person is abnormal; In the case where the vital signs of the target person are abnormal, determining the warning level of the vital sign recognition result; Based on the warning level, a warning operation corresponding to the warning level is performed.

4. The method according to claim 1, characterized in that Before performing a weighted average operation on the vital sign information respectively corresponding to the at least two target sensors and obtaining the weighted averaged data as the target sensing data, the method further includes: The weight of the vital sign information in the weighted averaging operation is determined according to the posture of the target person and the positions of the at least two target sensors relative to the target space area.

5. The method according to claim 1, characterized in that: The method further comprises: Generate a vital sign information data threshold based on historical vital sign information; the historical vital sign information is vital sign information acquired by the at least two target sensors before acquiring the vital sign information of the target person this time; When the data of the vital sign information respectively corresponding to the at least two target sensors exceed the vital sign information data threshold, a vital sign warning message is issued; the vital sign warning message is used to indicate that the vital sign of the target person is abnormal.

6. The method according to claim 5, characterized in that The method further comprises: When the difference between the vital signs information data corresponding to the at least two target sensors and the vital signs information data threshold reaches a specified value, a help message is sent to the emergency service department through the vehicle networking platform; the help message includes the location information of the vehicle and the vital signs information of the target person.

7. A vehicle warning device, characterized in that: The device comprises: A position and posture information acquisition module, used to acquire the position and posture information of the target person inside the vehicle; the position and posture information is used to indicate the position and posture of the target person; A target space region positioning module, used for positioning a target space region based on the position and posture information of the target person, wherein the target space region is a space region where the target person is located in the at least two space regions; a vital sign information acquisition module, configured to perform detection on the target person through at least two target sensors, and obtain vital sign information corresponding to the at least two target sensors respectively; the at least two target sensors are at least two vital sign sensors corresponding to the target space area; the vital sign information is used to indicate the vital signs of the target person; the vital sign information includes at least body temperature, blood sample saturation, heart rate and respiratory rate; A target sensing data acquisition module, used to perform a weighted average operation on the vital sign information respectively corresponding to the at least two target sensors, and acquire the weighted average data as the target sensing data; An intelligent early warning detection module inputs the target sensor data into an intelligent early warning system in the vehicle to obtain an output result of the intelligent early warning system; The warning operation execution module is used to execute the vehicle warning operation based on the output result of the intelligent warning system.

8. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores instructions, and the instructions are executed by the processor to implement the vehicle warning method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The storage medium stores instructions, and the instructions are executed by a processor of a computer device to implement the vehicle warning method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; the computer instructions are read and executed by a processor of a computer device to implement the vehicle warning method as described in any one of claims 1 to 6.

Images