Vehicle control method and device, storage medium, electronic equipment and vehicle

Abstract

The invention provides a vehicle control method and device, a storage medium, electronic equipment and a vehicle. The method comprises the following steps: in response to a detected cough sound of a user, obtaining a face image corresponding to the user; then based on the face image corresponding to the user, identifying mouth features corresponding to the user; according to the mouth features and the volume of the cough sound, a droplet propagation path corresponding to the user is detected; and finally, ventilation in the vehicle is controlled according to the droplet propagation path. The method can be applied to an intelligent cabin, automatic detection of the propagation path of droplets generated by cough and automatic control of ventilation in the vehicle are achieved, the droplets generated by cough are treated in time in a ventilation mode in the vehicle, air containing the droplets is exhausted out of the vehicle, the air quality of the space in the vehicle is improved, and the safety of the vehicle is improved. Therefore, the risk that viruses or bacteria are transmitted to other users in the vehicle through droplets released by cough is reduced.

Description

Technical Field

[0001] The present disclosure relates to the technical field of vehicles, and in particular, to a vehicle control method, apparatus, storage medium, electronic device, and vehicle. Background Art

[0002] With the diversification of travel needs, users usually choose to drive a vehicle for travel, which allows them to adjust the destination or make stops midway at any time, facilitating commuting or traveling.

[0003] During the use of a vehicle, there is currently no effective means to handle the situation where a user coughs inside the vehicle. Since the interior space of the vehicle is small and enclosed, it increases the risk of viruses or bacteria being transmitted to other users inside the vehicle through the droplets released by coughing. Summary of the Invention

[0004] The present disclosure provides a vehicle control method, apparatus, storage medium, electronic device, and vehicle, mainly aiming to improve the problem that during the use of a vehicle, there is currently no effective means to handle the situation where a user coughs inside the vehicle. Since the interior space of the vehicle is small and enclosed, it increases the risk of viruses or bacteria being transmitted to other users inside the vehicle through the droplets released by coughing.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided a vehicle control method, including:

[0006] In response to detecting the coughing sound of a user, obtain the facial image corresponding to the user;

[0007] Based on the facial image corresponding to the user, identify the oral feature corresponding to the user;

[0008] According to the oral feature and the volume of the coughing sound, detect the droplet transmission path corresponding to the user;

[0009] Control the in-vehicle ventilation according to the droplet transmission path.

[0010] Optionally, the detecting the droplet transmission path corresponding to the user according to the oral feature and the volume of the coughing sound includes:

[0011] Based on the oral feature and the volume of the coughing sound, detect the droplet transmission direction and droplet transmission range corresponding to the user;

[0012] Determine the droplet transmission path corresponding to the user according to the droplet transmission direction and the droplet transmission range.

[0013] Optionally, the controlling the in-vehicle ventilation according to the droplet transmission path includes:

[0014] Obtain the seat area corresponding to the droplet transmission path;

[0015] Determine the fresh air ventilation mode of the seat area;

[0016] Control the in-vehicle ventilation of the seat area according to the fresh air ventilation mode.

[0017] Optionally, the determining the fresh air ventilation mode of the seat area includes:

[0018] Determine the fresh air ventilation direction corresponding to the seat area according to the droplet transmission direction;

[0019] Determine the fresh air ventilation speed corresponding to the seat area according to the droplet transmission range.

[0020] Optionally, the recognizing the mouth feature corresponding to the user based on the facial image corresponding to the user includes:

[0021] Recognize the mouth opening direction and the mouth opening range corresponding to the user based on the facial image corresponding to the user;

[0022] Optionally, the detecting the droplet transmission direction and the droplet transmission range corresponding to the user based on the mouth feature and the volume of the cough sound includes:

[0023] Detect the droplet transmission direction corresponding to the user based on the mouth opening direction;

[0024] Detect the droplet transmission range corresponding to the user based on the mouth opening range and the volume of the cough sound.

[0025] Optionally, the recognizing the mouth opening direction and the mouth opening range corresponding to the user based on the facial image corresponding to the user includes:

[0026] Detect the facial orientation of the user according to the facial area and the background area in the facial image;

[0027] Recognize the mouth opening direction based on the shooting angle of the in-vehicle shooting device and the facial orientation of the user;

[0028] Recognize the mouth opening range according to the mouth area in the facial image.

[0029] Optionally, microphones are respectively arranged in different seat areas in the vehicle;

[0030] The obtaining the facial image corresponding to the user includes:

[0031] Determine the seat area corresponding to the user according to the position of the microphone that collects the cough sound;

[0032] Directly capture the seat area to obtain the facial image of the user when coughing.

[0033] Optionally, before obtaining the facial image of the user corresponding to the detected cough sound of the user, the method further includes:

[0034] Collect in-vehicle audio data and preprocess the in-vehicle audio data;

[0035] Segment the preprocessed in-vehicle audio data into audio segments and detect the cough sound in the audio segments.

[0036] Optionally, the method further includes:

[0037] Detect the droplet landing points corresponding to the droplet transmission path;

[0038] Based on the in-vehicle scene image, display the droplet transmission path and the droplet landing points.

[0039] Optionally, the method further includes:

[0040] Generate a cleaning reminder corresponding to the droplet transmission path and the droplet landing points.

[0041] According to the second aspect of the embodiments of the present disclosure, a vehicle control device is provided, including:

[0042] An acquisition module configured to obtain the facial image of the user corresponding to the detected cough sound of the user;

[0043] An identification module configured to identify the oral features of the user based on the facial image of the user;

[0044] A detection module configured to detect the droplet transmission path of the user according to the oral features and the volume of the cough sound;

[0045] A control module configured to control the in-vehicle ventilation according to the droplet transmission path.

[0046] According to the third aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the vehicle control method described in the first aspect is implemented.

[0047] According to the fourth aspect of the embodiments of the present disclosure, an electronic device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor. When the processor executes the computer program, the vehicle control method described in the first aspect is implemented.

[0048] According to a fifth aspect of the embodiments of the present disclosure, a chip is provided, including one or more interface circuits and one or more processors; the interface circuit is configured to receive a signal from a memory of an electronic device and send the signal to the processor, the signal including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device is caused to execute the method as described in the first aspect.

[0049] According to a sixth aspect of the embodiments of the present disclosure, a vehicle is provided, including the device as described in the second aspect, or the electronic device as described in the fourth aspect, or the chip as described in the fifth aspect.

[0050] By means of the above technical solutions, a vehicle control method, device, storage medium, electronic device and vehicle provided by the present disclosure, wherein the method includes: in response to detecting a user's cough sound, acquiring a facial image corresponding to the user; then, based on the facial image corresponding to the user, identifying oral features corresponding to the user; further, according to the oral features and the volume of the cough sound, detecting a droplet transmission path corresponding to the user; and finally, controlling the in-vehicle ventilation according to the droplet transmission path. By applying the technical solutions of the present disclosure, when detecting a cough sound of a user in the vehicle, the present disclosure can automatically collect the facial image of the user, identify the oral features of the facial image, obtain the oral state of the user when coughing, combine the volume of the cough sound, detect the droplet transmission path generated by the user when coughing, and automatically control the in-vehicle ventilation according to the path, realizing the automatic detection of the droplet transmission path generated by coughing and the automatic control of the in-vehicle ventilation. By means of in-vehicle ventilation, the droplets generated by coughing are timely processed to discharge the air containing droplets out of the vehicle, improving the air quality of the in-vehicle space, and further reducing the risk of viruses or bacteria being transmitted to other users in the vehicle through the droplets released by coughing.

[0051] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure and used together with the specification to explain the principles of the present disclosure.

[0053] Figure 1 A flowchart showing a vehicle control method provided by an embodiment of the present disclosure is shown;

[0054] Figure 2 A flowchart showing another vehicle control method provided by an embodiment of the present disclosure is shown;

[0055] Figure 3 A flowchart showing an example provided by an embodiment of the present disclosure is shown;

[0056] Figure 4 The figure shows a schematic structural diagram of a vehicle control device provided by an embodiment of the present disclosure;

[0057] Figure 5 The figure shows a schematic functional block diagram of a vehicle provided by an embodiment of the present disclosure. Detailed implementation manners

[0058] Here, some embodiments of the present disclosure will be described in detail, and examples thereof are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. Various changes, modifications, and equivalents of the methods, apparatuses, and / or systems described herein will become apparent after understanding the present disclosure. For example, the order of operations described herein is merely an example and is not limited to those set forth herein, but may be changed as will be apparent after understanding the present disclosure, except for operations that must be performed in a specific order. Additionally, descriptions of features known in the art may be omitted for the sake of clarity and conciseness. It should be noted that, without conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.

[0059] The implementation manners described in some embodiments of the present disclosure below do not represent all implementation manners consistent with the present disclosure. On the contrary, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0060] During the use of a vehicle, when a user (occupant) coughs or the like inside the vehicle, several hazards may arise, affecting the air quality inside the vehicle, as well as the health and safety of the user. Specifically, coughing is a common symptom of many respiratory diseases, such as influenza, colds, and COVID-19. Coughing inside a vehicle, especially without proper ventilation, can increase the risk of viruses or bacteria being transmitted to other people inside the vehicle through droplets, and coughing in a closed vehicle space, especially when the air conditioner recirculates the internal air, will reduce the air quality inside the vehicle, which can not only increase the risk of disease transmission but also cause discomfort to other passengers inside the vehicle.

[0061] To improve the above situation where there is no effective means to handle the case of a user coughing inside a vehicle during vehicle use, and due to the small and enclosed space inside the vehicle, which increases the risk of viruses or bacteria being transmitted to other users inside the vehicle through droplets released by coughing, an embodiment of the present disclosure provides a vehicle control method, as Figure 1 shown, this method can be applied to a vehicle control device or equipment, etc. for execution, and this method includes the following steps.

[0062] Step 101: In response to detecting the user's cough sound, obtain the facial image corresponding to the user.

[0063] In some embodiments, at least one highly sensitive microphone may be installed in the critical areas of the vehicle to ensure that all user areas inside the vehicle are covered, and the audio data inside the vehicle is recorded in real time. The audio data information may include, but is not limited to, sounds such as conversations and coughs of the occupants, ensuring continuous collection and preservation of the audio data.

[0064] In a specific application scenario, the cough sound in the audio data can be detected in real time. When a cough sound is recognized in the audio data, the in-vehicle image capture device (such as a camera) is activated to capture the facial image of the user who emits the cough sound, so as to detect the cough behavior of the occupants based on the in-vehicle camera and microphone, facilitating subsequent ventilation processing and reducing the risk of droplet transmission caused by coughing. Exemplarily, at least one image capture device can be deployed in an intelligent driving vehicle, distributed at different positions in the intelligent cockpit area to cover the overall in-vehicle scene. In response to detecting a coughing user, the camera corresponding to the user is called to capture the facial image of the user.

[0065] Step 102: Based on the facial image corresponding to the user, identify the mouth features corresponding to the user.

[0066] In some embodiments, when a cough sound is detected, a deep learning model can be used to process the facial image of the user to identify the mouth features of the user. Exemplarily, the mouth features may include, but are not limited to, the opening direction of the mouth, the opening range of the mouth, the opening size of the mouth, the opening degree of the mouth, the mouth opening, the lip shape, etc. The deep learning model (such as a recurrent neural network, a long short-term memory network, etc.) can be used to locate the key feature points in the mouth area (the mouth region) of the user, such as the corners of the mouth, the peaks of the lips, the valleys of the lips, etc.

[0067] Step 103: According to the mouth features and the volume of the cough sound, detect the droplet transmission path corresponding to the user.

[0068] Among them, the droplet transmission path may include, but is not limited to, the droplet transmission direction, the droplet transmission range, the droplet transmission speed, etc., which can be used to represent the transmission range of droplets generated by behaviors such as coughing.

[0069] In some embodiments, according to the identified mouth features of each user, the facial direction of the user during coughing can be detected, the direction of droplet transmission or diffusion can be detected based on this direction, the coughing intensity can be judged according to the volume of the cough sound, and the range of droplet transmission or diffusion can be predicted based on the coughing intensity, so as to predict the droplet transmission risk when a cough behavior is detected, which helps to take effective preventive measures and reduce the transmission risk of the virus in enclosed spaces such as inside the vehicle.

[0070] Step 104: Control the in-vehicle ventilation according to the droplet transmission path.

[0071] In some embodiments, after detecting a droplet transmission path, the corresponding local or overall ventilation device for the droplet transmission path can be activated, and the ventilation mode of the ventilation device can be adjusted to quickly discharge the air containing droplets outside the vehicle or guide it to a high-efficiency filter, reducing the risk of virus or bacteria transmission in the enclosed space, which helps to create a safer and healthier travel environment.

[0072] Exemplarily, for the case of multiple passengers, independent ventilation devices corresponding to different cockpit positions can be separately called based on the intelligent cockpit positions corresponding to different users for zoned or overall ventilation control. Among them, the ventilation devices can include, but are not limited to, air conditioning systems, fresh air devices, windows, etc. By controlling the in-vehicle ventilation devices, in-vehicle ventilation can be carried out in a timely manner to reduce the risk of cross-infection. For example, the direction and speed of the air-conditioning air outlets can be intelligently adjusted to dynamically adjust the air flow direction to avoid the spread of droplets to other passenger areas.

[0073] Compared with the current related technologies, when the cough sound of a user in the vehicle is detected in the embodiments of the present disclosure, the facial image of the user can be automatically collected, the oral features of the facial image can be recognized, the oral state of the user when coughing can be obtained, combined with the volume of the cough sound, the droplet transmission path generated by the user when coughing can be detected, and the in-vehicle ventilation can be automatically controlled according to this path, realizing the automatic detection of the droplet transmission path generated by coughing and the automatic control of in-vehicle ventilation. By means of in-vehicle ventilation, the droplets generated by coughing can be processed in a timely manner to discharge the air containing droplets outside the vehicle, improving the air quality of the in-vehicle space, and further reducing the risk of virus or bacteria being transmitted to other users in the vehicle through the droplets released by coughing.

[0074] For further illustrating the specific implementation process of the method as Figure 1 shown, as an optional way, the embodiments of the present disclosure provide the specific method as Figure 2 shown, and this method includes:

[0075] Step 201: Collect in-vehicle audio data and preprocess the in-vehicle audio data.

[0076] In some embodiments, microphones can be respectively arranged in different seat areas in the vehicle to collect the audio data of different users in the vehicle and record the audio data of the users in the vehicle in real time. There may be some noises in the audio data collected during driving. The collected in-vehicle audio data can be preprocessed first, and advanced audio processing technologies can be used to remove background noises, such as the sound of vehicle driving, air-conditioning sound, etc., to improve the recognition accuracy of cough sounds.

[0077] Step 202: Segment the preprocessed in-car audio data into audio segments, and detect cough sounds in the audio segments.

[0078] In some embodiments, sounds such as coughs usually last for a very short time. The pre-processed in-vehicle audio data can be used to divide the continuous audio stream into multiple short segments according to preset time. Processing shorter audio segments requires less computing resources than processing long continuous audio streams. This allows the system to process more data while maintaining efficient operation, helps to improve the accuracy of cough feature extraction, and can increase the recognition speed of cough sounds.

[0079] Specifically, a large number of labeled cough and non-cough audios can be used to train a machine learning model to distinguish coughs from other sounds. The pre-processed audio data can be input into the trained model, and the model can be used to analyze the audio features to determine whether there is a cough. Accordingly, when multiple people cough at the same time, the model can be used to identify the voices of different users, separate different sound sources, and locate the seat positions of multiple users.

[0080] Step 203: In response to detecting the coughing sound of the user, obtaining a corresponding facial image of the user.

[0081] In some embodiments, when the in-car audio data collected by the microphone identifies that a passenger begins to cough, the in-car camera can be activated to immediately take a picture of the user and obtain the user's facial image, ensuring clear facial expressions and mouth movements to capture the coughing behavior and facilitate subsequent droplet detection.

[0082] Optionally, obtaining a facial image corresponding to the user may specifically include: determining a seat area corresponding to the user based on a position of a microphone that collects coughing sounds; and taking a directionally photographed shot of the seat area to obtain a facial image of the user when coughing.

[0083] Specifically, in a smart cockpit, multiple cameras can be configured to capture the voices of passengers in different seat positions, and multiple microphones can be configured to capture the images of passengers in different seat positions. The time difference, phase difference or intensity difference of the sound signals received by the multiple microphones can be used to determine the location of the cough source, and then the seat position of the user who makes the coughing sound in the car can be determined. After locating the user's position, the camera corresponding to the position can be called to capture the user's facial image to obtain the user's facial information when coughing, so as to provide more personalized ventilation services for that seat position.

[0084] Step 204: Based on the facial image corresponding to the user, identify the mouth features corresponding to the user.

[0085] Optionally, step 204 may specifically include: identifying the mouth opening direction and the mouth opening range corresponding to the user based on the facial image corresponding to the user.

[0086] In some embodiments, the collected facial image may be preprocessed first. The preprocessing of the facial image may include, but is not limited to, steps such as adjusting brightness, contrast, cropping the image, removing irrelevant backgrounds, etc., to improve the accuracy and robustness of subsequent object detection algorithms. Then, a large amount of facial image data is used to train the model to learn different features of the mouth during coughing. The preprocessed image data is input into the model, and the model is used to identify the mouth opening direction and the mouth opening range of the user when coughing.

[0087] Optionally, identifying the mouth opening direction and the mouth opening range corresponding to the user based on the facial image corresponding to the user may specifically include: detecting the facial orientation of the user according to the facial area and the background area in the facial image; identifying the mouth opening direction based on the shooting angle of the in-vehicle shooting device and the facial orientation of the user; and identifying the mouth opening range according to the mouth area in the facial image.

[0088] In some embodiments, the facial area and the background area in the facial image may be identified first, and the facial orientation of the user may be detected according to the relative angle of the facial features in the facial area to the background area. Then, the system may determine the shooting angle of the camera relative to each user according to the installation positions of the cameras. Finally, based on the shooting angle and the facial orientation, the mouth opening direction of each user is identified for droplet transmission direction detection. Correspondingly, a target detection model may be used to mark the key feature points of the mouth area, and the mouth opening range of the user is calculated according to the detected feature point coordinates for droplet transmission range detection.

[0089] Step 205, detecting the droplet transmission direction and the droplet transmission range corresponding to the user based on the mouth features and the volume of the coughing sound.

[0090] Optionally, step 205 may specifically include: detecting the droplet transmission direction corresponding to the user based on the mouth opening direction; and detecting the droplet transmission range corresponding to the user based on the mouth opening range and the volume of the coughing sound.

[0091] In some embodiments, comprehensive analysis may be performed based on information such as the sound volume, the mouth direction, and the opening size, and an algorithm may be used to simulate the droplet transmission path to infer the possible transmission path and range of the droplets generated by coughing, so as to improve the detection accuracy of the droplet transmission path.

[0092] Exemplarily, the droplets mainly spread along the direction in which the mouth is opened. The direction in which the user's mouth is opened can be used as the droplet transmission direction corresponding to the user, or the droplet transmission direction can be obtained by spreading according to the direction in which the user's mouth is opened. Correspondingly, the range of mouth opening can be measured by calculating the vertical distance between the key points of the upper lip and the lower lip. The larger this distance is, the larger the range of mouth opening is, and more droplets may be released. A louder volume is usually associated with stronger airflow and may result in a farther droplet transmission range. Multiple mouth opening thresholds and multiple volume thresholds can be set to divide the range of mouth opening and the volume of the cough sound into multiple levels, and the corresponding droplet transmission range can be predicted according to different levels.

[0093] Step 205: Determine the droplet transmission path corresponding to the user according to the droplet transmission direction and the droplet transmission range.

[0094] In some embodiments, based on the detected droplet transmission direction and droplet transmission range, as well as in-vehicle environment information such as the current in-vehicle air flow direction and flow rate, the droplet transmission path corresponding to the user can be predicted to facilitate intelligent adjustment of the ventilation equipment, avoid the spread of droplets to the areas where other passengers are located, and reduce the risk of droplet transmission to other users in the vehicle.

[0095] Step 206: Control the in-vehicle ventilation according to the droplet transmission path.

[0096] Optionally, step 206 may specifically include: obtaining the seat area corresponding to the droplet transmission path; determining the fresh air ventilation mode of the seat area; and controlling the in-vehicle ventilation of the seat area according to the fresh air ventilation mode.

[0097] In some embodiments, according to the predicted droplet transmission path, the seat area involved in the droplet transmission path is identified, and the fresh air equipment corresponding to the seat area can be controlled to adjust the fresh air ventilation mode according to the droplet transmission path, and the in-vehicle ventilation treatment is performed on the seat area. To increase the local exhaust air volume for the area involved in the droplet transmission path, the air containing droplets is preferentially discharged outside the vehicle rather than recycled back into the vehicle, and clean air can be provided to this area to ensure a healthy environment, thereby reducing the risk of bacteria transmission in the enclosed space and providing a healthier driving environment for users.

[0098] Optionally, determining the fresh air ventilation mode of the seat area may specifically include: determining the fresh air ventilation direction corresponding to the seat area according to the droplet transmission direction; and determining the fresh air ventilation wind speed corresponding to the seat area according to the droplet transmission range.

[0099] Exemplarily, in the application of the intelligent cockpit, the ventilation direction of the fresh air system can be adjusted by adjusting the angles of the local air inlets and outlets, and the air flow direction inside the vehicle can be adjusted to quickly remove droplets and avoid blowing the air containing droplets to the unaffected areas. The air inlets can effectively introduce fresh external air to replace the polluted air inside the vehicle. The air inlets should be located in relatively clean areas inside the vehicle to avoid inhaling the polluted air just discharged from the outlets, while the outlets are close to the pollution sources. Correspondingly, if the range of droplet transmission is large, the ventilation speed can be increased in the seat areas involved in the droplet transmission range to quickly increase the fresh air volume, thereby ensuring the rapid discharge of the air containing droplets.

[0100] In this way, the fresh air system can be called to automatically adjust the ventilation mode according to the droplet transmission path, specifically strengthen the local ventilation or adjust the overall ventilation effect to ensure the best air quality inside the vehicle.

[0101] Optionally, the method of this embodiment may specifically further include: detecting the droplet landing points corresponding to the droplet transmission path; and displaying the droplet transmission path and the droplet landing points based on the in-vehicle scene image.

[0102] In some embodiments, after predicting the droplet transmission paths of each user, devices such as cameras can be called to continuously observe the transmission situation of the droplets and detect the landing points of the droplets in the in-vehicle space. Combining with the captured in-vehicle scene image, the droplet transmission path and the droplet landing points can be marked and displayed in the vehicle head unit for the users to disinfect and clean.

[0103] Correspondingly, a cleaning reminder corresponding to the droplet transmission path and the droplet landing points can also be generated. If the user disinfects the area involved in the droplet transmission path within a certain time, a cleaning reminder can be generated in the vehicle head unit, and the reminder methods can include pop-up reminders and voice reminders to maintain a healthy environment inside the vehicle.

[0104] As a possible implementation manner, as Figure 3 shown, the vehicle control may include the following steps:

[0105] 1. Collect audio data through the microphone

[0106] Device preparation: Install high-sensitivity microphones in key areas of the vehicle to ensure that all passenger areas inside the vehicle can be covered;

[0107] Data collection: Record the in-vehicle audio data in real time, including the voices of passengers' conversations, coughs, etc., to ensure the continuous collection and preservation of the audio data.

[0108] 2. Audio data preprocessing

[0109] Noise reduction: Use advanced audio processing technologies to remove background noises such as vehicle driving sounds and air conditioner sounds to improve the recognition accuracy of cough sounds;

[0110] Sound segmentation: Segment the continuous audio stream into short segments for subsequent cough detection and analysis.

[0111] 3. Detect whether the occupant is coughing based on audio data through the model

[0112] Model training: Use a large number of labeled cough audio and non-cough audio to train a machine learning model to distinguish cough sounds from other sounds;

[0113] Cough recognition: Input the preprocessed audio data into the trained model, and the model analyzes the audio features to determine whether there is a cough sound.

[0114] 4. When a cough is detected, collect the facial image data of the occupant during coughing through the camera

[0115] Camera activation: When a cough sound is detected, immediately activate the in-vehicle camera to take a directed shot of the area of the occupant who made the cough sound;

[0116] Image acquisition: Capture the facial image of the occupant at the moment of coughing to ensure clear facial expressions and mouth movements are obtained.

[0117] 5. Image data preprocessing

[0118] Image cleaning: Clean the captured images, including adjusting brightness, contrast, and cropping the images to remove irrelevant backgrounds;

[0119] Feature extraction: Extract key facial features such as the position and size of the mouth area for subsequent analysis.

[0120] 6. Detect the direction of the occupant's mouth and the size of the mouth opening during coughing through a machine learning model

[0121] Model training: Use a large amount of facial image data to train the model to learn different features of the mouth during coughing;

[0122] Mouth state recognition: Input the preprocessed image data into the model, and the model recognizes the direction and opening degree of the mouth during coughing.

[0123] 7. Comprehensively judge the droplet path of the occupant's cough

[0124] Data fusion: Comprehensively analyze information such as sound volume, mouth direction, and opening size, and use algorithms to simulate the droplet transmission path;

[0125] Path inference: Based on the above data, infer the possible transmission path and range of cough droplets.

[0126] 8. The detected droplet path is reported to the vehicle system

[0127] Information reporting: The analysis result of the droplet transmission path is reported to the vehicle control system in real time;

[0128] Intelligent ventilation adjustment: The vehicle system intelligently adjusts the working mode of the ventilation system according to the information of droplet transmission, such as increasing the fresh air volume and adjusting the air flow direction, to reduce the spread of droplets in the vehicle.

[0129] By applying the technical solution of the embodiment of the present disclosure, the preprocessed in-vehicle audio data can be segmented into multiple short segments, reducing the computing resources for identifying cough sounds, improving the recognition speed of cough sounds, and based on the position of the microphone that collects the cough sound, locating the seat area of the user who makes the cough sound, taking a directional photograph of the seat area, obtaining the facial image of the user when coughing, identifying the opening direction and opening range of the user's mouth based on the facial image, then detecting the droplet transmission direction according to the opening direction of the mouth, and detecting the corresponding droplet transmission range of the user according to the opening range of the mouth and the volume of the cough sound, improving the detection accuracy of the droplet transmission path, and finally automatically adjusting the fresh air ventilation mode according to the droplet transmission path, providing clean air for the seat area corresponding to the droplet transmission path, thereby reducing the risk of bacteria transmission in the enclosed space and providing a healthier driving environment for the user.

[0130] Figure 4 is a block diagram of a vehicle control device shown according to some embodiments of the present disclosure. Referring to Figure 4 , the device includes: an acquisition module 31, an identification module 32, a detection module 33, and a control module 34.

[0131] The acquisition module 31 is configured to obtain a facial image corresponding to the user in response to detecting the cough sound of the user;

[0132] The identification module 32 is configured to identify the mouth features corresponding to the user based on the facial image corresponding to the user;

[0133] The detection module 33 is configured to detect the droplet transmission path corresponding to the user according to the mouth features and the volume of the cough sound;

[0134] The control module 34 is configured to control the in-vehicle ventilation according to the droplet transmission path.

[0135] In some embodiments, the detection module 33 is specifically configured to detect the droplet transmission direction and droplet transmission range corresponding to the user based on the mouth features and the volume of the cough sound; determine the droplet transmission path corresponding to the user according to the droplet transmission direction and droplet transmission range.

[0136] In some embodiments, the control module 34 is specifically configured to obtain the seat area corresponding to the droplet transmission path; determine the fresh air ventilation mode of the seat area; and control the in-vehicle ventilation of the seat area according to the fresh air ventilation mode.

[0137] In some embodiments, the control module 34 is specifically configured to determine the fresh air ventilation direction corresponding to the seat area according to the droplet transmission direction; and determine the fresh air ventilation speed corresponding to the seat area according to the droplet transmission range.

[0138] In some embodiments, the recognition module 32 is specifically configured to recognize the mouth opening direction and the mouth opening range corresponding to the user based on the facial image of the user; and detect the droplet transmission direction and the droplet transmission range corresponding to the user based on the mouth characteristics and the volume of the cough sound, including: detecting the droplet transmission direction corresponding to the user based on the mouth opening direction; and detecting the droplet transmission range corresponding to the user based on the mouth opening range and the volume of the cough sound.

[0139] In some embodiments, the detection module 33 is specifically configured to detect the facial orientation of the user according to the facial area and the background area in the facial image; recognize the mouth opening direction based on the shooting angle of the in-vehicle shooting device and the facial orientation of the user; and recognize the mouth opening range according to the mouth area in the facial image.

[0140] In some embodiments, microphones are respectively arranged in different seat areas in the vehicle; the acquisition module 31 is specifically configured to determine the seat area corresponding to the user according to the position of the microphone that collects the cough sound; and perform directional shooting on the seat area to obtain the facial image of the user when coughing.

[0141] In some embodiments, the acquisition module 31 is specifically further configured to collect in-vehicle audio data, preprocess the in-vehicle audio data; segment the preprocessed in-vehicle audio data into audio segments, and detect the cough sound in the audio segments.

[0142] In some embodiments, the acquisition module 31 is specifically further configured to detect the droplet landing points corresponding to the droplet transmission path; and display the droplet transmission path and the droplet landing points based on the in-vehicle scene image.

[0143] In some embodiments, the acquisition module 31 is specifically further configured to generate a cleaning prompt corresponding to the droplet transmission path and the droplet landing points.

[0144] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0145] It should be noted that for other corresponding descriptions of the various functional units involved in the vehicle control device provided in the embodiments of the present disclosure, reference may be made to Figures 1 to 2 the corresponding descriptions therein, which will not be elaborated herein.

[0146] Figure 5 FIG. Figures 1 to 2 is a block diagram of a vehicle 600 shown according to an exemplary embodiment. For example, the vehicle 600 may be a hybrid vehicle, or a non - hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicles. The vehicle 600 may be an autonomous vehicle, a semi - autonomous vehicle, or a non - autonomous vehicle.

[0147] Referring to Figure 5 , the vehicle 600 may include various subsystems. For example, the infotainment system 610, the perception system 620, the decision - making and control system 630, the drive system 640, and the computing platform 650. Among them, the vehicle 600 may also include more or fewer subsystems, and each subsystem may include multiple components. In addition, each subsystem and each component of the vehicle 600 may be interconnected by wired or wireless means.

[0148] In some embodiments, the infotainment system 610 may include a communication system, an entertainment system, and a navigation system, etc.

[0149] The perception system 620 may include several types of sensors for sensing information about the environment around the vehicle 600. For example, the perception system 620 may include a global positioning system (the global positioning system may be a GPS system, or a Beidou system, or other positioning systems), an inertial measurement unit (IMU), lidar, millimeter - wave radar, ultrasonic radar, and a camera device.

[0150] The decision - making and control system 630 may include a computing system, a vehicle controller, a steering system, an accelerator, and a braking system.

[0151] The drive system 640 may include components that provide motive power for the vehicle 600. In one embodiment, the drive system 640 may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of an internal combustion engine, an electric motor, and an air - compression engine. The engine is capable of converting the energy provided by the energy source into mechanical energy.

[0152] Some or all of the functions of the vehicle 600 are controlled by the computing platform 650. The computing platform 650 may include at least one processor 651 and a memory 652. The processor 651 may execute instructions 653 stored in the memory 652.

[0153] The processor 651 can be any conventional processor, such as a commercially available CPU. The processor may also include, for example, a Graphic Process Unit (GPU), a Field Programmable Gate Array (FPGA), a System on Chip (SOC), an Application Specific Integrated Circuit (ASIC), or a combination thereof.

[0154] The memory 652 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0155] In addition to the instructions 653, the memory 652 can also store data, such as road maps, route information, data on the position, direction, speed, etc. of the vehicle. The data stored in the memory 652 can be used by the computing platform 650.

[0156] In the embodiments of the present disclosure, the processor 651 can execute the instructions 653 to complete all or part of the steps of the above-mentioned vehicle control method.

[0157] The present disclosure also provides a computer-readable storage medium, on which computer program instructions are stored, and when the program instructions are executed by a processor, the above-mentioned Figures 1 to 2 shown method is implemented.

[0158] Based on such an understanding, the technical solution of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various implementation scenarios of the present disclosure.

[0159] Based on the above-mentioned Figures 1 to 2 shown method, and Figure 4 shown virtual device embodiments, in order to achieve the above object, the embodiments of the present disclosure also provide an electronic device, such as a high-definition TV, a computer, a projector, etc. with a liquid crystal display, a plasma display, a digital light processing display, etc. The device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to implement the above-mentioned Figures 1 to 2 shown method.

[0160] Optionally, the above-mentioned entity device may further include a user interface, a network interface, a camera, a Radio Frequency (RF) circuit, sensors, an audio circuit, a WI-FI module, and so on. The user interface may include a display screen (Display), an input unit such as a keyboard (Keyboard), etc. Optionally, the user interface may further include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.

[0161] Those skilled in the art can understand that the above-mentioned entity device structure provided by the embodiments of the present disclosure does not constitute a limitation on the entity device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0162] The storage medium may further include an operating system and a network communication module. The operating system is a program for managing the hardware and software resources of the above-mentioned entity device, and supports the operation of information processing programs and other software and / or programs. The network communication module is used to implement communication between components inside the storage medium, as well as communication between other hardware and software in the information processing entity device.

[0163] Based on the above method as Figures 1 to 2 shown, and Figure 4 the virtual device embodiment as Figures 1 to 2 shown, the embodiments of the present disclosure further provide a chip, including one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from the memory of the electronic device and send the signal to the processor, and the signal includes computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device is caused to execute the above method as

[0164] Through the description of the above embodiments, those skilled in the art can clearly understand that the present disclosure can be implemented by means of software plus a necessary general hardware platform, or can also be implemented by hardware. Compared with the current related technologies, the embodiments of the present disclosure can segment the preprocessed in-vehicle audio data into multiple short segments, reduce the computing resources for identifying cough sounds, improve the recognition speed of cough sounds, and based on the position of the microphone that collects the cough sound, locate the seat area of the user who makes the cough sound, perform a directional shot on the seat area, obtain the facial image of the user when coughing, identify the opening direction and opening range of the user's mouth according to the facial image, then detect the droplet transmission direction according to the opening direction of the mouth, and detect the corresponding droplet transmission range of the user according to the opening range of the mouth and the volume of the cough sound, improve the detection accuracy of the droplet transmission path, and finally automatically adjust the fresh air ventilation mode according to the droplet transmission path, provide clean air for the seat area corresponding to the droplet transmission path, thereby reducing the transmission risk of bacteria in the enclosed space and providing a healthier driving environment for the user.

[0165] It should be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

[0166] The above are only specific embodiments of the present disclosure, enabling those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments described herein, but will be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A vehicle control method, characterized in that, Including: In response to detecting the user's cough sound, obtaining the facial image corresponding to the user; Based on the facial image corresponding to the user, identifying the oral features corresponding to the user; According to the oral features and the volume of the cough sound, detecting the droplet transmission path corresponding to the user; Controlling the in-vehicle ventilation according to the droplet transmission path.

2. The method according to claim 1, characterized in that The step of detecting the droplet transmission path corresponding to the user according to the oral features and the volume of the cough sound includes: Based on the oral features and the volume of the cough sound, detecting the droplet transmission direction and the droplet transmission range corresponding to the user; According to the droplet transmission direction and the droplet transmission range, determining the droplet transmission path corresponding to the user.

3. The method according to claim 2, wherein The step of controlling the in-vehicle ventilation according to the droplet transmission path includes: Obtaining the seat area corresponding to the droplet transmission path; Determining the fresh air ventilation mode of the seat area; Controlling the in-vehicle ventilation of the seat area according to the fresh air ventilation mode.

4. The method according to claim 3, wherein The step of determining the fresh air ventilation mode of the seat area includes: According to the droplet transmission direction, determining the fresh air ventilation direction corresponding to the seat area; According to the droplet transmission range, determining the fresh air ventilation speed corresponding to the seat area.

5. The method according to claim 2, wherein The step of identifying the oral features corresponding to the user based on the facial image corresponding to the user includes: Based on the facial image corresponding to the user, identifying the oral opening direction and the oral opening range corresponding to the user; The step of detecting the droplet transmission direction and the droplet transmission range corresponding to the user based on the oral features and the volume of the cough sound includes: Based on the oral opening direction, detecting the droplet transmission direction corresponding to the user; Based on the oral opening range and the volume of the cough sound, detecting the droplet transmission range corresponding to the user.

6. The method according to claim 5, wherein The step of identifying the oral opening direction and the oral opening range corresponding to the user based on the facial image corresponding to the user includes: According to the facial area and the background area in the facial image, detecting the facial orientation of the user; Based on the shooting angle of the in-vehicle shooting device and the facial orientation of the user, identifying the oral opening direction; According to the oral area in the facial image, identifying the oral opening range.

7. The method according to claim 1, wherein Microphones are respectively arranged in different seat areas in the vehicle; The step of obtaining the facial image corresponding to the user includes: According to the position of the microphone that collects the cough sound, determining the seat area corresponding to the user; Performing directional shooting on the seat area to obtain the facial image of the user during coughing.

8. The method according to claim 7, wherein Before the step of in response to detecting the user's cough sound and obtaining the facial image corresponding to the user, the method further includes: Collecting in-vehicle audio data and preprocessing the in-vehicle audio data; Dividing the preprocessed in-vehicle audio data into audio segments and detecting the cough sound in the audio segments.

9. The method according to claim 1, wherein The method further includes: Detecting the droplet landing point corresponding to the droplet transmission path; Based on the in-vehicle scene image, displaying the droplet transmission path and the droplet landing point.

10. The method according to claim 9, characterized in that, The method further includes: Generate cleaning tips corresponding to the droplet transmission path and the droplet landing point.

11. A vehicle control device, characterized in that, Including: An acquisition module, configured to acquire a facial image corresponding to the user in response to detecting the user's coughing sound; An identification module, configured to identify the oral features corresponding to the user based on the facial image corresponding to the user; A detection module, configured to detect the droplet transmission path corresponding to the user according to the oral features and the volume of the coughing sound; A control module, configured to control the in-vehicle ventilation according to the droplet transmission path.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 10.

13. An electronic device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 10.

14. A vehicle, characterized in that, Including: The device according to claim 11, or the electronic device according to claim 13.

Images