Vehicle cabin control method and device, electronic equipment and storage medium

Abstract

The invention discloses a vehicle cabin control method and device, electronic equipment and a storage medium, and the method comprises the steps: recognizing a child passenger of a vehicle cabin, and monitoring a first emotion state of the child passenger; in response to monitoring that the child occupant is in the to-be-pacified emotional state, obtaining identity information of the child occupant, determining a playlist and playing timbre matched with the child occupant according to the identity information, controlling the vehicle cabin to play pacifying content of the playlist by adopting the playing timbre, obtaining a second emotional state of the child occupant at a preset sampling interval, and sending the second emotional state to the child occupant; and according to the second emotional state, determining a disappearance proportion of the to-be-pacified emotional state of the child passenger, and according to the disappearance proportion of the to-be-pacified emotional state, adjusting the played pacifying content until the emotional state of the child passenger recovers to the normal emotional state. According to the method, the personalized pacifying content is provided for the child passenger to be pacified, the pacifying content is adjusted in time based on the pacifying effect, and it is ensured that the emotional state of the child passenger can be pacified flexibly and effectively.

Description

Technical Field

[0001] This application belongs to the field of vehicle control technology, specifically relating to a vehicle cockpit control method, device, electronic equipment, and storage medium. Background Technology

[0002] With the significant increase in family travel, children have become an important passenger group in vehicle cabins. During the journey, due to factors such as the enclosed cabin environment, boredom, physical discomfort, or emotional fluctuations, children often cry. This not only affects the atmosphere inside the car but also seriously interferes with the driver's attention, posing a potential driving safety risk. Therefore, how to soothe crying children in the car has become the key to improving the driving experience and driving safety.

[0003] Currently, the mainstream vehicle cabin systems offer the following soothing methods for this scenario: when a child cries, triggering or manually selecting a preset playlist of soothing music, or adjusting the ambient lighting to preset soft tones such as blue or pink. However, the music content and ambient lighting adjustment parameters are all pre-set uniformly, failing to consider the personalized needs of children of different ages, personalities, and causes of crying. Furthermore, simply activating music or adjusting lights makes it difficult to determine whether the current soothing method has a positive effect or alleviates the child's crying, thus failing to ensure that the goal of quickly and effectively calming the child's emotions is achieved, thereby affecting the driving experience and driving safety. Summary of the Invention

[0004] The purpose of this application is to provide a vehicle cockpit control method, device, electronic device, and storage medium that can solve the problem that traditionally used uniformly set soothing methods cannot ensure the rapid and effective achievement of soothing children's emotions, thereby affecting the driving experience and driving safety.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide a vehicle cockpit control method, the method comprising: Identify child occupants in the vehicle cabin and monitor the child occupants' primary emotional state; wherein, the emotional state includes a normal emotional state and an emotional state requiring reassurance; In response to detecting that the child occupant is in a state of needing to be comforted, the identity information of the child occupant is obtained; Based on the identity information, a playlist and playback tone matching the child occupant are determined, and the vehicle cabin is controlled to play the soothing content of the playlist using the playback tone; The second emotional state of the child occupant is obtained at a preset sampling interval, and the percentage of the disappearance of the child occupant's emotional state requiring reassurance is determined based on the second emotional state. Based on the percentage of the disappearance of the emotional state requiring reassurance, adjust the reassurance content played until the child passenger's emotional state returns to the normal emotional state.

[0006] Optionally, identifying child occupants in the vehicle cabin and monitoring their initial emotional state includes: The vehicle cabin occupants are detected, and child occupants are identified from among them; The facial image of the child occupant is monitored, and emotion recognition is performed on the facial image to obtain the child occupant's first emotional state.

[0007] Optionally, the step of determining a playlist and playback tone matching the child occupant based on the identity information, and controlling the vehicle cabin to play soothing content from the playlist using the playback tone, includes: Based on the identity information, a playlist is matched from a pre-created playlist library to be played to the child occupant; wherein, the playlist includes a general playlist or a personalized playlist of the child occupant; Obtain at least one playback tone that matches the child occupant's preference; The current audio and video stream of the soothing content in the playlist is converted to a different timbre to obtain the soothing content to be output as the playback timbre; Based on the playlist, control the vehicle cabin to play soothing content with the specified sound tone.

[0008] Optionally, the step of matching a playlist to be played for the child occupant from a pre-created playlist library based on the identity information includes: Based on the identity information, query whether the child passenger's personalized playlist exists in the pre-created playlist library; If so, the child passenger's personalized playlist is determined as the playlist to be played to the child passenger; If not, then based on the identity information, select comforting content that matches the child passenger's age group from the general playlist, generate a playable list, and determine the playable list as the playlist to be played to the child passenger.

[0009] Optionally, the step of obtaining the second emotional state of the child occupant at a preset sampling interval, and determining the percentage of the disappearance of the child occupant's emotional state requiring reassurance based on the second emotional state, includes: The second emotional state of the child occupant is obtained at a preset sampling interval; Based on a preset monitoring period, the second emotional state is classified and statistically analyzed to obtain the number of occurrences of normal emotional state and emotional state requiring reassurance. The percentage of the absence of the child passenger's emotional state requiring reassurance is determined based on the number of occurrences of normal emotional states and emotional states requiring reassurance within the monitoring period.

[0010] Optionally, adjusting the played comforting content based on the percentage of disappearance of the emotional state to be comforted until the child passenger's emotional state returns to the normal emotional state includes: Based on the percentage of the disappearance of the emotional state requiring reassurance, it is determined whether the reassurance content is effective for the child passenger, thus obtaining the effectiveness result of the reassurance content; wherein, the effectiveness result includes ineffective reassurance and effective reassurance; Based on the effectiveness results, adjust the soothing content played until the child passenger's emotional state returns to the normal emotional state.

[0011] Optionally, adjusting the played soothing content based on the effectiveness result until the child passenger's emotional state returns to the normal emotional state includes: If the playlist played to the child occupant is the child occupant's personalized playlist, and the cumulative number of invalid reassurances exceeds a preset threshold, then the reassurance content will be removed from the child occupant's personalized playlist. If the playlist played to the child occupant is the playable playlist, and the validity result is effective soothing, then the soothing content is marked in the child occupant's personalized playlist. Control the playback of soothing content from the adjusted playlist in the vehicle cabin until the child occupant's emotional state returns to its normal emotional state.

[0012] Secondly, embodiments of this application provide a vehicle cockpit control device, the device comprising: A status monitoring module is used to identify child occupants in the vehicle cabin and monitor the child occupants' first emotional state; wherein, the emotional state includes a normal emotional state and an emotional state that needs to be comforted; The information acquisition module is used to acquire the child occupant's identity information in response to detecting that the child occupant is in the state of needing to be comforted. The first control module is used to determine a playlist and playback tone that match the child passenger's preferences based on the identity information, and control the vehicle cabin to play the soothing content of the playlist using the playback tone; The status determination module is used to acquire the second emotional state of the child occupant at a preset sampling interval, and determine the percentage of the disappearance of the child occupant's emotional state requiring reassurance based on the second emotional state. The second control module is used to adjust the playing comforting content according to the percentage of the disappearance of the emotional state to be comforted until the child passenger's emotional state returns to the normal emotional state.

[0013] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the vehicle cockpit control method as described in the first aspect.

[0014] Fourthly, embodiments of this application provide a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the vehicle cockpit control method as described in the first aspect.

[0015] The vehicle cabin control method provided in this application identifies child occupants in the vehicle cabin and monitors their first emotional state. This emotional state includes a normal emotional state and a state requiring reassurance. The method monitors the child occupant's emotional state in real time, captures emotional changes, triggers reassurance measures, and, in response to detecting a child occupant in a state requiring reassurance, obtains the child occupant's identity information. Based on this identity information, it determines a playlist and playback tone that match the child occupant, and controls the vehicle cabin to play the reassurance content from the playlist using the appropriate playback tone. This method can adapt playlists to each child occupant based on their identity and control the playback tone accordingly. The system selects and plays content and sounds that child occupants prefer, providing personalized comforting content for those needing comfort. It acquires the child occupants' second emotional state at preset sampling intervals, determines the percentage of time their uncomforted emotional state disappears based on this second emotional state, and adjusts the comforting content accordingly until the child occupants' emotional state returns to normal. The system monitors and accurately quantifies the comforting and alleviating effects of the content on the child occupants' uncomforted emotional state, allowing for timely adjustments based on the comforting effect. This ensures flexible and effective comforting of child occupants' emotional states, further enhancing the driving experience and road safety.

[0016] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart illustrating the steps of a vehicle cockpit control method provided in an embodiment of this application; Figure 2 This is a schematic diagram of a vehicle cockpit control method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a vehicle cockpit control device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0020] The vehicle cockpit control method, device, electronic equipment, and storage medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0021] Reference Figure 1 The diagram illustrates a flowchart of the vehicle cockpit control method provided in an embodiment of this application. The method may include: Step 101: Identify child occupants in the vehicle cabin and monitor their initial emotional state; the emotional state includes a normal emotional state and an emotional state requiring reassurance.

[0022] In this embodiment, to effectively soothe the emotional states of child occupants in the vehicle cabin, such as crying, it is necessary to identify the child occupants and monitor their initial emotional state. Based on this initial emotional state, the system analyzes whether to trigger the cabin's intelligent soothing function. In this embodiment, the intelligent soothing function sends activation information to the cabin interface via the cabin domain controller. The cockpit interface can interact with cockpit users, such as the driver or front passenger, through voice prompts. For example, the cockpit interface might prompt: "The little master seems a little unhappy. Would you like to turn on the smart soothing function and play a soothing picture book for the little master?" Upon receiving voice or touch confirmation from the user, the cockpit interface will automatically open the smart soothing function page to play soothing content in the form of audio and video, such as picture books or music.

[0023] It should be noted that, referring to Figure 2 This illustration shows a scenario diagram of a vehicle cockpit control method provided in an embodiment of this application. The vehicle cockpit system includes a cockpit domain controller, a cockpit interface, a visual recognition module for collecting occupant information, and a cockpit speaker. This embodiment describes the cockpit domain controller as the executing entity. The cockpit domain controller receives the occupant's emotional state collected by the visual recognition module and makes decisions. The cockpit interface receives instructions from the cockpit domain controller, displays the page of the intelligent soothing function, and interacts with the cockpit user. The visual recognition module can be a camera for collecting occupant images, such as a camera installed in the rear of the cockpit. The cockpit speaker outputs the audio of the currently playing content of the cockpit interface. Further details are omitted here.

[0024] In practice, cameras installed in the rear of the vehicle detect occupants in the cabin. Since children's facial features differ significantly from adults, such as facial proportions, distribution of facial features, and facial contours, deep learning models, such as convolutional neural networks (CNNs), are used to analyze facial images and identify child occupants in the vehicle cabin. Emotional recognition is then performed on the facial images to determine the child occupant's initial emotional state. The facial expressions of the child occupant are continuously monitored to promptly capture emotional changes. The initial emotional state is the child occupant's emotional state monitored in real time, which includes a normal emotional state and a state requiring reassurance. A normal emotional state is characterized by a calm expression with no obvious emotional fluctuations, while a state requiring reassurance is characterized by crying, anxiety, and unease, such as drooping eyebrows, tearing eyes, and an open mouth.

[0025] Step 102: In response to the detection that the child occupant is in a state of needing to be comforted, obtain the child occupant's identity information.

[0026] In this embodiment of the application, in response to detecting that a child occupant is in a state of needing to be comforted, the cockpit domain controller sends monitoring information to the cockpit interface, prompts the driver or other passengers that a child occupant is in a state of needing to be comforted through the cockpit interface, and interacts with the user to confirm the identity of the child occupant currently needing to be comforted. Upon receiving the identity of the child occupant confirmed by the user through the cockpit interface, the controller obtains the locally stored identity information of the child occupant.

[0027] It should be noted that the cockpit interface stores pre-configured child occupant identity information, specifically the names, photos, and dates of birth of the children in the family that the user has pre-set in the cockpit interface module. The child occupant identity information can be adjusted according to actual needs or changes. For example, taking child occupant A and child occupant B as an example, the user has pre-configured in the cockpit interface that there are two children in the family, named child occupant A and child occupant B, as well as their photos and dates of birth.

[0028] Step 103: Based on the identity information, determine the playlist and playback tone that match the child occupant, and control the vehicle cabin to play the soothing content of the playlist using the playback tone.

[0029] In this embodiment, after determining the identity information of the child occupant to be comforted, a suitable playlist is matched from a pre-created playlist library based on the identity information. A suitable playback tone can be selected from a preset tone library based on the child occupant's preference information. Based on the playlist, the vehicle cabin is controlled to play comforting content using the selected tone. The comforting content in the playlist, such as music or picture books, is converted to a playback tone that matches the child occupant's preferences to enhance the comforting effect.

[0030] In practice, the playlists include general playlists or personalized playlists for child occupants. General playlists include playlists of types such as music and picture books, and are suitable for all children. Personalized playlists include personalized playlists that child A likes and personalized playlists that child B likes. Personalized playlists are obtained based on children's preference tags. Based on the identity information of the child occupant who needs to be comforted, the personalized playlist of that child occupant is matched first from the pre-created playlist library. If the personalized playlist does not exist or is empty, the general playlist is matched.

[0031] In this embodiment, the playback tone can be automatically selected from a preset tone library to suit the child occupant, or it can be a playback tone confirmed by the cockpit interface and the cockpit user. For example, if the cockpit domain controller recognizes that the user requests to play a tone name that the child occupant likes stored in the cockpit interface, such as the tone of an anime or cartoon character, then the audio stream to be fed back from the cockpit interface is transmitted to the tone conversion module in real time. The audio stream transmission protocol between the cockpit audio stream transmission and the tone conversion module can use mainstream audio stream transmission protocols such as DLNA, RTSP, TCP, etc., so that the tone conversion module can receive and convert the audio stream into the specified playback tone in real time, and perform tone conversion on the current audio and video stream of the soothing content in the playlist to obtain the soothing content to be output as the playback tone.

[0032] Step 104: Obtain the second emotional state of the child occupant at a preset sampling interval, and determine the percentage of the child occupant's unsoothed emotional state based on the second emotional state.

[0033] In this embodiment of the application, in order to ensure that the comforting content played is effective in comforting the child occupants, it is necessary to monitor whether the child occupants' emotions awaiting comfort have disappeared, that is, whether the crying and waiting for comforting emotions have been relieved. Specifically, the second emotional state of the child occupants is acquired in real time at a preset sampling interval. Based on the second emotional state, the percentage of the child occupants' emotions awaiting comfort is determined. The second emotional state is the current emotional state of the child occupants that is monitored and sampled in real time. The second emotional state is also divided into a normal emotional state and an emotional state awaiting comforting.

[0034] In this embodiment, the percentage of the disappearance of the emotional state requiring reassurance is the ratio of the number of occurrences of the normal emotional state within the monitoring period to the total number of occurrences of both the normal emotional state and the emotional state requiring reassurance within the monitoring period. Specifically, the percentage of the disappearance of the emotional state requiring reassurance for the child occupant is determined based on the number of occurrences of the normal emotional state and the emotional state requiring reassurance within the monitoring period. The monitoring period is set according to actual needs, and this embodiment does not limit it. For example, the current emotional state of the child occupant is collected once every 30 seconds at a preset sampling interval, and the percentage of the disappearance of the emotional state requiring reassurance for the child occupant is calculated every 3 minutes as the monitoring period, up to the current moment, in order to determine whether the reassurance content is effective for the child occupant.

[0035] Step 105: Based on the percentage of the disappearance of the emotional state that needs to be soothed, adjust the soothing content played until the child passenger's emotional state returns to a normal emotional state.

[0036] In this embodiment, the soothing content is adjusted based on the percentage of the disappearance of the child's emotional state, until the child's emotional state is detected to have returned to a normal emotional state, at which point the intelligent soothing function is stopped. Specifically, based on the percentage of the child's emotional state disappearing during the playback of soothing content within the monitoring period, it is determined whether the soothing content is effective for the child. If the soothing is ineffective, the soothing content is switched or adjusted according to the current playlist, such as switching the soothing content or changing the playback tone of the soothing content, until the child's emotional state returns to a normal emotional state.

[0037] The vehicle cabin control method provided in this application identifies child occupants in the vehicle cabin and monitors their first emotional state. This emotional state includes a normal emotional state and a state requiring reassurance. The method monitors the child occupant's emotional state in real time, captures emotional changes, triggers reassurance measures, and, in response to detecting a child occupant in a state requiring reassurance, obtains the child occupant's identity information. Based on this identity information, it determines a playlist and playback tone that match the child occupant, and controls the vehicle cabin to play the reassurance content from the playlist using the appropriate playback tone. This method can adapt playlists to each child occupant based on their identity and control the playback tone accordingly. The system selects and plays content and sounds that child occupants prefer, providing personalized comforting content for those needing comfort. It acquires the child occupants' second emotional state at preset sampling intervals, determines the percentage of time their uncomforted emotional state disappears based on this second emotional state, and adjusts the comforting content accordingly until the child occupants' emotional state returns to normal. The system monitors and accurately quantifies the comforting and alleviating effects of the content on the child occupants' uncomforted emotional state, allowing for timely adjustments to the content based on the comforting effect. This ensures flexible and effective comforting of child occupants' emotional states, further enhancing the driving experience and vehicle safety.

[0038] In some embodiments of this application, step 101, which identifies child occupants in the vehicle cabin and monitors their initial emotional state, may specifically include the following steps: S11, Detect the occupants in the vehicle cabin and identify child occupants from among the occupants; S12, monitor the facial image of the child occupant, perform emotion recognition on the facial image of the child occupant, and obtain the first emotional state of the child occupant.

[0039] In this embodiment, the occupants of the vehicle cabin can be detected using object detection algorithms such as YOLO, the occupants' positions can be located and bounding boxes of the occupants can be generated. After detecting the occupants of the vehicle cabin, it can be determined whether the occupants in the bounding boxes are children, and child occupants can be identified from the occupants. In some embodiments, it can be determined whether the occupants are children by analyzing their facial features such as facial proportions and facial contours. It can also be combined with the occupants' posture information, such as body shape, to assist in determining whether they are children, thereby identifying child occupants from the occupants.

[0040] In this embodiment, to obtain the emotional state of child occupants in real time and activate the intelligent soothing function when the child occupants need comforting, the emotional state can be identified by analyzing the child occupants' facial images to determine whether they are in a normal emotional state or a crying and waiting-to-be-soothed emotional state. Specifically, the child occupants' facial images are monitored, and emotions are identified through the facial images to obtain the child occupants' first emotional state. The child occupants' facial expressions are continuously monitored to ensure that emotional changes can be captured in a timely manner. The first emotional state is the child occupants' emotional state monitored in real time, and the emotional state includes a normal emotional state and a waiting-to-be-soothed emotional state.

[0041] It should be noted that emotion recognition of child occupants through facial images can be achieved by detecting facial landmarks and extracting key feature points of the child's face, such as eyebrows, eyes, and mouth. Based on the extracted facial landmark features, an emotion classification model, such as a CNN or Transformer-based model, is used to identify the child occupant's emotions. Common emotional states include normal emotional states and emotional states requiring reassurance. A normal emotional state is characterized by a calm expression without obvious emotional fluctuations, while an emotional state requiring reassurance is characterized by crying, anxiety, and unease, such as drooping eyebrows, tearing eyes, and an open mouth.

[0042] This application accurately identifies child occupants in the vehicle cabin and monitors their emotional state in real time to determine whether they are in a normal emotional state or in a state that needs to be soothed, so as to promptly capture emotional changes and trigger soothing measures.

[0043] In some embodiments of this application, step 103, based on the identity information, determines a playlist and playback tone matching the child occupant, and controls the vehicle cabin to play the soothing content of the playlist using the playback tone. Specifically, this may include the following steps: S21, based on the identity information, match the playlist to be played to the child occupant from a pre-created playlist library; wherein, the playlist includes a general playlist or the child occupant's personalized playlist; S22, Obtain at least one playback tone that matches the child occupant's preference; S23, convert the current audio and video stream of the soothing content in the playlist to obtain the soothing content to be output as the playback tone; S24, based on the playlist, control the vehicle cabin to play soothing content with the specified sound tone.

[0044] In this embodiment, after determining the identity information of the child passenger to be comforted, a suitable playlist is matched from a pre-created playlist library based on the identity information. The playlists include general playlists or personalized playlists for the child passenger. General playlists include music and picture book playlists, applicable to all children. Personalized playlists include playlists liked by child A and child B, obtained based on child preference tags. The playlist library is stored in a database format, including playlist name, playlist type, and playlist content. If a personalized playlist for the child passenger to be comforted exists in the pre-created playlist library, that personalized playlist is preferentially selected for playback. If a personalized playlist is empty or does not exist, a general playlist is selected for playback.

[0045] In this embodiment, after determining the playlist to be played to the child occupant, a suitable playback timbre can be selected from a preset timbre library based on the child occupant's preference information. The timbre library contains a variety of timbres. Specifically, the current audio and video stream of the soothing content in the playlist is timbre-converted to obtain the soothing content to be output as the playback timbre. The timbre conversion can use an end-to-end system based on acoustic models and vocoders (Variational Inference with adversarial learning for end-to-end Text-to-Speech, VITS). VITS is an advanced end-to-end deep learning model used to directly convert text into realistic and natural human speech. Alternatively, it can generate and output cloned timbres by inputting the phonemes of the target timbre. Or, it can use the speech cloning and few-shot learning technique Index-TTS (Index-based Prompt TTS). Index-TTS is a speech synthesis model based on index prompts that extracts the embedding vector of the target timbre from a small number of reference audios to generate speech with a specific timbre. Other timbre conversion schemes can also be selected, but this embodiment does not specifically limit them.

[0046] In this embodiment, based on the playlist, the vehicle cabin is controlled to play soothing content with a specific timbre. According to the matched playlist and the converted timbre, the soothing content in the vehicle cabin is controlled to play soothing content. The soothing content in the playlist, such as music and picture books, is converted into a timbre that matches the child passenger's preferences to enhance the soothing effect.

[0047] This application embodiment obtains the playback content and playback timbre preferred by child passengers by matching playlists and performing high-quality timbre customization, providing personalized comforting content for child passengers who need to be comforted, improving the comforting effect, and better meeting the comforting needs of children's emotional state.

[0048] As a specific implementation of this application embodiment, step S21, based on the identity information, matches a playlist to be played to the child passenger from a pre-created playlist library, which may specifically include: Sub-step 01: Based on the identity information, query whether the child passenger's personalized playlist exists in the pre-created playlist library; Sub-step 02: If yes, then the child passenger's personalized playlist is determined as the playlist to be played to the child passenger; Sub-step 03: If not, then based on the identity information, select comforting content that matches the child passenger's age group from the general playlist, generate a playable list, and determine the playable list as the playlist to be played to the child passenger.

[0049] In this embodiment of the application, based on the identity information, the system queries a pre-created playlist library to determine if there is a personalized playlist for the child passenger who needs to be comforted. For example, based on the identity information of child passenger A, the system searches the playlist library to determine if there is a personalized playlist related to child passenger A, such as "Child A's Favorite Playlists". If so, the personalized playlist of the child passenger is determined as the playlist to be played to the child passenger. If the query result shows that there is a personalized playlist of child passenger A and that the personalized playlist of child passenger A includes comforting content, then the personalized playlist is directly used as the playlist to be played.

[0050] In this embodiment, when the intelligent soothing function of the vehicle cabin is activated, it identifies the current emotional state that needs to be soothed. For example, if it is determined that the child occupant who is crying is child occupant A, if there is soothing content in child A's favorite personalized playlist, then the personalized playlist that child A likes will be played automatically. Otherwise, based on the identity information, soothing content that matches the child occupant's age group is selected from the general playlist to generate a playable list, and the playable list is determined as the playlist to be played to the child occupant.

[0051] In this embodiment, to ensure that comforting content suitable for children can still be provided even if a personalized playlist does not exist, since there is no personalized playlist for child occupants in the pre-created playlist library, when a general playlist needs to be played, comforting content matching the child occupant's age group is selected from the general playlist based on the occupant's identity information. The cockpit interface pre-stores general playlists that children of each age group may like. It determines which age group child occupant A belongs to, such as 0-0.5 years, 0.5-1 years, 1-2 years, 2-3 years, or 3-6 years. A playable playlist is generated from the comforting content matching the child occupant's age group selected from the general playlist, and this playable playlist is determined as the playlist to be played to the child occupant.

[0052] Based on the child occupant's identity information, this application embodiment quickly queries and filters playlists, prioritizes personalized playlists that the child occupant likes, and combines them with age group matching of general playlists to provide the child occupant with accurate and flexible comforting content, meeting the needs of comforting the child's emotional state.

[0053] In some embodiments of this application, step 104, obtaining the second emotional state of the child occupant at a preset sampling interval, and determining the percentage of the child occupant's unsoothed emotional state based on the second emotional state, may specifically include the following steps: S31, Obtain the second emotional state of the child occupant at a preset sampling interval; S32, based on a preset monitoring period, classify and statistically analyze the second emotional state to obtain the number of occurrences of normal emotional state and emotional state requiring reassurance respectively; S33, Based on the number of times normal emotional states and emotional states requiring reassurance occurred within the monitoring period, determine the percentage of the disappearance of the emotional states requiring reassurance for the child passenger.

[0054] In this embodiment of the application, in order to ensure that the comforting content played is effective in comforting the child occupants, the current emotional state of the child occupants is monitored in real time to determine whether the emotional state to be comforted has been effectively alleviated. Specifically, the current emotional state can be collected once at a preset sampling interval to obtain the second emotional state of the child occupants at the preset sampling interval. The preset sampling interval can be set according to the actual situation, for example, once every 30 seconds. The second emotional state is the current emotional state of the child occupants that is monitored and sampled in real time. The second emotional state is also divided into a normal emotional state and an emotional state to be comforted.

[0055] In this embodiment, based on a preset monitoring period, the second emotional state is classified and statistically analyzed to obtain the number of occurrences of the normal emotional state and the emotional state awaiting solace. Based on the number of occurrences of the normal emotional state and the emotional state awaiting solace within the monitoring period, the percentage of the child passenger's emotional state awaiting solace disappears is determined. In specific implementation, the preset monitoring period is set according to actual needs. For example, every 3 minutes is used as a monitoring period. The current emotional state of the child passenger is collected every 30 seconds within 3 minutes to obtain the number of occurrences of the normal emotional state and the emotional state awaiting solace. If the current emotional state is collected to switch back to the normal emotional state, it indicates that the child passenger's emotional state awaiting solace has disappeared, that is, the crying and waiting for solace emotional state has been relieved.

[0056] In this embodiment, the percentage of child occupants whose pending emotional states have disappeared is determined based on the number of occurrences of normal emotional states and pending emotional states within the monitoring period. The percentage of pending emotional states that have disappeared is the ratio of the number of occurrences of normal emotional states to the total number of occurrences of both normal and pending emotional states within the monitoring period. For example, the current emotional state is collected every 30 seconds, and the percentage of pending emotional states that have disappeared up to the current moment is calculated every 3 minutes. For example, if the number of occurrences of normal emotional states within the monitoring period is 5, and the total number of occurrences of both normal and pending emotional states within the monitoring period is 6, then the percentage of pending emotional states that have disappeared is approximately 83%.

[0057] This application embodiment samples and statistically analyzes the percentage of child occupants' unsoothed emotional state during the playback of comforting content within a monitoring period. This allows for real-time and accurate quantification of the effect of the comforting content in alleviating the unsoothed emotional state of child occupants, enabling timely adjustments based on the comforting effect to ensure effective comforting of child occupants.

[0058] In some embodiments of this application, step 105 adjusts the played comforting content according to the percentage of the disappearance of the emotional state to be comforted until the child passenger's emotional state returns to a normal emotional state. Specifically, this may include the following steps: S41, based on the percentage of the disappearance of the emotional state to be comforted, determine whether the comforting content is effective for the child passenger, and obtain the effectiveness result of the comforting content; wherein, the effectiveness result includes ineffective comforting and effective comforting; S42, based on the effectiveness results, adjust the soothing content being played until the child passenger's emotional state returns to the normal emotional state.

[0059] In this embodiment, the effectiveness of the comforting content for the child occupants is determined by the percentage of the disappearance of the child occupants' unsoothed emotional state during the playback of comforting content within the monitoring period. The effectiveness result includes ineffective and effective comforting. Ineffective comforting means the child occupants' unsoothed emotional state is not alleviated, while effective comforting means the child occupants' unsoothed emotional state is alleviated. Specifically, a preset threshold for the percentage of disappearance is used, and the percentage of the unsoothed emotional state disappearing is compared with the preset threshold to determine whether the comforting content is effective for the child occupants. For example, the threshold includes an effective threshold and an ineffective threshold. The effective threshold can be set to 80%, and the ineffective threshold can be set to 50%. If the percentage of the child occupants' unsoothed emotional state disappearing within the monitoring period is greater than or equal to the effective threshold of 80%, it indicates that the child occupants' unsoothed emotional state has been alleviated. If the percentage of the child occupants' unsoothed emotional state disappearing within the monitoring period is less than or equal to the ineffective threshold of 50%, it indicates that the child occupants' unsoothed emotional state has not been alleviated.

[0060] In this embodiment, based on the effectiveness results, the playing soothing content is adjusted until the child passenger's emotional state returns to a normal emotional state. Specifically, if the effectiveness result within the current monitoring period is invalid soothing, the currently playing soothing content is determined to be invalid, and playback of the soothing content is stopped. Soothing content is then switched according to the currently playing playlist. If the effectiveness result within the current monitoring period is valid soothing, the currently playing soothing content is determined to be valid, and playback of the soothing content continues. At this time, if the currently playing soothing content is not in the child passenger's personalized playlist, the currently playing soothing content is marked as soothing content preferred by the child passenger and added to the child passenger's personalized playlist.

[0061] This application embodiment determines whether the child passenger's emotional state has been effectively alleviated by real-time monitoring and calculating the percentage of the disappearance of the emotional state to be soothed. Based on the effectiveness results, the soothing content played is adjusted until the child passenger's emotional state returns to a normal emotional state, ensuring that the child passenger's emotional state is effectively soothed.

[0062] As a specific implementation of this application embodiment, step S42, adjusting the played soothing content according to the effectiveness result until the child passenger's emotional state returns to the normal emotional state, may specifically include: Sub-step 01: If the playlist played to the child occupant is the child occupant's personalized playlist, and the cumulative number of invalid reassurances is greater than a preset threshold, then the reassurance content is removed from the child occupant's personalized playlist. Sub-step 02: If the playlist played to the child passenger is the playable playlist, and the validity result is effective soothing, then the soothing content is marked to the child passenger's personalized playlist. Sub-step 03: Control the vehicle cabin to play the adjusted playlist of soothing content until the child occupant's emotional state returns to the normal emotional state.

[0063] In this embodiment, based on the effectiveness results, the played soothing content is adjusted until the child passenger's emotional state returns to a normal state. Specifically, when the playlist played to the child passenger is a personalized playlist, if the cumulative number of times the soothing effect is deemed ineffective exceeds a preset threshold, the soothing content is removed from the child passenger's personalized playlist. The cumulative number can be the sum of the number of effective results when the same soothing content is played. For example, if a soothing content is deemed ineffective within a monitoring period, it is counted once. If the cumulative count exceeds a preset threshold, such as five times, it indicates that the soothing content has a poor soothing effect on the child passenger, and therefore the soothing content is removed from the child passenger's personalized playlist. When the playlist played to the child passenger is a playable playlist, if the effectiveness result is deemed effective, the soothing content is marked in the child passenger's personalized playlist, a personalized playlist is created for the child passenger, and the marked soothing content is added to the child passenger's personalized playlist.

[0064] In this embodiment, when the playlist played to the child occupant is the child occupant's personalized playlist, the vehicle cabin is controlled to play the adjusted playlist's soothing content until the child occupant's emotional state returns to a normal emotional state. When the playlist played to the child occupant is a playable playlist, and the marked soothing content is added to the child occupant's personalized playlist, the vehicle cabin is controlled to play the adjusted playlist's soothing content. Based on the real-time monitoring of the percentage of the child occupant's unsoothed emotional state disappearing, the currently played soothing content is adjusted until the child occupant's emotional state returns to a normal emotional state.

[0065] It should be noted that when the playlist played for child occupants is a personalized playlist or a playable playlist, the percentage of the child occupant's unsoothed emotional state that disappears during the playback of soothing content within the monitoring period is determined. For example, if every 10 minutes is considered a monitoring period, the current emotional state of the child occupant is collected every 30 seconds within 10 minutes. Based on the number of occurrences of normal emotional state and unsoothed emotional state within 10 minutes, the percentage of the child occupant's unsoothed emotional state that disappears within 10 minutes is determined. If the percentage of the child occupant's unsoothed emotional state that disappears within 10 minutes is less than or equal to 50%, it indicates that the current playlist has failed to alleviate the child occupant's emotional state. An interactive request can be made to the user in the cabin through the cabin interface to ask if it is necessary to switch to another type of playlist. Based on the user's confirmation, it is determined whether to switch to another type of playlist to soothe the child occupant. This will not be elaborated on in detail here.

[0066] This application embodiment adjusts the playback of comforting content in a timely and flexible manner according to whether the playback content is effective, until the child passenger's emotional state returns to a normal emotional state, ensuring effective comforting of the child passenger's emotional state and providing a timely and effective comforting experience for the child passenger in the vehicle cabin.

[0067] Reference Figure 3 The diagram shows a structural schematic of a vehicle cockpit control device according to an embodiment of this application. The device includes: The status monitoring module 201 is used to identify child occupants in the vehicle cabin and monitor the first emotional state of the child occupants; wherein, the emotional state includes a normal emotional state and an emotional state that needs to be comforted; Information acquisition module 202 is used to acquire the identity information of the child occupant in response to detecting that the child occupant is in the state of needing to be comforted; The first control module 203 is used to determine a playlist and playback tone that match the child passenger's preferences based on the identity information, and control the vehicle cabin to play the soothing content of the playlist using the playback tone; The state determination module 204 is used to acquire the second emotional state of the child occupant at a preset sampling interval, and determine the percentage of the disappearance of the child occupant's emotional state requiring reassurance based on the second emotional state. The second control module 205 is used to adjust the playing comforting content according to the percentage of the disappearance of the emotional state to be comforted until the child passenger's emotional state returns to the normal emotional state.

[0068] Optionally, the status monitoring module 201 includes: The identification submodule is used to detect occupants in the vehicle cabin and identify child occupants from among them; The monitoring submodule is used to monitor the facial image of the child occupant, and to perform emotion recognition on the child occupant through the facial image to obtain the child occupant's first emotional state.

[0069] Optionally, the first control module 203 includes: The matching submodule is used to match a playlist to be played to the child occupant from a pre-created playlist library based on the identity information; wherein the playlist includes a general playlist or a personalized playlist of the child occupant. The first acquisition submodule is used to acquire at least one playback timbre that matches the child occupant's preference; The conversion submodule is used to convert the timbre of the current audio and video stream of the soothing content in the playlist to obtain the soothing content to be output as the playback timbre; The first control submodule is used to control the vehicle cabin to play soothing content with the specified timbre based on the playlist.

[0070] Optionally, the matching submodule includes: The query unit is used to query whether the child passenger's personalized playlist exists in the pre-created playlist library based on the identity information; The first determining unit is used to determine the child passenger's personalized playlist as the playlist to be played to the child passenger if the condition is met. The second determining unit is used to, if not, filter comforting content matching the child occupant's age group from the general playlist based on the identity information, generate a playable list, and determine the playable list as the playlist to be played to the child occupant.

[0071] Optionally, the state determination module 204 includes: The second acquisition submodule is used to acquire the second emotional state of the child occupant at a preset sampling interval; The statistics submodule is used to classify and statistically analyze the second emotional state based on a preset monitoring period, and obtain the number of occurrences of the normal emotional state and the emotional state that needs to be soothed. The determination submodule is used to determine the percentage of the disappearance of the child passenger's emotional state requiring reassurance based on the number of times the normal emotional state and the emotional state requiring reassurance occurred within the monitoring period.

[0072] Optionally, the second control module 205 includes: The judgment submodule is used to determine whether the comforting content is effective for the child passenger based on the percentage of disappearance of the emotional state to be comforted, and to obtain the effectiveness result of the comforting content; wherein, the effectiveness result includes ineffective comforting and effective comforting; The second control submodule is used to adjust the soothing content played according to the effectiveness results until the child passenger's emotional state returns to the normal emotional state.

[0073] Optionally, the second control submodule includes: The first processing unit is configured to remove the comforting content from the child passenger's personalized playlist if the cumulative number of invalid comforting responses exceeds a preset threshold when the playlist being played to the child passenger is the child passenger's personalized playlist. The second processing unit is configured to, when the playlist played to the child occupant is the playable playlist, mark the soothing content to the child occupant's personalized playlist if the validity result is effective soothing. The control unit is used to control the playback of soothing content from the adjusted playlist in the vehicle cabin until the child occupant's emotional state returns to its normal emotional state.

[0074] The vehicle cockpit control device provided in this application embodiment can realize the various processes of the vehicle cockpit control method in the above embodiments of this application. To avoid repetition, it will not be described again here.

[0075] The vehicle cabin control device provided in this application identifies child occupants in the vehicle cabin and monitors their initial emotional state. This emotional state includes a normal emotional state and a state requiring reassurance. The device monitors the child occupant's emotional state in real time, detects emotional changes, and triggers reassurance measures. In response to detecting a child occupant in a state requiring reassurance, the device obtains the child occupant's identity information. Based on this information, it determines a playlist and playback tone that matches the child occupant, and controls the vehicle cabin to play the reassurance content from the playlist using the appropriate playback tone. This allows for matching playlists to each child occupant based on their identity and controlling the playback tone. The system obtains the preferred playback content and tone of child occupants, providing personalized comforting content for those needing comfort. It acquires the second emotional state of the child occupants at preset sampling intervals, determines the percentage of time the child occupants' uncomfortable emotional state disappears based on this second emotional state, and adjusts the playback content according to this percentage until the child occupants' emotional state returns to normal. The system monitors and accurately quantifies the comforting and alleviating effects of the content on the child occupants' uncomfortable emotional state, and adjusts the content promptly based on the comforting effect to ensure flexible and effective comforting of child occupants' emotional states, further improving the driving experience and vehicle safety.

[0076] Reference Figure 4 This application also provides an electronic device, such as... Figure 4 As shown, it includes a processor 301, a communication interface 302, a memory 303, and a communication bus 304, wherein the processor 301, the communication interface 302, and the memory 303 communicate with each other through the communication bus 304. Processor 301, memory 303 for storing processor-executable instructions; The processor 301 is configured to execute the instructions to implement the vehicle cockpit control method described below: Identify child occupants in the vehicle cabin and monitor the child occupants' primary emotional state; wherein, the emotional state includes a normal emotional state and an emotional state requiring reassurance; In response to detecting that the child occupant is in a state of needing to be comforted, the identity information of the child occupant is obtained; Based on the identity information, a playlist and playback tone matching the child occupant are determined, and the vehicle cabin is controlled to play the soothing content of the playlist using the playback tone; The second emotional state of the child occupant is obtained at a preset sampling interval, and the percentage of the disappearance of the child occupant's emotional state requiring reassurance is determined based on the second emotional state. Based on the percentage of the disappearance of the emotional state requiring reassurance, adjust the reassurance content played until the child passenger's emotional state returns to the normal emotional state.

[0077] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0078] The communication interface is used for communication between the aforementioned terminal and other devices.

[0079] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0080] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0081] In another embodiment provided in this application, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements any of the vehicle cockpit control methods described in the above embodiments.

[0082] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0083] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0084] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0085] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A vehicle cockpit control method, characterized in that, The method includes: Identify child occupants in the vehicle cabin and monitor the child occupants' primary emotional state; wherein, the emotional state includes a normal emotional state and an emotional state requiring reassurance; In response to detecting that the child occupant is in a state of needing to be comforted, the identity information of the child occupant is obtained; Based on the identity information, a playlist and playback tone matching the child occupant are determined, and the vehicle cabin is controlled to play the soothing content of the playlist using the playback tone; The second emotional state of the child occupant is obtained at a preset sampling interval, and the percentage of the disappearance of the child occupant's emotional state requiring reassurance is determined based on the second emotional state. Based on the percentage of the disappearance of the emotional state requiring reassurance, adjust the reassurance content played until the child passenger's emotional state returns to the normal emotional state.

2. The method according to claim 1, characterized in that, The identification of child occupants in the vehicle cabin and the monitoring of the child occupants' initial emotional state include: The vehicle cabin occupants are detected, and child occupants are identified from among them; The facial image of the child occupant is monitored, and emotion recognition is performed on the facial image to obtain the child occupant's first emotional state.

3. The method according to claim 1, characterized in that, The step of determining a playlist and playback tone matching the child occupant based on the identity information, and controlling the vehicle cabin to play the soothing content of the playlist using the playback tone, includes: Based on the identity information, a playlist is matched from a pre-created playlist library to be played to the child occupant; wherein, the playlist includes a general playlist or a personalized playlist of the child occupant; Obtain at least one playback tone that matches the child occupant's preference; The current audio and video stream of the soothing content in the playlist is converted to a different timbre to obtain the soothing content to be output as the playback timbre; Based on the playlist, control the vehicle cabin to play soothing content with the specified sound tone.

4. The method according to claim 3, characterized in that, The step of matching a playlist for the child passenger from a pre-created playlist library based on the identity information includes: Based on the identity information, query whether the child passenger's personalized playlist exists in the pre-created playlist library; If so, the child passenger's personalized playlist is determined as the playlist to be played to the child passenger; If not, then based on the identity information, select comforting content that matches the child passenger's age group from the general playlist, generate a playable list, and determine the playable list as the playlist to be played to the child passenger.

5. The method according to claim 1, characterized in that, The step of acquiring the second emotional state of the child occupant at a preset sampling interval, and determining the percentage of the disappearance of the child occupant's emotional state requiring reassurance based on the second emotional state, includes: The second emotional state of the child occupant is obtained at a preset sampling interval; Based on a preset monitoring period, the second emotional state is classified and statistically analyzed to obtain the number of occurrences of normal emotional state and emotional state requiring reassurance. The percentage of the absence of the child passenger's emotional state requiring reassurance is determined based on the number of occurrences of normal emotional states and emotional states requiring reassurance within the monitoring period.

6. The method according to claim 1, characterized in that, The step of adjusting the playing comforting content based on the percentage of disappearance of the desired emotional state until the child passenger's emotional state returns to the normal emotional state includes: Based on the percentage of the disappearance of the emotional state requiring reassurance, it is determined whether the reassurance content is effective for the child passenger, thus obtaining the effectiveness result of the reassurance content; wherein, the effectiveness result includes ineffective reassurance and effective reassurance; Based on the effectiveness results, adjust the soothing content played until the child passenger's emotional state returns to the normal emotional state.

7. The method according to claim 6, characterized in that, The step of adjusting the soothing content played based on the effectiveness results until the child passenger's emotional state returns to the normal emotional state includes: If the playlist played to the child occupant is the child occupant's personalized playlist, and the cumulative number of invalid reassurances exceeds a preset threshold, then the reassurance content will be removed from the child occupant's personalized playlist. If the playlist played to the child occupant is the playable playlist, and the validity result is effective soothing, then the soothing content is marked in the child occupant's personalized playlist. Control the playback of soothing content from the adjusted playlist in the vehicle cabin until the child occupant's emotional state returns to its normal emotional state.

8. A vehicle cockpit control device, characterized in that, The device includes: A status monitoring module is used to identify child occupants in the vehicle cabin and monitor the child occupants' first emotional state; wherein, the emotional state includes a normal emotional state and an emotional state that needs to be comforted; The information acquisition module is used to acquire the child occupant's identity information in response to detecting that the child occupant is in the state of needing to be comforted. The first control module is used to determine a playlist and playback tone that match the child passenger's preferences based on the identity information, and control the vehicle cabin to play the soothing content of the playlist using the playback tone; The status determination module is used to acquire the second emotional state of the child occupant at a preset sampling interval, and determine the percentage of the disappearance of the child occupant's emotional state requiring reassurance based on the second emotional state. The second control module is used to adjust the playing comforting content according to the percentage of the disappearance of the emotional state to be comforted until the child passenger's emotional state returns to the normal emotional state.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the instructions to implement the vehicle cockpit control method as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that, A computer program is stored on the readable storage medium, which, when executed by a processor, implements the vehicle cockpit control method as described in any one of claims 1 to 7.

Images