Psychological consultation service method and device based on intelligent cockpit and vehicle

Abstract

The invention relates to a psychological consultation service method and device based on an intelligent cockpit and a vehicle, which can deeply integrate environment adjustment, intelligent perception and professional psychological support, can intelligently switch service modes in different scenes, and provides active and personalized emotion support for users on the premise of ensuring safety and privacy. The method comprises the following steps: in response to a psychological counseling service request initiated by a user, providing a deep psychological counseling service for the user; evaluating a current emotional state of the user based on the user physiological data, the user face data and the dialogue data; if the user is in the high emotion fluctuation state, generating an environment adjusting instruction; and sending the environment adjustment instruction to a multi-mode environment adjustment subsystem of the vehicle to drive at least one execution unit in the multi-mode environment adjustment subsystem to execute a first type of action, and adjusting the environment in the vehicle to a preset healing state matched with the high emotion fluctuation state to assist the psychological counseling service.

Description

Technical Field

[0001] This application relates to the field of smart cockpits, specifically to a method, device, and vehicle for providing psychological counseling services based on a smart cockpit. Background Technology

[0002] With the rapid development of intelligent connected vehicle technology, the modern vehicle cabin has gradually transformed from a simple driving space into a third living space that integrates work, entertainment, and life. Currently, some high-end models have integrated environmental adjustment functions such as seat massage, multi-color ambient lighting, and fragrance generators into their cabin systems. However, these functions are mostly limited to manual selection by the user or static switching based on simple scenarios (such as rest mode and sport mode). The functions are independent of each other and lack synergy and intelligence.

[0003] At the same time, the psychological health needs of drivers and passengers are becoming increasingly prominent. Long-distance driving and traffic congestion can easily trigger anxiety and irritability, but traditional in-vehicle systems have significant shortcomings in this regard: First, they lack the ability to perceive and understand users' psychological states in real time and cannot identify emotional fluctuations; second, even if negative emotions are identified, existing systems lack professional intervention methods and can only provide basic entertainment content (such as playing music), failing to provide truly effective psychological counseling; third, and more importantly, existing solutions fail to fully consider the rigid constraints of driving safety and user privacy. For example, complex interactions during driving may distract the driver, and processing psychological data when multiple passengers are in the vehicle poses a risk of privacy leaks. Summary of the Invention

[0004] This application provides a method, device, and vehicle for providing psychological counseling services based on a smart cockpit. It can deeply integrate environmental adjustment, intelligent perception, and professional psychological support, and can intelligently switch service modes in different scenarios (such as parking / driving, single / multiple people). Under the premise of ensuring safety and privacy, it provides users with a proactive and personalized emotional support in-vehicle system solution.

[0005] The technical solution of this invention is as follows: This application provides a method for providing psychological counseling services based on a smart cockpit, including: When the system detects that the vehicle is parked and there is only one person inside, it responds to the user's request for psychological counseling services and provides the user with in-depth psychological counseling services. Based on user physiological data, user facial data, and dialogue data obtained from in-vehicle sensors, the user's current emotional state is assessed. If the assessment results show that the user is in a state of high emotional fluctuation, then an environmental adjustment instruction will be generated; The environmental adjustment command is sent to the vehicle's multimodal environmental adjustment subsystem to drive at least one execution unit in the multimodal environmental adjustment subsystem to perform a first type of action, adjusting the in-vehicle environment to a preset healing state that matches the high emotional fluctuation state, in order to assist in the conduct of psychological counseling services.

[0006] Preferably, the multimodal environment control subsystem includes at least one of a seat adjustment module, a lighting control system, a fragrance system, and an acoustic system; The first type of action performed by the execution unit includes: starting a preset seat massage program, switching the ambient light to a preset calming color scheme, releasing a soothing fragrance, and playing at least one of white noise or soothing music.

[0007] Preferably, the method further includes: When the system detects that the vehicle is parked and there are multiple people inside, it responds to the user's request for psychological counseling services and provides the user with basic psychological information.

[0008] Preferably, the method further includes: The steps for assessing a user's current emotional state based on user physiological data, facial data, and conversation data acquired from in-vehicle sensors include: Physiological signals are filtered and denoised to extract time-domain and frequency-domain features of heart rate variability as the first type of features; key point detection is performed on facial image data to extract the intensity of specific action units in the facial action coding system as the second type of features; speech audio data is framed and windowed to extract Mel-frequency cepstral coefficients, fundamental frequency, and their standard deviation as the third type of features; speech audio data is automatically transcribed into text using speech recognition, and based on an emotion dictionary and a pre-trained word vector model, the frequency of occurrence of emotion keywords, semantic similarity of word vectors, and syntactic complexity in the text are extracted as the fourth type of features. The first to fourth types of features are input into a multimodal fusion neural network model based on an attention mechanism. The multimodal fusion neural network model first generates a modality-specific preliminary emotional state vector for each type of feature, then calculates the weight of each modality in the final decision through an attention network, and finally fuses all preliminary vectors in a weighted summation manner to generate the final emotional state evaluation result. The evaluation result includes discrete emotional classification labels and continuous emotional arousal values. The emotional arousal value is compared with a preset arousal threshold; and the emotional classification label is combined with whether it belongs to a preset negative emotion category. If the emotional arousal value exceeds the arousal threshold and the emotional category label is a preset negative emotion category, then the user is determined to be in a state of high emotional fluctuation.

[0009] Preferably, the method further includes: When the vehicle is detected to be in motion, the system acquires the user's physiological data, facial data, dialogue data, and vehicle driving data through in-vehicle sensors. The driving risk level is determined based on the user's physiological data, facial data, dialogue data, and vehicle driving data; When the driving risk level is determined to be high-risk, a driving safety intervention command is generated. The driving safety intervention command is sent to the vehicle's multimodal environment adjustment subsystem to drive at least one execution unit in the multimodal environment adjustment subsystem to perform a second type of action, adjusting the in-vehicle environment to a preset safety state that helps the driver regain calm and focus.

[0010] Preferably, the method further includes: When the driving risk level is determined to be high-risk, the emergency intervention mechanism is triggered. The emergency contact mechanism includes: Play a reassuring voice message and ask the user if they want to contact their emergency contact. If no response or confirmation is received from the user within a preset time, the system will automatically dial the emergency contact and upload the vehicle's location information to the cloud-based rescue platform.

[0011] Preferably, the multimodal environment control subsystem includes at least one of a seat adjustment module, a lighting control system, a fragrance system, and an acoustic system; The second type of action performed by the execution unit includes at least one of the following actions: Activate the seat's short vibration or slight vibration function to provide a tactile warning; Switch the ambient lighting to the preset warning color scheme; Play steady, monotonous white noise or rhythmic sound effects that guide breathing; Release an invigorating fragrance, or stop releasing a soothing and sleep-inducing fragrance.

[0012] Preferably, the step of determining the driving risk level based on the user's physiological data, facial data, dialogue data, and vehicle driving data includes: The low-frequency to high-frequency power ratio of heart rate growth rate and heart rate variability was extracted from physiological data as a first-type risk feature. The cumulative duration of the gaze deviating from the road ahead per unit time and the average number of blinks per minute were extracted from facial data as the second type of risk characteristics. The variance of the fundamental frequency is extracted from the speech of the dialogue data, and the frequency of occurrence of preset high-risk keywords is extracted from the transcribed text as a third type of risk feature. The trigger frequency of lane departure warning signals and the percentage of time when the distance to the vehicle in front is less than the safe threshold are extracted from vehicle driving data as the fourth type of risk characteristics. Input the first to fourth categories of risk characteristics into the driving risk fusion decision model; The driving risk fusion decision model assigns a preset weight to each type of risk feature and calculates a comprehensive risk score. The comprehensive risk score is compared with a preset threshold range to finally output a discrete driving risk level.

[0013] This application also provides a psychological counseling service device based on a smart cockpit, comprising: The service module is used to respond to the user's request for psychological counseling services and provide the user with in-depth psychological counseling services when the vehicle is identified as being parked and the scene is a single person inside the vehicle. The assessment module is used to assess the user's current emotional state based on user physiological data, user facial data, and dialogue data obtained from in-vehicle sensors. The instruction generation module is used to generate environmental adjustment instructions if the assessment results show that the user is in a state of high emotional fluctuation. The control module is used to send the environmental adjustment command to the vehicle's multimodal environmental adjustment subsystem to drive at least one execution unit in the multimodal environmental adjustment subsystem to perform a first type of action, adjusting the in-vehicle environment to a preset healing state that matches the high emotional fluctuation state, so as to assist in the conduct of psychological counseling services.

[0014] This application also provides a vehicle that includes the aforementioned smart cockpit-based psychological counseling service device.

[0015] The beneficial effects of this invention are as follows: By constructing a closed-loop system encompassing scene perception, emotion assessment, intelligent decision-making, and multimodal execution, the system achieves, for the first time, safe and controllable proactive emotional support within a smart cockpit. Its core breakthrough lies in the intelligent adaptation of services to different scenarios: the system accurately identifies various scenarios such as parking / driving, single / multi-person use, and automatically switches to in-depth consultation, safety intervention, or privacy protection modes, fundamentally eliminating safety and privacy risks. Simultaneously, through proactive coordination between environment and emotion, the system drives the linkage of multi-dimensional environmental execution units, including seats, lighting, fragrance, and acoustics, dynamically adjusting the in-vehicle space into a healing or safe space that matches the user's psychological state, effectively assisting in emotional regulation from a physiological perspective. Finally, the system also constructs a complete protection chain from the vehicle to the cloud, automatically triggering an emergency rescue mechanism in extreme situations, achieving seamless integration from proactive vehicle care to timely human intervention, truly making the vehicle an intelligent travel companion that understands emotions and ensures safety. Attached Figure Description

[0016] Figure 1 This is a flowchart of the method in the embodiments of this application; Figure 2 This is a structural block diagram of the device in the embodiments of this application; Figure 3 This is a structural block diagram of the vehicle in the embodiments of this application. Detailed Implementation

[0017] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings. While the description is quite detailed, it should not be construed as limiting the scope of the present invention. Obvious variations and substitutions of the following examples are all within the scope of protection of this patent.

[0018] Reference Figure 1 This application provides a method for providing psychological counseling services based on a smart cockpit, including: S101, when it recognizes that the vehicle is parked and there is only one person inside, responds to the user's request for psychological counseling services and provides the user with in-depth psychological counseling services.

[0019] S102 assesses the user's current emotional state based on user physiological data, user facial data, and dialogue data obtained from in-vehicle sensors.

[0020] S103, if the assessment results show that the user is in a state of high emotional fluctuation, then generate an environmental adjustment instruction.

[0021] S104, the environmental adjustment command is sent to the vehicle's multimodal environmental adjustment subsystem to drive at least one execution unit in the multimodal environmental adjustment subsystem to perform a first type of action, adjusting the in-vehicle environment to a preset healing state that matches the high emotional fluctuation state, so as to assist in the conduct of psychological counseling services.

[0022] By continuously monitoring vehicle status and occupant scenarios, a psychological counseling service request initiated by a user via voice or touch will only be responded to when both conditions are met: the vehicle is parked and the user is alone in the vehicle. Once the request is responded to, a psychological counseling session is initiated, and necessary user authentication is completed to ensure personalized service and privacy.

[0023] During the psychological counseling service, a sensor array integrated into the cabin simultaneously collects the user's physiological, facial, and conversational data. Data analysis is not conducted in isolation; instead, data fusion technology is employed to align, reduce dimensionality, and extract features from different modalities, before inputting them into a dedicated emotion calculation model. This model comprehensively analyzes the user's physiological responses, facial micro-expressions, and acoustic and semantic features in their speech to calculate an assessment of the user's current emotional state.

[0024] In this embodiment of the application, step S102, which assesses the user's current emotional state based on user physiological data, user facial data, and dialogue data acquired by in-vehicle sensors, includes: Physiological signals are filtered and denoised to extract time-domain and frequency-domain features of heart rate variability (HRV) as the first type of features; facial image data is subjected to keypoint detection to extract the intensity of specific action units in the Facial Action Coding System (FACS) as the second type of features; speech audio data is framed and windowed to extract Mel-frequency cepstral coefficients (MFCC), fundamental frequency (F0), and their standard deviation as the third type of features; speech audio data is transcribed into text using Automatic Speech Recognition (ASR), and based on a sentiment dictionary and a pre-trained word vector model, the frequency of occurrence of sentiment keywords, semantic similarity of word vectors, and syntactic complexity in the text are extracted as the fourth type of features. The first to fourth types of features are input into a multimodal fusion neural network model based on an attention mechanism. The multimodal fusion neural network model first generates a modality-specific preliminary emotional state vector for each type of feature, then calculates the weight of each modality in the final decision through an attention network, and finally fuses all preliminary vectors in a weighted summation manner to generate the final emotional state evaluation result. The evaluation result includes discrete emotional classification labels and continuous emotional arousal values.

[0025] In this embodiment of the application, the step of determining whether a user is in a state of high emotional fluctuation is, for example: The emotional arousal value is compared with a preset arousal threshold; and the emotional classification label is combined with whether it belongs to a preset negative emotion category. If the emotional arousal value exceeds the arousal threshold and the emotional category label is a preset negative emotion category, then the user is determined to be in a state of high emotional fluctuation.

[0026] In this embodiment of the application, the multimodal environment control subsystem includes at least one of a seat adjustment module, a lighting control system, a fragrance system, and an acoustic system; The first type of actions performed by the execution unit includes: activating a preset seat massage program, switching the ambient lighting to a preset calming color scheme, releasing a soothing fragrance, and playing at least one of white noise or soothing music. Through the coordinated adjustment of one or more of the above execution units, the system dynamically creates a supportive space with healing functions in the car from multiple sensory dimensions of touch, sight, smell, and hearing, thereby non-invasively assisting in the smooth conduct of core psychological counseling services.

[0027] In this embodiment of the application, the method further includes: When the S105 detects that the vehicle is parked and there are multiple people inside, it responds to the user's request for psychological counseling services and provides the user with basic psychological science popularization.

[0028] When the system detects that the vehicle is parked but there are multiple occupants, it automatically enters a service degradation and privacy protection mode. In this mode, if a user requests psychological counseling, the in-depth counseling service described in step S101 will not be provided; instead, basic psychological information will be offered. In-depth counseling in a multi-person environment carries the risk of privacy breaches, and the presence of other occupants may interfere with the professionalism and effectiveness of the counseling process. Therefore, the system intelligently switches service modes based on scenario recognition. By providing general informational content rather than personalized counseling dialogue, the system effectively avoids exposing any user's sensitive information, such as their personal psychological state, emotional distress, or counseling history, in a public environment, strictly adhering to privacy protection principles. Furthermore, while protecting privacy, the core value of disseminating mental health knowledge is retained. It can respond to requests by providing content such as stress management techniques, basic knowledge of emotion regulation, or common mental health information, thus providing value to all occupants even in inappropriate scenarios, improving the system's applicability and user experience.

[0029] In this embodiment of the application, the method further includes: S105, when the vehicle is detected to be in a driving state, acquires the user's physiological data, facial data, dialogue data and vehicle driving data through in-vehicle sensors; S106 determines the driving risk level based on the user's physiological data, facial data, dialogue data, and vehicle driving data; S107, when the driving risk level is determined to be high-risk driving level, a driving safety intervention command is generated; S108, the driving safety intervention command is sent to the vehicle's multimodal environment adjustment subsystem to drive at least one execution unit in the multimodal environment adjustment subsystem to perform a second type of action, adjusting the in-vehicle environment to a preset safety state that helps the driver regain calm and focus.

[0030] Upon detecting that the vehicle is in motion, the primary task shifts to ensuring driving safety. Through the in-vehicle sensor network, four-dimensional data is continuously and synchronously collected: physiological data (such as heart rate) to monitor the driver's arousal level, facial data (such as gaze direction) to determine their attention level, dialogue data (such as voice tone and keywords) to analyze their emotional urgency, and vehicle driving data (such as lateral acceleration) to directly reflect vehicle handling stability. This simultaneous collection of multi-source data provides a perceptual foundation for a comprehensive risk assessment.

[0031] The multi-source heterogeneous data obtained in step S105 is used as input and processed through a dedicated driving risk fusion assessment model. This model is used to perform fusion analysis on the aforementioned four-dimensional data. The design principle of this model is to use an algorithm to weightedly and comprehensively assess the inherent correlation and risk coupling between the driver's state (reflected by physiological, facial, and dialogue data) and vehicle behavior (reflected by driving data), ultimately outputting a quantitative and comprehensive driving risk level, thus overcoming the limitations of assessment based on a single data dimension.

[0032] In this embodiment, step S106, which determines the driving risk level based on the user's physiological data, facial data, dialogue data, and vehicle driving data, includes: The low-frequency to high-frequency power ratio (LF / HF) of heart rate growth rate and heart rate variability (HRV) was extracted from physiological data as a type I risk characteristic. The cumulative duration of the gaze deviating from the road ahead per unit time and the average number of blinks per minute were extracted from facial data as the second type of risk characteristics. The variance of the fundamental frequency (F0) is extracted from the speech of the dialogue data, and the frequency of occurrence of preset high-risk keywords is extracted from its transcribed text as a third type of risk feature. The trigger frequency of lane departure warning (LDW) signals and the percentage of time when the distance to the vehicle in front is less than the safe threshold are extracted from vehicle driving data as the fourth type of risk characteristics. Input the first to fourth categories of risk characteristics into the driving risk fusion decision model; The driving risk fusion decision model assigns a preset weight to each type of risk feature and calculates a comprehensive risk score. The comprehensive risk score is compared with a preset threshold range to finally output a discrete driving risk level.

[0033] The generated instructions are sent to the multimodal environment conditioning subsystem, driving its execution unit to perform a second type of action. The second type of action includes at least one of the following actions: Activate the seat's short vibration or slight vibration function to provide a tactile warning; Switch the ambient lighting to the preset warning color scheme; Play steady, monotonous white noise or rhythmic sound effects that guide breathing; Release an invigorating fragrance, or stop releasing a soothing and sleep-inducing fragrance.

[0034] The design of such actions follows the principle of minimal interference and maximum alertness. Their purpose is not to provide comfort, but to quickly redirect the driver's attention back to the driving task through rapid and brief sensory interventions (such as playing a concise warning sound through the acoustic system, increasing the stimulation of cold air through the air conditioning system, or generating a short vibration through the seat), thereby resetting the in-vehicle environment to a preset safe state conducive to focused driving.

[0035] This safety intervention process reflects the intelligent nature of the system's functions, which dynamically switch according to the scenario, ensuring that mental health services are based on the fundamental premise of unconditionally guaranteeing driving safety under any circumstances.

[0036] In this embodiment of the application, the method further includes: S109, when the driving risk level is determined to be high-risk driving level, the emergency intervention mechanism is triggered; The emergency contact mechanism includes: Play a reassuring voice message and ask the user if they want to contact their emergency contact. If no response or confirmation is received from the user within a preset time, the system will automatically dial the emergency contact and upload the vehicle's location information to the cloud-based rescue platform.

[0037] The system plays pre-recorded reassuring messages to calm the user and clearly asks if they need help and to contact emergency contacts. This aims to give users the opportunity to make their own decisions.

[0038] If the system does not detect any valid response from the user within the preset waiting time, or if the user explicitly gives an instruction to confirm the contact, it indicates that the situation may have exceeded the user's control or the user is actively seeking external assistance.

[0039] At this point, the system will automatically execute the final protection procedure: based on the pre-stored emergency contact information, it will automatically dial the emergency contact's phone number to establish a communication link; at the same time, it will upload key information such as the current vehicle's precise location and status indicators to the cloud rescue platform in real time, thereby initiating the external rescue process and providing final protection for the driver's safety.

[0040] This mechanism establishes a complete safety chain from proactive vehicle intervention to final manual rescue, greatly enhancing safety assurance capabilities in extreme situations.

[0041] Reference Figure 2This application also provides a psychological counseling service device based on a smart cockpit, comprising: Service module 101 is used to respond to the user's request for psychological counseling services and provide the user with in-depth psychological counseling services when the vehicle is identified as being parked and the scene inside the vehicle is a single person. The assessment module 102 is used to assess the user's current emotional state based on the user's physiological data, facial data, and dialogue data obtained from in-vehicle sensors. The instruction generation module 103 is used to generate environmental adjustment instructions if the assessment results show that the user is in a state of high emotional fluctuation. The control module 104 is used to send the environmental adjustment command to the vehicle's multimodal environmental adjustment subsystem to drive at least one execution unit in the multimodal environmental adjustment subsystem to perform a first type of action, adjusting the in-vehicle environment to a preset healing state that matches the high emotional fluctuation state, so as to assist in the conduct of psychological counseling services.

[0042] This application also provides a vehicle that includes the aforementioned smart cockpit-based psychological counseling service device.

[0043] Figure 3 This is a block diagram illustrating a vehicle 400 according to an exemplary embodiment. For example, vehicle 400 may be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicle. Vehicle 400 may be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.

[0044] Reference Figure 3 The vehicle 400 may include various subsystems, such as an infotainment system 410, a perception system 420, a decision control system 430, a drive system 440, and a computing platform 450. The vehicle 400 may also include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of the vehicle 400 can be interconnected via wired or wireless means. In some embodiments, the infotainment system 410 may include a communication system, an entertainment system, and a navigation system, etc.

[0045] The perception system 420 may include several sensors for sensing information about the environment surrounding the vehicle 400. For example, the perception system 420 may include a global positioning system (which may be GPS, BeiDou, or other positioning systems), an inertial measurement unit (IMU), lidar, millimeter-wave radar, ultrasonic radar, and a camera device.

[0046] The decision control system 430 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system. The drive system 440 may include components that provide power to the vehicle 400. In one embodiment, the drive system 440 may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of internal combustion engines, electric motors, and compressed air engines. The engine is capable of converting energy provided by the energy source into mechanical energy.

[0047] Some or all of the functions of the vehicle 400 are controlled by a computing platform 450. The computing platform 450 may include at least one processor 451 and a memory 452, the processor 451 being able to execute instructions 353 stored in the memory 452.

[0048] Processor 451 can be any conventional processor, such as a commercially available CPU. The processor may also include, for example, a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), a System on Chip (SOC), an Application Specific Integrated Circuit (ASIC), or a combination thereof.

[0049] The memory 452 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0050] In addition to instruction 353, memory 452 can also store data, such as road maps, route information, vehicle position, direction, speed, and other data. The data stored in memory 452 can be used by computing platform 450.

[0051] In this embodiment of the disclosure, the processor 451 may execute instructions 353 to complete all or part of the steps of the vehicle control method described above.

[0052] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the vehicle control method provided in this disclosure.

[0053] Furthermore, the term “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as advantageous compared to other aspects or designs. Rather, the use of the term “exemplary” is intended to present the concept in a concrete manner. As used herein, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clear from the context, “X applies A or B” is intended to mean any of the natural inclusive arrangements. That is, “X applies A or B” satisfies any of the foregoing instances if X applies A; X applies B; or both X applies A and B. Additionally, unless otherwise specified or clear from the context to refer to the singular form, the articles “a” and “an” as used in this application and the appended claims are generally understood to mean “one or more.”

[0054] Similarly, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding the specification and drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terminology used to describe such components is intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if structurally not equivalent to the disclosed structure. Furthermore, although specific features of this disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations, as may be desired and advantageous to any given or particular application. Moreover, with regard to the terms “comprising,” “owning,” “having,” “having,” or variations thereof as used in the detailed description or claims, such terms are intended to be inclusive in a manner similar to the term “including.”

[0055] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0056] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

[0057] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0058] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0059] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0060] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processing module, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (control method), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic device, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0061] It should be understood that various parts of the embodiments of this disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0062] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0063] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a single processing module, or each unit can exist physically separately, or two or more units can be integrated into a single module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The aforementioned storage medium can be a read-only memory, a hard disk, or an optical disk, etc.

[0064] Although embodiments of the present disclosure have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A method for providing psychological counseling services based on an intelligent cockpit, characterized in that, include: When the system detects that the vehicle is parked and there is only one person inside, it responds to the user's request for psychological counseling services and provides the user with in-depth psychological counseling services. Based on user physiological data, user facial data, and dialogue data obtained from in-vehicle sensors, the user's current emotional state is assessed. If the assessment results show that the user is in a state of high emotional fluctuation, then an environmental adjustment instruction will be generated; The environmental adjustment command is sent to the vehicle's multimodal environmental adjustment subsystem to drive at least one execution unit in the multimodal environmental adjustment subsystem to perform a first type of action, adjusting the in-vehicle environment to a preset healing state that matches the high emotional fluctuation state, in order to assist in the conduct of psychological counseling services.

2. The method according to claim 1, characterized in that, The multimodal environment control subsystem includes at least one of a seat adjustment module, a lighting control system, a fragrance system, and an acoustic system; The first type of action performed by the execution unit includes: starting a preset seat massage program, switching the ambient light to a preset calming color scheme, releasing a soothing fragrance, and playing at least one of white noise or soothing music.

3. The method according to claim 1, characterized in that, The method further includes: When the system detects that the vehicle is parked and there are multiple people inside, it responds to the user's request for psychological counseling services and provides the user with basic psychological information.

4. The method according to claim 1, characterized in that, The method further includes: The steps for assessing a user's current emotional state based on user physiological data, facial data, and conversation data acquired from in-vehicle sensors include: Physiological signals are filtered and denoised to extract time-domain and frequency-domain features of heart rate variability as the first type of features; key point detection is performed on facial image data to extract the intensity of specific action units in the facial action coding system as the second type of features; speech audio data is framed and windowed to extract Mel-frequency cepstral coefficients, fundamental frequency, and their standard deviation as the third type of features; speech audio data is automatically transcribed into text using speech recognition, and based on an emotion dictionary and a pre-trained word vector model, the frequency of occurrence of emotion keywords, semantic similarity of word vectors, and syntactic complexity in the text are extracted as the fourth type of features. The first to fourth types of features are input into a multimodal fusion neural network model based on an attention mechanism. The multimodal fusion neural network model first generates a modality-specific preliminary emotional state vector for each type of feature, then calculates the weight of each modality in the final decision through an attention network, and finally fuses all preliminary vectors in a weighted summation manner to generate the final emotional state evaluation result. The evaluation result includes discrete emotional classification labels and continuous emotional arousal values. The emotional arousal value is compared with a preset arousal threshold; and the emotional classification label is combined with whether it belongs to a preset negative emotion category. If the emotional arousal value exceeds the arousal threshold and the emotional category label is a preset negative emotion category, then the user is determined to be in a state of high emotional fluctuation.

5. The method according to claim 1, characterized in that, The method further includes: When the vehicle is detected to be in motion, the system acquires the user's physiological data, facial data, dialogue data, and vehicle driving data through in-vehicle sensors. The driving risk level is determined based on the user's physiological data, facial data, dialogue data, and vehicle driving data; When the driving risk level is determined to be high-risk, a driving safety intervention command is generated. The driving safety intervention command is sent to the vehicle's multimodal environment adjustment subsystem to drive at least one execution unit in the multimodal environment adjustment subsystem to perform a second type of action, adjusting the in-vehicle environment to a preset safety state that helps the driver regain calm and focus.

6. The method according to claim 5, characterized in that, The method further includes: When the driving risk level is determined to be high-risk, the emergency intervention mechanism is triggered. The emergency contact mechanism includes: Play a reassuring voice message and ask the user if they want to contact their emergency contact. If no response or confirmation is received from the user within a preset time, the system will automatically dial the emergency contact and upload the vehicle's location information to the cloud-based rescue platform.

7. The method according to claim 5, characterized in that, The multimodal environment control subsystem includes at least one of a seat adjustment module, a lighting control system, a fragrance system, and an acoustic system; The second type of action performed by the execution unit includes at least one of the following actions: Activate the seat's short vibration or slight vibration function to provide a tactile warning; Switch the ambient lighting to the preset warning color scheme; Play steady, monotonous white noise or rhythmic sound effects that guide breathing; Release an invigorating fragrance, or stop releasing a soothing and sleep-inducing fragrance.

8. The method according to claim 5, characterized in that, The steps for determining a driving risk level based on a user's physiological data, facial data, conversation data, and vehicle driving data include: The low-frequency to high-frequency power ratio of heart rate growth rate and heart rate variability was extracted from physiological data as a first-type risk feature. The cumulative duration of the gaze deviating from the road ahead per unit time and the average number of blinks per minute were extracted from facial data as the second type of risk characteristics. The variance of the fundamental frequency is extracted from the speech of the dialogue data, and the frequency of occurrence of preset high-risk keywords is extracted from the transcribed text as a third type of risk feature. The trigger frequency of lane departure warning signals and the percentage of time when the distance to the vehicle in front is less than the safe threshold are extracted from vehicle driving data as the fourth type of risk characteristics. Input the first to fourth categories of risk characteristics into the driving risk fusion decision model; The driving risk fusion decision model assigns a preset weight to each type of risk feature and calculates a comprehensive risk score. The comprehensive risk score is compared with a preset threshold range to finally output a discrete driving risk level.

9. A psychological counseling service device based on an intelligent cockpit, characterized in that, include: The service module is used to respond to the user's request for psychological counseling services and provide the user with in-depth psychological counseling services when the vehicle is identified as being parked and the scene is a single person inside the vehicle. The assessment module is used to assess the user's current emotional state based on user physiological data, user facial data, and dialogue data obtained from in-vehicle sensors. The instruction generation module is used to generate environmental adjustment instructions if the assessment results show that the user is in a state of high emotional fluctuation. The control module is used to send the environmental adjustment command to the vehicle's multimodal environmental adjustment subsystem to drive at least one execution unit in the multimodal environmental adjustment subsystem to perform a first type of action, adjusting the in-vehicle environment to a preset healing state that matches the high emotional fluctuation state, so as to assist in the conduct of psychological counseling services.

10. A vehicle, characterized in that, Includes the smart cockpit-based psychological counseling service device as described in claim 9.

Images