Self-adaptive environment adjusting method and device for vehicle-mounted intelligent cabin

Abstract

The invention relates to a self-adaptive environment adjusting method and device for a vehicle-mounted intelligent cockpit, and the method comprises the steps: collecting the multi-modal data of a target user in a current vehicle, and enabling the multi-modal data to comprise voice data, behavior data and physiological feature data; fusing the voice data and the behavior data to generate a corresponding user intention instruction, and inputting the physiological feature data into a pre-constructed passenger comfort level model to output a comfort level score corresponding to the target user; and determining priorities of a plurality of target vehicle-mounted devices in the current vehicle, so as to perform adaptive environment adjustment operation on the plurality of target vehicle-mounted devices based on the user intention instruction and the comfort level score in combination with the priorities and a preset energy distribution strategy. Through the multi-mode interactive fusion intelligent cabin system, the interaction reliability in a complex scene is improved, the in-vehicle environment parameters can be adjusted in real time based on the physiological characteristics of passengers, and energy distribution can be controlled and optimized through equipment cooperation, so that the power consumption of the system is reduced.

Description

Technical Field

[0001] The present application relates to the field of automotive electronic technology, and in particular to a method and device for adaptively adjusting the environment of an on-board intelligent cockpit. Background Art

[0002] Existing in-vehicle intelligent cockpit systems usually integrate voice interaction, touch screen, environmental control and other functions, but still have the following limitations:

[0003] 1. Single interaction mode: It relies on a single mode (such as voice or touch) and is prone to failure in complex scenarios (such as loud noise in the car).

[0004] 2. Fixed environmental adjustment: Parameters such as air conditioning, seats, and lighting need to be set manually and cannot be dynamically adjusted based on passengers’ biological characteristics (such as body temperature and heart rate).

[0005] 3. Inefficient energy management: The independent operation of multiple devices leads to energy waste. For example, there is a lack of coordinated optimization when the screen and air conditioner run at high power at the same time.

[0006] In summary, the interaction methods of existing technologies cannot adapt to complex scenarios, have a low level of intelligence, and the independent operation of multiple devices leads to energy waste and low system energy efficiency, which needs to be solved urgently. Summary of the invention

[0007] The present application provides an adaptive environmental adjustment method and device for an in-vehicle intelligent cockpit to solve the problems that the interaction mode of the prior art cannot adapt to complex scenarios, has a low degree of intelligence, and the independent operation of multiple devices leads to energy waste and low system energy efficiency.

[0008] The first aspect of the present application provides an adaptive environmental adjustment method for an in-vehicle intelligent cockpit, comprising the following steps: collecting multimodal data of a target user in a current vehicle, wherein the multimodal data includes voice data, behavior data, and physiological characteristic data; fusing the voice data and the behavior data to generate corresponding user intention instructions, and inputting the physiological characteristic data into a pre-built passenger comfort model to output a comfort score corresponding to the target user; determining the priorities of multiple target in-vehicle devices in the current vehicle, and performing adaptive environmental adjustment operations on the multiple target in-vehicle devices based on the user intention instructions and the comfort scores, and in combination with the priorities and a preset energy allocation strategy.

[0009] Optionally, in an embodiment of the present application, the fusing the voice data and the behavior data to generate a corresponding user intention instruction includes: collecting environmental noise information and light information of the current vehicle, and quantifying the environmental noise information and the light information to obtain corresponding noise quantization information and light quantization information; dynamically allocating corresponding voice interaction weights and behavior interaction weights for the voice data and the behavior data based on the noise quantization information and the light quantization information, and determining a target interaction mode according to the voice interaction weights and the behavior interaction weights, so as to generate the user intention instruction through the target interaction mode.

[0010] Optionally, in an embodiment of the present application, the inputting the physiological characteristic data into a pre-constructed passenger comfort model to output a comfort score corresponding to the target user includes: performing data preprocessing on the physiological characteristic data to obtain standard physiological data, and extracting time-frequency features and non-linear features corresponding to the heart rate data in the standard physiological data; inputting the time-frequency features and the non-linear features into the passenger comfort model to generate a fatigue state corresponding to the target user, and determining a corresponding comfort score according to the fatigue state.

[0011] Optionally, in an embodiment of the present application, the adaptively adjusting the environment of the multiple target in-vehicle devices based on the user intention instruction and the comfort score, and in combination with the priority and a preset energy allocation strategy includes: determining whether the comfort score is less than a preset comfort threshold, wherein when the comfort score is less than the comfort threshold, controlling the current vehicle to turn on the autonomous driving assistance mode; adaptively adjusting the target in-vehicle devices based on the user intention instruction, the body temperature data in the physiological characteristic data, and the priority, and obtaining the remaining power of the current vehicle, and when the remaining power is less than a preset power threshold, controlling the current vehicle to turn on the energy-saving mode.

[0012] Optionally, in an embodiment of the present application, after performing the adaptive environment adjustment operation on the multiple target in-vehicle devices, it further includes: generating an adaptive environment adjustment report corresponding to the current vehicle, sending the adaptive environment adjustment report to the target user in an acoustic and / or optical form, and obtaining feedback information corresponding to the target user; optimizing data fusion and environment adjustment according to the feedback information to generate a new user intention instruction, and re-performing the adaptive environment adjustment operation through the new user intention instruction.

[0013] The second aspect of the embodiments of the present application provides an adaptive environment adjustment device for an in-vehicle intelligent cockpit, including: an acquisition module, configured to acquire multimodal data of a target user in a current vehicle, where the multimodal data includes voice data, behavior data, and physiological characteristic data; a fusion module, configured to fuse the voice data and the behavior data to generate a corresponding user intention instruction, and input the physiological characteristic data into a pre-constructed passenger comfort model to output a comfort score corresponding to the target user; an adjustment module, configured to determine the priorities of multiple target in-vehicle devices in the current vehicle, and based on the user intention instruction and the comfort score, and in combination with the priorities and a preset energy allocation strategy, perform an adaptive environment adjustment operation on the multiple target in-vehicle devices.

[0014] Optionally, in an embodiment of the present application, the fusion module includes: a quantization unit, configured to acquire environmental noise information and light information of the current vehicle, and quantize the environmental noise information and the light information to obtain corresponding noise quantization information and light quantization information; an allocation unit, configured to dynamically allocate voice interaction weights and behavior interaction weights corresponding to the voice data and the behavior data based on the noise quantization information and the light quantization information, and determine a target interaction method according to the voice interaction weights and the behavior interaction weights, so as to generate the user intention instruction through the target interaction method.

[0015] Optionally, in an embodiment of the present application, the fusion module further includes: an extraction unit, configured to perform data preprocessing on the physiological characteristic data to obtain standard physiological data, and extract time-frequency features and non-linear features corresponding to heart rate data in the standard physiological data; a determination unit, configured to input the time-frequency features and the non-linear features into the passenger comfort model to generate a fatigue state corresponding to the target user, and determine a corresponding comfort score according to the fatigue state.

[0016] Optionally, in an embodiment of the present application, the adjustment module includes: a judgment unit, configured to judge whether the comfort score is less than a preset comfort threshold, where when the comfort score is less than the comfort threshold, control the current vehicle to turn on the automatic driving assistance mode; a control unit, configured to perform adaptive adjustment on the target in-vehicle devices based on the user intention instruction, body temperature data in the physiological characteristic data, and the priorities, and obtain the remaining power of the current vehicle, and when the remaining power is less than a preset power threshold, control the current vehicle to turn on the energy-saving mode.

[0017] Optionally, in an embodiment of the present application, it further includes: a feedback module, configured to generate an adaptive environment adjustment report corresponding to the current vehicle after performing an adaptive environment adjustment operation on the multiple target vehicle-mounted devices, send the adaptive environment adjustment report to the target user in an acoustic and / or optical form, and obtain feedback information corresponding to the target user; an optimization module, configured to optimize data fusion and environment adjustment according to the feedback information to generate a new user intention instruction, and re-perform the adaptive environment adjustment operation through the new user intention instruction.

[0018] An embodiment of the third aspect of the present application provides a vehicle, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the program to implement the adaptive environment adjustment method of the in-vehicle intelligent cockpit as described in the above embodiment.

[0019] An embodiment of the fourth aspect of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the program is executed by a processor, it implements the adaptive environment adjustment method of the in-vehicle intelligent cockpit as above.

[0020] An embodiment of the fifth aspect of the present application provides a computer program product, including a computer program, where the computer program is executed to implement the adaptive environment adjustment method of the in-vehicle intelligent cockpit as above.

[0021] Therefore, the embodiments of the present application have the following beneficial effects:

[0022] The embodiments of the present application can collect multi-modal data of the target user in the current vehicle, where the multi-modal data includes voice data, behavior data, and physiological characteristic data; fuse the voice data and behavior data to generate a corresponding user intention instruction, and input the physiological characteristic data into a pre-constructed passenger comfort model to output a comfort score corresponding to the target user; determine the priorities of multiple target vehicle-mounted devices in the current vehicle, and based on the user intention instruction and the comfort score, combine the priorities and a preset energy distribution strategy to perform an adaptive environment adjustment operation on the multiple target vehicle-mounted devices. The present application improves the interaction reliability in complex scenarios through a multi-modal interaction fusion intelligent cockpit system, can adjust the in-vehicle environment parameters in real time based on the passenger's physiological characteristics, and can also optimize the energy distribution through device collaborative control, thereby reducing the system power consumption. Thus, it solves the problems that the interaction methods in the prior art cannot adapt to complex scenarios, have a low degree of intelligence, and the independent operation of multiple devices leads to energy waste and low system energy efficiency.

[0023] Additional aspects and advantages of the present application will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present application. Brief Description of the Drawings

[0024] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description of embodiments in conjunction with the accompanying drawings, where:

[0025] Figure 1 It is a flowchart of an adaptive environment adjustment method for an in-vehicle intelligent cockpit provided according to an embodiment of the present application;

[0026] Figure 2 It is a schematic diagram of the execution logic of an adaptive environment adjustment method for an in-vehicle intelligent cockpit provided by an embodiment of the present application;

[0027] Figure 3 It is an example diagram of an adaptive environment adjustment device for an in-vehicle intelligent cockpit according to an embodiment of the present application;

[0028] Figure 4 It is a schematic diagram of the structure of a vehicle provided by an embodiment of the present application.

[0029] Among them, 10 - an adaptive environment adjustment device for an in-vehicle intelligent cockpit; 100 - a collection module, 200 - a fusion module, 300 - an adjustment module; 401 - a memory, 402 - a processor, 403 - a communication interface. Detailed Description of the Embodiments

[0030] The embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present application and should not be construed as limiting the present application.

[0031] The following describes the adaptive environment adjustment method and device for an in-vehicle intelligent cockpit according to an embodiment of the present application. In view of the problems mentioned in the above background art, the present application provides an adaptive environment adjustment method for an in-vehicle intelligent cockpit. In this method, by collecting multimodal data of a target user in the current vehicle, where the multimodal data includes voice data, behavior data, and physiological characteristic data; fusing the voice data and behavior data to generate a corresponding user intention instruction, and inputting the physiological characteristic data into a pre-constructed passenger comfort model to output a comfort score corresponding to the target user; determining the priorities of multiple target in-vehicle devices in the current vehicle, and based on the user intention instruction and the comfort score, combining the priorities and a preset energy distribution strategy to perform an adaptive environment adjustment operation on the multiple target in-vehicle devices. The present application uses an intelligent cockpit system with multimodal interaction fusion to improve the interaction reliability in complex scenarios, can adjust the in-vehicle environment parameters in real time based on the passenger's physiological characteristics, and can also optimize the energy distribution through device collaborative control, thereby reducing the system power consumption. Thus, the problems in the prior art that the interaction method cannot adapt to complex scenarios, the degree of intelligence is low, and the independent operation of multiple devices leads to energy waste and low system energy efficiency are solved.

[0032] Specifically, Figure 1 is a flowchart of an adaptive environment adjustment method for an in-vehicle intelligent cockpit provided by an embodiment of the present application.

[0033] As Figure 1 shown, the adaptive environment adjustment method for the in-vehicle intelligent cockpit includes the following steps:

[0034] In step S101, collect multimodal data of a target user in the current vehicle, where the multimodal data includes voice data, behavior data, and physiological characteristic data.

[0035] In step S102, fuse the voice data and behavior data to generate a corresponding user intention instruction, and input the physiological characteristic data into a pre-constructed passenger comfort model to output a comfort score corresponding to the target user.

[0036] In an embodiment of the present application, first, the voice data of the user can be obtained through a voice recognition unit constructed by a microphone array, and the environmental noise can be filtered to extract a clear voice signal; at the same time, in an embodiment of the present application, a gesture sensor constructed by a TOF camera can also be used to capture the hand movements of the passenger, etc., so as to obtain corresponding behavior data to identify preset instructions such as swiping to change songs; in addition, an embodiment of the present application can also construct a corresponding physiological characteristic acquisition unit through an infrared body temperature sensor, a heart rate monitoring seat, etc., to collect physiological characteristic data such as the body temperature and heart rate of the user.

[0037] Secondly, embodiments of the present application can also fuse multimodal input data such as voice data and behavior data through a central processing unit to generate user intent instructions, and predict the comfort requirements of passengers based on physiological characteristic data through machine learning algorithms, etc.

[0038] Thus, embodiments of the present application provide an intelligent cockpit system that integrates multimodal interactions (voice, gesture, physiological signals), thereby improving the interaction reliability in complex scenarios.

[0039] Optionally, in an embodiment of the present application, fusing voice data and behavior data to generate corresponding user intent instructions includes: collecting environmental noise information and light information of the current vehicle, and quantifying the environmental noise information and light information to obtain corresponding noise quantization information and light quantization information; based on the noise quantization information and light quantization information, dynamically allocating the corresponding voice interaction weight and behavior interaction weight for voice data and behavior data, and determining the target interaction method according to the voice interaction weight and behavior interaction weight, so as to generate user intent instructions through the target interaction method.

[0040] During the actual execution process, when there is a conflict between voice and gesture instructions, embodiments of the present application can select the execution instruction according to the confidence weight through the central processing unit.

[0041] As a feasible way, embodiments of the present application can automatically select the optimal interaction method according to factors such as environmental noise and light conditions through a weighted decision algorithm, which is specifically described as follows:

[0042] 1. Collect data such as environmental noise and light conditions through sensors;

[0043] 2. Extract features such as the spectrum, energy, and signal-to-noise ratio of voice data; extract features such as the shape, position, and movement trajectory of gestures, and quantify the environmental noise level (such as decibel value) and light intensity (such as brightness value) to obtain corresponding noise quantization information and light quantization information;

[0044] 3. Dynamically allocate the weights of voice and gesture interactions according to the noise quantization information and light quantization information, and thus select the optimal interaction method according to the weight values;

[0045] For example, embodiments of the present application can reduce the weight of voice interaction and increase the weight of gesture interaction in a high-noise environment, and reduce the weight of gesture interaction and increase the weight of voice interaction in a low-light environment. If the voice weight is greater than the gesture weight, voice interaction is preferred; otherwise, gesture interaction is preferred;

[0046] 4. Integrate voice and gesture information. For example, when the voice instruction is unclear, combine gesture information for supplementation to improve the robustness of the interaction.

[0047] Thus, the embodiments of the present application can implement multi-modal interaction fusion of gestures and voices in a vehicle, be able to process interaction data in complex environments, improve the efficiency of human-computer interaction and the user experience, and optimize the selection logic of interaction methods.

[0048] Optionally, in an embodiment of the present application, physiological feature data is input into a pre-constructed passenger comfort model to output a comfort score corresponding to the target user, including: performing data preprocessing on the physiological feature data to obtain standard physiological data, and extracting the time-frequency features and non-linear features corresponding to the heart rate data in the standard physiological data; inputting the time-frequency features and non-linear features into the passenger comfort model to generate the fatigue state corresponding to the target user, and determining the corresponding comfort score according to the fatigue state.

[0049] Furthermore, the embodiments of the present application analyze the passenger fatigue state through heart rate variability (HRV), which is specifically described as follows:

[0050] 1. Use wearable devices (such as smart watches, chest straps) or in-vehicle sensors to collect heart rate data, and perform data preprocessing operations such as removing noise and outliers on it to ensure data quality;

[0051] 2. Extract the HRV features of the heart rate data after data preprocessing, which include time domain features (such as average heart rate, standard deviation, root mean square difference, etc.), frequency domain features (extracting low-frequency and high-frequency components through Fourier transform or wavelet transform), and non-linear features such as sample entropy and fractal dimension;

[0052] 3. Select HRV features with strong correlation with the fatigue state, and use machine learning or deep learning models (such as SVM, random forest, LSTM) to construct a passenger comfort model to classify the fatigue state and determine the corresponding comfort score according to the fatigue state.

[0053] Thus, the embodiments of the present application can analyze the passenger fatigue state through heart rate variability, so as to be able to monitor the physiological state of passengers in real time and provide an important reference for the intelligent cockpit or driving assistance system.

[0054] In step S103, determine the priorities of multiple target in-vehicle devices in the current vehicle, and based on the user intention instruction and the comfort score, and in combination with the priorities and the preset energy allocation strategy, perform adaptive environment adjustment operations on the multiple target in-vehicle devices.

[0055] Furthermore, as Figure 2As shown, the embodiments of the present application can dynamically adjust in-vehicle environment parameters (temperature, seat angle, lighting) based on passenger biometrics (body temperature, heart rate, fatigue level) in real time, and allocate energy according to priorities to optimize energy distribution through device collaborative control. For example, non-essential devices are preferentially turned off when the power is low, thereby reducing system power consumption.

[0056] Optionally, in an embodiment of the present application, based on user intent instructions and comfort scores, combined with priorities and preset energy distribution strategies, adaptive environment adjustment operations are performed on multiple target in-vehicle devices, including: determining whether the comfort score is less than a preset comfort threshold. When the comfort score is less than the comfort threshold, control the current vehicle to turn on the autonomous driving assistance mode; based on the user intent instructions, body temperature data in the physiological characteristic data, and priorities, perform adaptive adjustment on the target in-vehicle devices, and obtain the remaining power of the current vehicle. When the remaining power is less than the preset power threshold, control the current vehicle to turn on the energy-saving mode.

[0057] In the embodiments of the present application, the fatigue state of passengers can be analyzed through heart rate variability, and the comfort threshold is set according to HRV characteristics to determine whether the comfort score is less than the comfort threshold. When the comfort score is less than the comfort threshold, trigger a voice reminder and automatically switch to the driving assistance mode.

[0058] Secondly, the embodiments of the present application can use an infrared sensor to detect the forehead temperature. If it is >37.3°C, the air conditioner temperature is automatically reduced by 2°C and the seat ventilation is turned on.

[0059] In addition, the embodiments of the present application can also perform priority sorting. For example, safety devices (such as instrument panels) have the highest priority, and entertainment devices (such as screens) have the second highest priority. The power is dynamically adjusted based on device usage frequency and priorities. For example, the screen brightness is reduced at night and the ambient light is turned off, and the remaining power of the current vehicle is obtained. When the battery power is <30%, the co-pilot screen is turned off and the air conditioner is switched to the energy-saving mode.

[0060] In the specific experimental and testing process, when the in-vehicle noise >70dB, the accuracy of voice and gesture fusion recognition reaches 95%, and the success rate is increased by 30%; the satisfaction of 100 subjects in the biometric adaptive mode is 4.8 / 5, and the comfort score is increased by 25%; in the urban commuting mode, device collaborative control reduces redundant energy consumption, and the overall power consumption is reduced by 20%.

[0061] Thus, the embodiments of the present application can achieve adaptive adjustment of environmental parameters, enable multi-device collaborative control, reduce system energy consumption, and greatly improve the user experience.

[0062] Optionally, in an embodiment of the present application, after performing the adaptive environment adjustment operation on multiple target vehicle-mounted devices, it further includes: generating an adaptive environment adjustment report corresponding to the current vehicle, sending the adaptive environment adjustment report to the target user in an acoustic and / or optical form, and obtaining feedback information corresponding to the target user; optimizing data fusion and environment adjustment according to the feedback information to generate a new user intention instruction, and re-performing the adaptive environment adjustment operation through the new user intention instruction.

[0063] It should be noted that the embodiment of the present application can also generate an adaptive environment adjustment report, and broadcast the adaptive environment adjustment report by voice or display the adaptive environment adjustment report on the in-vehicle display screen, and obtain the corresponding feedback information of the user. For example, the user can send a voice command to put forward specific environment optimization suggestions, etc., or input feedback opinions in the pop-up window of the in-vehicle display screen; then, the embodiment of the present application can optimize data fusion and environment adjustment according to the feedback information to generate a new user intention instruction, so as to re-perform the adaptive environment adjustment operation.

[0064] Thus, the embodiment of the present application optimizes the adaptive environment adjustment of the vehicle-mounted device according to the user's own needs, greatly improves the user experience, and improves the humanization degree and intelligent level of the vehicle.

[0065] According to the adaptive environment adjustment method of the in-vehicle intelligent cockpit proposed by the embodiment of the present application, by collecting multi-modal data of the target user in the current vehicle, where the multi-modal data includes voice data, behavior data, and physiological characteristic data; fusing the voice data and behavior data to generate a corresponding user intention instruction, and inputting the physiological characteristic data into a pre-constructed passenger comfort model to output a comfort score corresponding to the target user; determining the priorities of multiple target vehicle-mounted devices in the current vehicle, and based on the user intention instruction and the comfort score, combining the priorities and a preset energy allocation strategy, performing an adaptive environment adjustment operation on the multiple target vehicle-mounted devices. The present application improves the interaction reliability in complex scenarios through a multi-modal interaction fusion intelligent cockpit system, can adjust the in-vehicle environment parameters in real time based on the passenger's physiological characteristics, and can also optimize the energy allocation through device collaborative control, thereby reducing the system power consumption.

[0066] Secondly, an adaptive environment adjustment device of the in-vehicle intelligent cockpit according to an embodiment of the present application is described with reference to the accompanying drawings.

[0067] Figure 3 It is a block diagram of the adaptive environment adjustment device of the in-vehicle intelligent cockpit according to an embodiment of the present application.

[0068] As Figure 3As shown, the adaptive environment adjustment device 10 of the in-vehicle intelligent cockpit includes: a collection module 100, a fusion module 200, and an adjustment module 300.

[0069] Among them, the collection module 100 is used to collect multi-modal data of the target user in the current vehicle, where the multi-modal data includes voice data, behavior data, and physiological characteristic data.

[0070] The fusion module 200 is used to fuse the voice data and behavior data to generate corresponding user intention instructions, and input the physiological characteristic data into a pre-constructed passenger comfort model to output a comfort score corresponding to the target user.

[0071] The adjustment module 300 is used to determine the priorities of multiple target in-vehicle devices in the current vehicle, and based on the user intention instructions and comfort score, combined with the priorities and a preset energy allocation strategy, perform an adaptive environment adjustment operation on the multiple target in-vehicle devices.

[0072] Optionally, in an embodiment of the present application, the fusion module 200 includes: a quantization unit and an allocation unit.

[0073] Among them, the quantization unit is used to collect the environmental noise information and light information of the current vehicle, and quantize the environmental noise information and light information to obtain corresponding noise quantization information and light quantization information.

[0074] The allocation unit is used to dynamically allocate the voice interaction weight and behavior interaction weight corresponding to the voice data and behavior data based on the noise quantization information and light quantization information, and determine the target interaction method according to the voice interaction weight and behavior interaction weight, so as to generate user intention instructions through the target interaction method.

[0075] Optionally, in an embodiment of the present application, the fusion module 200 further includes: an extraction unit and a determination unit.

[0076] Among them, the extraction unit is used to perform data preprocessing on the physiological characteristic data to obtain standard physiological data, and extract the time-frequency characteristics and non-linear characteristics corresponding to the heart rate data in the standard physiological data.

[0077] The determination unit is used to input the time-frequency characteristics and non-linear characteristics into the passenger comfort model to generate the fatigue state corresponding to the target user, and determine the corresponding comfort score according to the fatigue state.

[0078] Optionally, in an embodiment of the present application, the adjustment module 300 includes: a judgment unit and a control unit.

[0079] Among them, a judgment unit is configured to judge whether the comfort score is less than a preset comfort threshold. When the comfort score is less than the comfort threshold, the current vehicle is controlled to turn on the assisted driving mode.

[0080] A control unit is configured to adaptively adjust a target vehicle-mounted device based on a user intention instruction, body temperature data in physiological characteristic data, and a priority, obtain the remaining power of the current vehicle, and when the remaining power is less than a preset power threshold, control the current vehicle to turn on an energy-saving mode.

[0081] Optionally, in an embodiment of the present application, the adaptive environment adjustment device 10 of the in-vehicle intelligent cockpit of the present application embodiment further includes: a feedback module and an optimization module.

[0082] Among them, the feedback module is configured to generate an adaptive environment adjustment report corresponding to the current vehicle after performing an adaptive environment adjustment operation on a plurality of target vehicle-mounted devices, send the adaptive environment adjustment report to a target user in an acoustic and / or optical form, and obtain feedback information corresponding to the target user.

[0083] The optimization module is configured to optimize data fusion and environment adjustment according to the feedback information to generate a new user intention instruction, and re-perform the adaptive environment adjustment operation through the new user intention instruction.

[0084] It should be noted that the foregoing explanation of the embodiment of the adaptive environment adjustment method of the in-vehicle intelligent cockpit also applies to the adaptive environment adjustment device of the in-vehicle intelligent cockpit of this embodiment, and will not be elaborated here.

[0085] The adaptive environment adjustment device of the in-vehicle intelligent cockpit according to the embodiment of the present application includes an acquisition module 100 configured to acquire multi-modal data of a target user in the current vehicle, where the multi-modal data includes voice data, behavior data, and physiological characteristic data; a fusion module 200 configured to fuse the voice data and the behavior data to generate a corresponding user intention instruction, and input the physiological characteristic data into a pre-constructed passenger comfort model to output a comfort score corresponding to the target user; an adjustment module 300 configured to determine the priorities of a plurality of target vehicle-mounted devices in the current vehicle, and based on the user intention instruction and the comfort score, and in combination with the priority and a preset energy distribution strategy, perform an adaptive environment adjustment operation on the plurality of target vehicle-mounted devices. The present application improves the interaction reliability in complex scenarios through a multi-modal interaction fusion intelligent cockpit system, can adjust the in-vehicle environment parameters in real time based on the passenger's physiological characteristics, and can also optimize the energy distribution through device collaborative control, thereby reducing the system power consumption.

[0086] Figure 4 It is a schematic structural diagram of a vehicle provided by an embodiment of the present application. The vehicle may include:

[0087] A memory 401, a processor 402, and a computer program stored on the memory 401 and executable on the processor 402.

[0088] When the processor 402 executes the program, it implements the adaptive environment adjustment method for the in-vehicle intelligent cockpit provided in the above embodiments.

[0089] Furthermore, the vehicle further includes:

[0090] A communication interface 403 for communication between the memory 401 and the processor 402.

[0091] The memory 401 is used to store a computer program executable on the processor 402.

[0092] The memory 401 may include a high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0093] If the memory 401, the processor 402, and the communication interface 403 are implemented independently, the communication interface 403, the memory 401, and the processor 402 can be interconnected through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 4 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0094] Optionally, in a specific implementation, if the memory 401, the processor 402, and the communication interface 403 are integrated on a chip, the memory 401, the processor 402, and the communication interface 403 can communicate with each other through an internal interface.

[0095] The processor 402 may be a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0096] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the adaptive environment adjustment method of the in-vehicle intelligent cockpit as described above is implemented.

[0097] An embodiment of the present application also provides a computer program product, including a computer program, which is used to implement the adaptive environment adjustment method of the in-vehicle intelligent cockpit as described above when the computer program is executed.

[0098] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0099] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "N" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0100] Any process or method description shown in the flowchart or described in other ways herein can be understood as representing a module, segment, or part of code including one or N executable instructions for implementing a customized logic function or process. The scope of the preferred embodiments of the present application includes additional implementations, where the functions may be executed in a substantially simultaneous manner or in an order opposite to that shown or discussed, according to the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0101] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definable sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or used in combination with these instruction execution systems, apparatuses, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in combination with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion (electronic device) having one or N wirings, a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing as appropriate, and then stored in a computer memory.

[0102] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or combinations thereof. In the above-described embodiments, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented using hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits having suitable combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0103] Those of ordinary skill in the art of the present technology can understand that all or part of the steps carried by the method of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0104] In addition, each functional unit in various embodiments of the present application may be integrated into a processing module, may exist separately as individual physical units, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0105] The above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disc, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. An adaptive environment adjustment method for an in-vehicle intelligent cockpit, characterized in that Including the following steps: Collect multi-modal data of the target user in the current vehicle, where the multi-modal data includes voice data, behavior data, and physiological characteristic data; Fuse the voice data and the behavior data to generate a corresponding user intention instruction, and input the physiological characteristic data into a pre-constructed passenger comfort model to output a comfort score corresponding to the target user; Determine the priorities of multiple target in-vehicle devices in the current vehicle, and based on the user intention instruction and the comfort score, and in combination with the priorities and a preset energy allocation strategy, perform an adaptive environment adjustment operation on the multiple target in-vehicle devices.

2. The method according to claim 1, wherein The fusing the voice data and the behavior data to generate a corresponding user intention instruction includes: Collect the ambient noise information and light information of the current vehicle, and quantify the ambient noise information and the light information to obtain corresponding noise quantization information and light quantization information; Based on the noise quantization information and the light quantization information, dynamically allocate the voice interaction weight and the behavior interaction weight corresponding to the voice data and the behavior data, and determine a target interaction method according to the voice interaction weight and the behavior interaction weight, so as to generate the user intention instruction through the target interaction method.

3. The method according to claim 2, wherein The inputting the physiological characteristic data into a pre-constructed passenger comfort model to output a comfort score corresponding to the target user includes: Perform data preprocessing on the physiological characteristic data to obtain standard physiological data, and extract the time-frequency characteristics and non-linear characteristics corresponding to the heart rate data in the standard physiological data; Input the time-frequency characteristics and the non-linear characteristics into the passenger comfort model to generate a fatigue state corresponding to the target user, and determine a corresponding comfort score according to the fatigue state.

4. The method according to claim 3, wherein The performing an adaptive environment adjustment operation on the multiple target in-vehicle devices based on the user intention instruction and the comfort score, and in combination with the priorities and a preset energy allocation strategy includes: Judge whether the comfort score is less than a preset comfort threshold, where when the comfort score is less than the comfort threshold, control the current vehicle to turn on the automatic driving assistance mode; Based on the user intention instruction, the body temperature data in the physiological characteristic data, and the priorities, perform adaptive adjustment on the target in-vehicle devices, obtain the remaining power of the current vehicle, and when the remaining power is less than a preset power threshold, control the current vehicle to turn on the energy-saving mode.

5. The method according to claim 1, wherein After performing the adaptive environment adjustment operation on the multiple target in-vehicle devices, it further includes: Generate an adaptive environment adjustment report corresponding to the current vehicle, send the adaptive environment adjustment report to the target user in an acoustic and / or optical form, and obtain feedback information corresponding to the target user; Optimize data fusion and environment adjustment according to the feedback information to generate a new user intention instruction, and re-perform the adaptive environment adjustment operation through the new user intention instruction.

6. An adaptive environment adjustment device for a vehicle intelligent cockpit, characterized in that, Including: A collection module, configured to collect multimodal data of a target user in a current vehicle, where the multimodal data includes voice data, behavior data, and physiological characteristic data; A fusion module, configured to fuse the voice data and the behavior data to generate a corresponding user intention instruction, and input the physiological characteristic data into a pre-constructed passenger comfort model to output a comfort score corresponding to the target user; An adjustment module, configured to determine the priorities of multiple target in-vehicle devices in the current vehicle, and based on the user intention instruction and the comfort score, and in combination with the priorities and a preset energy allocation strategy, perform an adaptive environment adjustment operation on the multiple target in-vehicle devices.

7. The device according to claim 6, wherein The fusion module includes: A quantization unit, configured to collect ambient noise information and light information of the current vehicle, and quantize the ambient noise information and the light information to obtain corresponding noise quantization information and light quantization information; An allocation unit, configured to dynamically allocate a voice interaction weight and a behavior interaction weight corresponding to the voice data and the behavior data based on the noise quantization information and the light quantization information, and determine a target interaction method according to the voice interaction weight and the behavior interaction weight, so as to generate the user intention instruction through the target interaction method.

8. A vehicle, characterized in that, including: A memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the program to implement the adaptive environment adjustment method of the in-vehicle intelligent cockpit according to any one of claims 1-5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to be used to implement the adaptive environment adjustment method of the in-vehicle intelligent cockpit according to any one of claims 1-5.

10. A computer program product, comprising a computer program, characterized in that, The computer program is executed to be used to implement the adaptive environment adjustment method of the in-vehicle intelligent cockpit according to any one of claims 1-5.

Images