Intelligent cabin system and control method thereof

Abstract

The invention provides an intelligent cabin system and a control method thereof, and belongs to the technical field of intelligent cabins, and the intelligent cabin system comprises an emotion perception module which is used for obtaining the emotion data of a user in a vehicle; the environment sensing module is used for acquiring environment data outside the vehicle; the central controller is used for receiving the in-vehicle user emotion data of the emotion sensing module and the out-vehicle environment data of the environment sensing module, generating vehicle body display content according to the in-vehicle user emotion data and / or the out-vehicle environment data, and controlling the vehicle body display module to display the vehicle body display content; and the vehicle body display module is used for displaying the vehicle body display content on the surface of the vehicle body. Linkage of in-vehicle emotion and out-of-vehicle display can be achieved, the interaction ability between the vehicle and the surrounding environment is enhanced, and the driving experience of a user is improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent cockpit technology, and in particular to an intelligent cockpit system and its control method. Background Technology

[0002] In related technologies, intelligent cockpit systems primarily focus on the immersive experience of the in-vehicle environment, lacking the ability to interact with the external environment. When the vehicle is moving or parked, it cannot effectively communicate with surrounding pedestrians or vehicles, nor can it adjust the content displayed on the vehicle body. Furthermore, vehicle advertising is typically static, lacking personalization and interactivity. Summary of the Invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art, and proposes an intelligent cockpit system and its control method.

[0004] In a first aspect, embodiments of the present invention provide an intelligent cockpit system, the intelligent cockpit system comprising:

[0005] The emotion perception module is used to acquire emotional data of users inside the vehicle;

[0006] The environmental perception module is used to acquire data about the external environment of the vehicle;

[0007] The central controller is used to receive in-vehicle user emotion data from the emotion perception module and external environment data from the environment perception module, and to generate vehicle body display content based on the in-vehicle user emotion data and / or the external environment data, and to control the vehicle body display module to display the vehicle body display content.

[0008] The vehicle body display module is used to display the vehicle body display content on the vehicle body surface.

[0009] Secondly, embodiments of the present invention provide a control method for an intelligent cockpit system, the intelligent cockpit system including the intelligent cockpit system of the above embodiments, the control method including:

[0010] Acquire in-vehicle user emotional data and external environment data;

[0011] The vehicle body display content is generated based on the in-vehicle user emotion data and / or the external environment data.

[0012] The vehicle body display content is displayed on the vehicle body surface.

[0013] The intelligent cockpit system provided by this invention generates vehicle display content by sensing the emotional data of users inside the vehicle and the external environment data, and displays it on the vehicle body. This realizes the linkage between in-vehicle emotions and external displays, enhances the vehicle's ability to interact with its surroundings, improves the emotional expression of the car owner, and supports personalized vehicle body advertising, thereby enhancing the user's driving experience. Attached Figure Description

[0014] Figure 1 A system block diagram of an intelligent cockpit system provided in an embodiment of the present invention;

[0015] Figure 2 A system block diagram of another intelligent cockpit system provided in an embodiment of the present invention;

[0016] Figure 3 This is a block diagram of an emotion perception module in an embodiment of the present invention;

[0017] Figure 4 This is a block diagram illustrating the composition of an environmental sensing module in an embodiment of the present invention.

[0018] Figure 5 This is a flowchart illustrating a control method for an intelligent cockpit system provided in an embodiment of the present invention. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions of the present invention, exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0020] Where there is no conflict, the various embodiments of the present invention and the features thereof may be combined with each other.

[0021] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0022] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Terms such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0023] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having the meaning consistent with their meaning in the context of the relevant art and the invention, and will not be interpreted as having an idealized or overly formal meaning unless expressly so defined herein.

[0024] In related technologies, intelligent cockpit systems primarily focus on the immersive experience of the in-vehicle environment, lacking the ability to interact with the external environment. When the vehicle is moving or parked, it cannot effectively communicate with surrounding pedestrians or vehicles, nor can it adjust the content displayed on the vehicle body. Furthermore, vehicle advertising is typically static, lacking personalization and interactivity.

[0025] To address at least one of the technical problems existing in the aforementioned related technologies, the present invention provides an intelligent cockpit system. Figure 1 A system block diagram of an intelligent cockpit system provided in an embodiment of the present invention is shown below. Figure 1 As shown, the intelligent cockpit system 100 includes:

[0026] The Emotion Perception Module (EPM) 101 is used to acquire in-vehicle user emotion data and transmit the in-vehicle user emotion data to the central controller 103.

[0027] The Environment Perception Module (EnPM) 102 is used to acquire external environmental data and transmit the external environmental data to the central controller 103.

[0028] The Central Control Unit (CCU) 103 is used to receive in-vehicle user emotion data from the emotion perception module 101 and external environment data from the environment perception module 102, generate vehicle body display content based on the in-vehicle user emotion data and / or external environment data, and control the vehicle body display module 104 to display the vehicle body display content.

[0029] The Vehicle Display Module (VDM) 104 is used to display vehicle display content on the vehicle body surface.

[0030] According to the intelligent cockpit system of the present invention, by sensing the emotional data of the user inside the vehicle and the external environment data, the system generates vehicle display content and displays it on the vehicle body, realizing the linkage between in-vehicle emotions and external displays, enhancing the vehicle's ability to interact with its surrounding environment, improving the emotional expression of the car owner, and supporting personalized vehicle advertising to enhance the user's driving experience.

[0031] Figure 2 The present invention provides another intelligent cockpit system system. In some embodiments, the intelligent cockpit system further includes a user interface (UI) 105. The user interface 105 can provide users with an operation interface for customizing vehicle body display content through a touch screen or voice control, so that users can perform vehicle body display operations according to their needs.

[0032] In some embodiments, the user interface 105 may also provide a vehicle body display content selection interface, where the vehicle owner can select the vehicle body display content by touch or voice command.

[0033] In some embodiments, the user interface 105 obtains vehicle body display content selected or customized by the user and sends a vehicle body display instruction to the central controller 103, which may carry the vehicle body display content selected or customized by the user.

[0034] In some embodiments, the user interface 105 may employ a high-resolution touchscreen (such as a central control screen of 10 inches or larger) and / or a voice control module (such as a microphone array).

[0035] The touchscreen interface can adopt an intuitive graphical design and support operations such as swiping and clicking. The voice control module supports natural language processing (NLP), and users can switch the content displayed on the vehicle body through voice commands.

[0036] In some embodiments, the user interface 105 is a graphical user interface (GUI) that supports touch operation and voice control. Users can select the content displayed on the vehicle body and view detailed information about the currently displayed content through the interface.

[0037] In some embodiments, the user interface 105 can also display detailed information of the current vehicle body display content, and supports user customization of the vehicle body display content, providing an interactive interface for user customization of the vehicle body display content.

[0038] In some embodiments, the central controller 103 is responsible for the coordination and control of the entire system. It can employ a high-performance embedded processor (such as an ARM Cortex-A series or an NVIDIA Tegra series) that supports multi-threaded processing and real-time computing capabilities. The central controller 103 is responsible for receiving vehicle display content input from the user interface, as well as receiving in-vehicle user emotion data from the emotion perception module and external environment data from the environment perception module. It generates vehicle display content based on the in-vehicle user emotion data and / or external environment data, and controls the vehicle display module 104 to display the vehicle display content.

[0039] In some embodiments, the central controller 101 includes multiple input and output interfaces, supporting CAN bus, LIN bus, or Ethernet communication to ensure real-time and reliable data transmission. The input interfaces are used to connect to the emotion perception module 101 and the environment perception module 102, etc., while the output interfaces are used to connect to the vehicle body display module 104. The central controller 101 can communicate with each module via CAN bus, LIN bus, or Ethernet. The communication protocol can adopt a standardized format (such as CAN protocol or TCP / IP protocol) to ensure the reliability and real-time performance of data transmission.

[0040] Figure 3 This is a block diagram of an emotion perception module in an embodiment of the present invention. In some embodiments, the in-vehicle user emotion data includes in-vehicle user facial data and / or in-vehicle user voice data. The emotion perception module 101 includes an in-vehicle camera module 1011 and a voice recognition module 1012.

[0041] The in-vehicle camera module 1011 is used to acquire facial data of the user inside the vehicle and transmit the facial data of the user inside the vehicle to the central controller 103. The voice recognition module 1012 is used to acquire user voice data and transmit user voice data to the central controller 103.

[0042] Among them, the facial data of the user in the vehicle includes, but is not limited to, the user's facial image, and the voice recognition module 1012 can be a microphone device in the vehicle.

[0043] In some embodiments, the in-vehicle camera module 1011 may be arranged near the dashboard or rearview mirror, and the voice recognition module 1012 may be integrated into the in-vehicle microphone array.

[0044] Figure 4 This is a block diagram illustrating the composition of an environmental sensing module according to an embodiment of the present invention. In some embodiments, such as... Figure 4 As shown, the vehicle exterior environment data includes at least one of the following: vehicle exterior temperature, vehicle exterior humidity, vehicle exterior light intensity, and environmental information surrounding the vehicle body. The environmental perception module 102 includes at least one of the following:

[0045] Temperature sensor 1021 is used to collect the outside temperature of the vehicle and transmit the outside temperature to the central controller 103.

[0046] Humidity sensor 1022 is used to collect the humidity outside the vehicle and transmit the humidity outside the vehicle to the central controller 103.

[0047] The light sensor 1023 is used to collect the light intensity outside the vehicle and transmit the light intensity outside the vehicle to the central controller 103.

[0048] The exterior camera module 1024 is used to detect environmental information around the vehicle and transmit the environmental information around the vehicle to the central controller 103.

[0049] Among them, the temperature sensor 1021 and humidity sensor 1022 can be arranged in multiple locations outside the vehicle to detect the outside temperature and humidity; the light sensor 1023 can be arranged near the window to monitor changes in the intensity of light outside the vehicle; the external camera module 1024 can be arranged around the vehicle to obtain environmental information around the vehicle, such as pedestrians and vehicles.

[0050] In some embodiments, the vehicle body display module 104 may include a display screen and a projection device; the display screen may be an LED screen, arranged on the surface of the vehicle body, such as the door, roof, or rear of the vehicle body, and is used to display vehicle body display content to the outside of the vehicle body, including but not limited to patterns, text, or dynamic effects; the projection device is used to project patterns, text, or dynamic effects of the vehicle body display content onto the display screen on the surface of the vehicle body, and the projection device may be arranged inside the vehicle body and projected through a transparent window.

[0051] In some embodiments, the vehicle display module 104 can display generated vehicle display content according to the instructions of the central controller 103, and supports a variety of display effects, such as gradient, flashing, dynamic patterns, etc.

[0052] In some embodiments, in-vehicle user emotion data includes in-vehicle user facial data. The central controller 103 is further configured to: perform emotion classification and prediction on the in-vehicle user facial data using a preset emotion perception algorithm to determine the user's facial expression; generate vehicle body display content based on the user's facial expression, and send a vehicle body display instruction to the vehicle body display module 104, the instruction carrying the vehicle body display content.

[0053] In some embodiments, the preset emotion perception algorithm may employ machine learning algorithms, such as support vector machines or neural networks, to classify and predict user facial expressions based on in-vehicle user facial data.

[0054] In some embodiments, the central controller 103 can generate corresponding vehicle display content based on the user's facial expressions using a content generation algorithm.

[0055] In some embodiments, the content generation algorithm may include vehicle display content corresponding to different user facial expressions, and a mapping relationship between user facial expressions and vehicle display content. When a user's facial expression is recognized, the corresponding vehicle display content is generated through the content generation algorithm.

[0056] For example, when a happy expression is detected from the car owner's face, a content generation algorithm is used to generate vehicle display content containing cheerful patterns and text.

[0057] In some embodiments, in-vehicle user emotion data includes in-vehicle user voice data. The central controller 103 is further configured to: perform emotion classification and prediction on the in-vehicle user voice data using a preset emotion perception algorithm to determine the user's voice emotion; generate vehicle body display content based on the user's voice emotion, and send a vehicle body display instruction to the vehicle body display module 104, the instruction carrying the vehicle body display content.

[0058] In some embodiments, the preset emotion perception algorithm may employ machine learning algorithms, such as support vector machines or neural networks, to classify and predict the emotions in a user's voice based on in-vehicle user voice data.

[0059] In some embodiments, the central controller 103 can generate corresponding vehicle display content based on the user's voice emotion using a content generation algorithm.

[0060] In some embodiments, the content generation algorithm may include vehicle display content corresponding to different user voice emotions, and a mapping relationship between user voice emotions and vehicle display content. When a user's voice emotion is identified, the corresponding vehicle display content is generated through the content generation algorithm.

[0061] For example, when a user's voice is detected to be in a pleasant mood, a content generation algorithm is used to generate vehicle display content containing cheerful patterns and text.

[0062] In some embodiments, the central controller 103 can also determine the user's emotional state based on the user's facial expressions and vocal emotions, and generate corresponding vehicle display content based on the user's emotional state. The user's facial expressions and vocal emotions are determined using the methods described above.

[0063] In some embodiments, the central controller 103 can generate corresponding vehicle display content based on the user's emotional state using a content generation algorithm.

[0064] In some embodiments, the content generation algorithm may include vehicle display content corresponding to different user emotional states, and a mapping relationship between user emotional states and vehicle display content. When a user's emotional state is identified, the corresponding vehicle display content is generated through the content generation algorithm.

[0065] For example, when a user's emotional state is detected to be excited, a content generation algorithm generates vehicle display content containing cheerful patterns and text.

[0066] In some embodiments, the preset emotion perception algorithm can run on the embedded AI chip (such as NVIDIA Jetson or Qualcomm Snapdragon) of the central controller 103 to ensure low latency and high accuracy.

[0067] In some embodiments, the central controller 103 is further configured to: perform environmental classification prediction on the external environment data through a preset environmental perception algorithm to determine the type of external environment; and generate vehicle body display content based on the type of external environment.

[0068] In some embodiments, the preset environment perception algorithm may employ an image processing algorithm (such as OpenCV) to perform environmental classification and prediction on the vehicle's external environment data.

[0069] In some embodiments, the central controller 103 can generate corresponding vehicle body display content based on the type of external environment through a content generation algorithm.

[0070] In some embodiments, the content generation algorithm may include vehicle body display content corresponding to different vehicle exterior environment types, i.e., a mapping relationship between vehicle exterior environment types and vehicle body display content. When a vehicle exterior environment type is identified, the corresponding vehicle body display content is generated through the content generation algorithm.

[0071] For example, when the external environment of the vehicle is detected to be rainy, a content generation algorithm is used to generate vehicle display content containing rain-related patterns and text.

[0072] In some embodiments, the central controller 103 is further configured to: generate vehicle body display content based on in-vehicle user emotional data and external environment data. Specifically, the central controller 103 determines the user's emotional state based on the in-vehicle user emotional data, determines the external environment type based on the external environment data, and generates corresponding vehicle body display content based on the user's emotional state and the external environment type. The determination of the user's emotional state and the determination of the external environment type can be referred to the relevant descriptions above, and will not be repeated here.

[0073] In some embodiments, the central controller 103 can generate corresponding vehicle body display content based on the user's emotional state and the type of external environment through a content generation algorithm.

[0074] In some embodiments, the content generation algorithm may include vehicle display content corresponding to different user emotional states and vehicle exterior environment types, i.e., a mapping relationship between user emotional states, vehicle exterior environment types, and vehicle display content. When a user's emotional state and vehicle exterior environment type are identified, the corresponding vehicle display content is generated through the content generation algorithm.

[0075] In some embodiments, the preset environment awareness algorithm and content generation algorithm can run on the embedded processor of the central controller 103 to ensure real-time performance and efficiency.

[0076] In some embodiments, the content generation algorithm employs a fuzzy control algorithm or a PID control algorithm to achieve dynamic optimization of content generation.

[0077] In some embodiments, the intelligent cockpit system 100 further includes a backend management system (not shown in the figure), which is remotely connected to the central controller 103. The backend management system (BMS) is used to store and manage the vehicle body display content and the system configuration of the intelligent cockpit system.

[0078] In some embodiments, the vehicle display content includes, but is not limited to, user-defined vehicle display content, advertising content uploaded by the backend administrator, and vehicle display content generated by the central controller 103 based on in-vehicle user emotional data and / or external environment data.

[0079] In some embodiments, the back-end management system can be implemented based on servers and network devices. The server is used to store and manage the vehicle display content and the system configuration of the intelligent cockpit system, and the network device is used to remotely communicate and control the central controller 103, supporting 4G / 5G communication to ensure the real-time nature of remote control.

[0080] In some embodiments, the back-end management system can provide a remote control interface, where administrators can remotely configure the content displayed on the vehicle body, support advertising, and remotely upload advertising content as the content displayed on the vehicle body.

[0081] In some embodiments, the back-end management system can provide a system configuration interface, allowing administrators to remotely configure the system parameters of the intelligent cockpit system.

[0082] This invention also provides a control method for an intelligent cockpit system. Figure 5 This is a flowchart illustrating a control method for an intelligent cockpit system provided in an embodiment of the present invention. The intelligent cockpit system includes the intelligent cockpit system described in the above embodiment, such as... Figure 5 As shown, the control methods of the intelligent cockpit system include:

[0083] Step S51: Obtain in-vehicle user emotion data and external environment data.

[0084] Step S52: Generate vehicle body display content based on in-vehicle user emotional data and / or external environment data.

[0085] Step S53: Display the vehicle body display content on the vehicle body surface.

[0086] In some embodiments, in-vehicle user emotion data includes in-vehicle user facial data and / or in-vehicle user voice data. The step of acquiring in-vehicle user emotion data and external environment data may further include: acquiring in-vehicle user facial data through an in-vehicle camera module; and acquiring user voice data through a voice recognition module.

[0087] In some embodiments, the step of generating vehicle display content based on in-vehicle user emotion data and / or external environment data may further include: performing emotion classification prediction on in-vehicle user facial data using a preset emotion perception algorithm to determine the user's facial expression; and generating vehicle display content based on the user's facial expression.

[0088] In some embodiments, the step of generating vehicle display content based on in-vehicle user emotion data and / or external environment data may further include: performing emotion classification and prediction on in-vehicle user voice data using a preset emotion perception algorithm to determine the user's voice emotion; and generating vehicle display content based on the user's voice emotion.

[0089] In some embodiments, the step of generating vehicle display content based on in-vehicle user emotion data and / or external environment data may further include: performing environmental classification prediction based on external environment data using a preset environmental perception algorithm to determine the type of external environment; and generating vehicle display content based on the type of external environment.

[0090] In this embodiment of the invention, the specific implementation of the control method for the intelligent cockpit system can be referred to the relevant description of the intelligent cockpit system in the above embodiments, and will not be repeated here.

[0091] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of the invention as set forth in the appended claims.

Claims

1. An intelligent cockpit system, characterized in that, include: The emotion perception module is used to acquire emotional data of users inside the vehicle; The environmental perception module is used to acquire data about the external environment of the vehicle; The central controller is used to receive in-vehicle user emotion data from the emotion perception module and external environment data from the environment perception module, and to generate vehicle body display content based on the in-vehicle user emotion data and / or the external environment data, and to control the vehicle body display module to display the vehicle body display content. The vehicle body display module is used to display the vehicle body display content on the vehicle body surface.

2. The intelligent cockpit system according to claim 1, characterized in that, The in-vehicle user emotion data includes in-vehicle user facial data and / or in-vehicle user voice data, and the emotion perception module includes an in-vehicle camera module and a voice recognition module; The in-vehicle camera module is used to acquire facial data of users inside the vehicle; The speech recognition module is used to acquire user voice data.

3. The intelligent cockpit system according to claim 1, characterized in that, The external environmental data includes at least one of the following: external temperature, external humidity, external light intensity, and environmental information surrounding the vehicle body; the environmental perception module includes at least one of these: Temperature sensor used to detect the outside temperature of the vehicle; A humidity sensor is used to detect the humidity outside the vehicle. A light sensor is used to detect the intensity of light outside the vehicle; The exterior camera module is used to detect information about the environment around the vehicle.

4. The intelligent cockpit system according to claim 2, characterized in that, The central controller is also used for: The facial data of the in-vehicle user is classified and predicted for emotion using a preset emotion perception algorithm to determine the user's facial expression. The vehicle display content is generated based on the user's facial expressions.

5. The intelligent cockpit system according to claim 2, characterized in that, The central controller is also used for: The in-vehicle user voice data is classified and predicted for emotion using a preset emotion perception algorithm to determine the emotion of the user's voice. Based on the user's voice emotion, generate vehicle display content.

6. The intelligent cockpit system according to claim 1, characterized in that, The central controller is also used for: The vehicle external environment data is classified and predicted by a preset environmental perception algorithm to determine the type of vehicle external environment. Based on the described external environment type, generate vehicle body display content.

7. The intelligent cockpit system according to claim 1, characterized in that, Also includes: A back-end management system, which is remotely connected to the central controller; The back-end management system is used to store and manage the vehicle display content and the system configuration of the intelligent cockpit system.

8. A control method for an intelligent cockpit system, characterized in that, The intelligent cockpit system includes the intelligent cockpit system according to any one of claims 1-7, and the control method includes: Acquire in-vehicle user emotional data and external environment data; The vehicle body display content is generated based on the in-vehicle user emotion data and / or the external environment data. The vehicle body display content is displayed on the vehicle body surface.

9. The control method according to claim 8, characterized in that, The in-vehicle user emotional data includes in-vehicle user facial data and / or in-vehicle user voice data. The acquisition of in-vehicle user emotional data and external environment data includes: Acquire facial data of users inside the vehicle using an in-vehicle camera module; User voice data is obtained through a speech recognition module.

10. The control method according to claim 9, characterized in that, The step of generating vehicle display content based on the in-vehicle user emotion data and / or the external environment data includes: The facial data of the in-vehicle user is classified and predicted for emotion using a preset emotion perception algorithm to determine the user's facial expression. Based on the user's facial expressions, generate vehicle display content; And / or, The in-vehicle user voice data is classified and predicted for emotion using a preset emotion perception algorithm to determine the emotion of the user's voice. Based on the user's voice emotion, generate vehicle display content.

11. The control method according to claim 8, characterized in that, The step of generating vehicle display content based on the in-vehicle user emotion data and / or the external environment data includes: Based on the external environment data, an environmental classification prediction is performed using a preset environmental perception algorithm to determine the type of external environment. Based on the described external environment type, generate vehicle body display content.

Images