Vehicle control method, cloud server, vehicle controller, client and vehicle

Abstract

A vehicle control method, a cloud server, a vehicle controller, a client and a vehicle. The vehicle control method for the cloud server comprises: determining a user requirement on the basis of client information; determining a target scenario of a vehicle interactive agent on the basis of the user requirement and vehicle electrical system status information; and sending information of the target scenario of the vehicle interactive agent.

Description

Vehicle control methods, cloud servers, vehicle controllers, clients, and vehicles

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese patent application No. 202410940482.2, filed on July 15, 2024, with the China National Intellectual Property Administration, entitled "Control Method for Vehicle, Cloud Server, Vehicle Controller, Client and Vehicle", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to the field of vehicle technology, and in particular to a vehicle control method, a cloud server, a vehicle controller, a client, and a vehicle. Background Technology

[0004] In related technologies, due to the dependencies between various modules in a vehicle, a failure in one module may prevent subsequent commands from being executed. Existing human-vehicle interaction technologies still suffer from inaccurate control strategies provided by the vehicle, which may reduce users' trust in new technologies and even lead to negative attitudes such as skepticism and avoidance.

[0005] Public content

[0006] This disclosure aims to address at least one of the technical problems existing in the related art. To this end, the first objective of this disclosure is to propose a vehicle control method for cloud servers, which provides users with a more accurate target scenario for the vehicle's interactive intelligent agent, satisfying human emotional needs, thereby improving user experience and increasing user stickiness.

[0007] The second objective of this disclosure is to propose a control method for vehicles.

[0008] The third objective of this disclosure is to propose a method for controlling a vehicle for a client.

[0009] The fourth objective of this disclosure is to propose a method for use with cloud servers.

[0010] The fifth objective of this disclosure is to provide a vehicle controller for vehicles.

[0011] The sixth objective of this disclosure is to propose a client application.

[0012] The seventh objective of this disclosure is to propose a vehicle.

[0013] To achieve the above objectives, the vehicle control method proposed in the first aspect of this disclosure is used in a cloud server. The control method includes: determining user needs based on client information; determining a target scenario for a vehicle interactive intelligent agent based on the user needs and vehicle electrical status information; and sending information about the target scenario of the vehicle interactive intelligent agent.

[0014] The vehicle control method for cloud servers proposed in this disclosure takes into account user needs and determines the target scenario of the vehicle interactive intelligent agent based on user needs and vehicle electrical status information. This allows for providing users with a more accurate target scenario of the vehicle interactive intelligent agent, thereby satisfying human emotional needs, improving user experience, and increasing user stickiness of the vehicle.

[0015] In some embodiments of this disclosure, determining the target scenario of the vehicle interactive intelligent agent based on the user needs and the vehicle electrical status information includes: generating text information about the user needs based on the user needs; and generating the target scenario of the vehicle interactive intelligent agent based on the text information and the vehicle electrical status information.

[0016] In some embodiments of this disclosure, after generating text information about the user's needs based on the user's needs, determining the target scenario of the vehicle interactive intelligent agent based on the user's needs and the vehicle's electrical status information further includes: obtaining driving scenario reference information, and generating the target scenario of the vehicle interactive intelligent agent based on the text information, the vehicle's electrical status information, and the driving scenario reference information, wherein the driving scenario reference information includes at least one of user status information and vehicle environment information.

[0017] In some embodiments of this disclosure, the control method further includes: acquiring feedback information after the target interactive intelligent agent in the vehicle interactive intelligent agent executes the target scenario; generating interactive information corresponding to the target interactive intelligent agent based on the feedback information; and sending the interactive information.

[0018] In some embodiments of this disclosure, generating interaction information corresponding to the target interactive agent based on the feedback information includes: generating the interaction information based on the feedback information and the stored historical information of the target interactive agent, wherein the historical information of the target interactive agent is determined based on the historical feedback information and historical state information of the target interactive agent.

[0019] The historical information of the target interactive intelligent agent can be understood as being acquired through "personality cultivation" of the target interactive intelligent agent. By analyzing the historical performance and "personality cultivation" of the vehicle's interactive intelligent agents, the various functional systems can be personified and given a sense of growth and companionship, satisfying human emotional needs, thereby improving user experience and increasing user stickiness. Furthermore, human-vehicle interaction is achieved through interaction methods that users are already accustomed to, resulting in low learning costs, ease of use, and, compared to voice assistants, more private interaction operations, enhancing the user experience.

[0020] In some embodiments of this disclosure, the feedback information includes execution status information and execution status judgment information of the target interactive intelligent agent executing the target scenario. By obtaining the execution status information and execution status judgment information of the target interactive intelligent agent executing the target scenario, it is possible to further judge the execution status of each target interactive intelligent agent, such as whether each target interactive intelligent agent is functioning normally, the execution order, and risks. If a fault exists, it also facilitates the cloud server to generate a new target scenario or remedial solution based on the fault information, thereby providing intelligent services to users and enhancing user trust in the functionality.

[0021] To achieve the above objectives, a second aspect of this disclosure also proposes a vehicle control method for a vehicle, the control method comprising: receiving information about a target scene from a vehicle interactive intelligent agent; and controlling a target interactive intelligent agent in the vehicle interactive intelligent agent to execute the target scene based on the information about the target scene.

[0022] According to the vehicle control method proposed in this disclosure, by controlling the target interactive intelligent agent in the whole vehicle interactive intelligent agent to execute the target scene, a suitable riding environment can be created for the user, satisfying human emotional needs, thereby improving user experience, increasing user stickiness of the vehicle, and enhancing the intelligence of the vehicle.

[0023] In some embodiments of this disclosure, before receiving information about the target scene from the vehicle interactive intelligent agent, the control method further includes: sending vehicle electrical status information.

[0024] In some embodiments of this disclosure, before receiving information about the target scenario from the vehicle interactive intelligent agent, the control method further includes: sending driving scenario reference information, wherein the driving scenario reference information includes at least one of user state information and vehicle environment information.

[0025] In some embodiments of this disclosure, the control method further includes: sending feedback information after the target interactive agent executes the target scene.

[0026] To achieve the above objectives, a third aspect of this disclosure also proposes a vehicle control method for a client, the control method comprising: sending client information; and receiving interaction information of a target interactive intelligent agent in a vehicle interactive intelligent agent and displaying the interaction information.

[0027] According to the vehicle control method for the client proposed in the embodiments of this disclosure, the client provides a natural, interesting, and anthropomorphic interaction method that can intuitively visualize the working status of the vehicle equipment. This method is more engaging and not only shows the relationship between the various devices, but also allows users to easily view the historical or real-time execution status of the vehicle equipment, such as whether each target interactive agent is normal, the execution order, and risks. The visibility of the functional status of each target device can enhance the user's trust in the function and also simplify the user's memory burden of complex interactive commands.

[0028] To achieve the above objectives, a fourth aspect of this disclosure also proposes a cloud server, including a multimodal encoder for determining user needs in response to client information; and a first controller connected to the multimodal encoder to implement the vehicle control method for the cloud server described in the above embodiments.

[0029] According to the cloud server proposed in this embodiment, through the fusion of a multimodal encoder and a first controller, client information input in formats such as natural language text, voice, images, and videos can be converted into vehicle-readable and executable instructions, thereby controlling the vehicle's interactive intelligent agent to execute the target scenario of the above embodiment, satisfying human emotional needs, thereby improving user experience and increasing vehicle user stickiness.

[0030] To achieve the above objectives, a fifth aspect of this disclosure also provides a vehicle controller, comprising: at least one processor and a memory, the memory being connected to at least one of the processors; the memory storing a computer program, which, when executed by at least one of the processors, implements the vehicle control method described in the above embodiments.

[0031] According to the vehicle controller proposed in the embodiments of this disclosure, a computer program stored in the memory implements the vehicle control method of the above embodiments when executed by at least one processor. This enables the target interactive intelligent agent to execute the target scenario, thereby creating a suitable riding environment for the user, satisfying human emotional needs, improving user experience, and increasing user stickiness of the vehicle.

[0032] To achieve the above objectives, a fifth aspect of this disclosure also proposes a client, which includes an information interaction platform for sending client information and / or receiving and displaying interaction information from a target interactive intelligent agent in a vehicle interactive intelligent agent.

[0033] According to the client proposed in the embodiments of this disclosure, by adopting a unified information interaction platform, it can provide users with a natural, interesting, and human-like interaction method that can intuitively visualize the working status of the entire vehicle equipment. It is more interesting and can not only show the relationship between the various devices, but also make it convenient for users to view the historical or real-time execution status of the entire vehicle equipment, such as whether each target interactive intelligent agent is normal, the execution order and risks, etc. The visibility of the functional status of each target device can enhance users' trust in the function and also simplify the user's memory burden of complex interaction commands.

[0034] In some embodiments of this disclosure, the client includes at least one of a mobile terminal or an in-vehicle interactive display terminal.

[0035] To achieve the above objectives, a seventh aspect of this disclosure also provides a vehicle, including a vehicle interaction agent, the vehicle interaction agent including at least vehicle electrical components; and a vehicle controller as described in the fifth aspect of the above, the vehicle controller being connected to the vehicle interaction agent.

[0036] The vehicle proposed according to the embodiments of this disclosure can take user needs into account in real time. When the vehicle controller controls the vehicle interactive intelligent agent to execute the target scenario, it can provide users with a more accurate and reasonable vehicle atmosphere, meet people's emotional needs, promote the intelligent development of the vehicle, thereby improving the user's driving experience and increasing the user stickiness of the vehicle.

[0037] In some embodiments of this disclosure, the vehicle further includes an in-vehicle interactive display terminal for sending client information.

[0038] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0039] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0040] Figure 1 is a block diagram of a cloud server according to an embodiment of the present disclosure;

[0041] Figure 2 is a flowchart of a vehicle control method for a cloud server according to an embodiment of the present disclosure;

[0042] Figure 3 is a flowchart of a vehicle control method for a cloud server according to another embodiment of the present disclosure;

[0043] Figure 4 is a flowchart of a vehicle control method for a cloud server according to yet another embodiment of the present disclosure;

[0044] Figure 5 is a schematic diagram of an information interaction platform in a client according to an embodiment of the present disclosure;

[0045] Figure 6 is a block diagram of a vehicle controller according to an embodiment of the present disclosure;

[0046] Figure 7 is a flowchart of a vehicle control method for a vehicle controller according to an embodiment of the present disclosure;

[0047] Figure 8 is a block diagram of a client according to an embodiment of the present disclosure;

[0048] Figure 9 is a flowchart of a vehicle control method for a client according to an embodiment of the present disclosure;

[0049] Figure 10 is a block diagram of a vehicle according to an embodiment of the present disclosure;

[0050] Figure 11 is a block diagram of a vehicle interactive intelligent agent according to an embodiment of the present disclosure;

[0051] Figure 12 is a schematic diagram of an interaction scheme between a client, a cloud server, and a vehicle according to an embodiment of the present disclosure;

[0052] Figure 13 is a schematic diagram of an interaction scheme between a client, a cloud server, and a vehicle according to another embodiment of the present disclosure;

[0053] Figure 14 is a schematic diagram of an interaction scheme between a client, a cloud server, and a vehicle according to yet another embodiment of the present disclosure.

[0054] Reference numerals: Cloud server 10; Vehicle 1; Client 1000; Vehicle controller 100; Vehicle interactive intelligent agent 200; Vehicle electrical system 300; Including mobile terminal 1001; In-vehicle interactive display terminal 1002; Processor 101; Memory 102; Multimodal encoder 11; First controller 2. Detailed Implementation

[0055] The embodiments of this disclosure are described in detail below, and the embodiments described with reference to the accompanying drawings are exemplary.

[0056] The first aspect of this disclosure provides a vehicle control method using a cloud server. The cloud server 10 of this disclosure embodiment can be understood in conjunction with FIG1. ​​FIG1 is a block diagram of a cloud server 10 according to an embodiment of this disclosure, wherein the cloud server 10 can be understood as a cloud-based AI large model, which has data storage, data processing, and analysis functions, and can convert inputs in formats such as natural language text, voice, images, and video into instructions that can be read and executed by the vehicle's interactive intelligent agent.

[0057] It can be understood that the vehicle interactive intelligent agent is an intelligent agent endowed with emotions and interaction. It can include hardware interactive devices or software interactive agents. Hardware interactive devices can include at least one of the following: vehicle electrical appliances such as air conditioning, cabin ambient lighting, seats, in-vehicle fragrance, windows and in-vehicle multimedia. Software interactive agents can include virtual humans, virtual pets, etc. This is just an example and no specific limitation is made.

[0058] The cloud server 10 includes a multimodal encoder 11 and a first controller 2. The multimodal encoder 11 responds to client information to determine user needs. In some embodiments, the multimodal encoder 11 has the ability to understand text, voice, images, and video content, and can recognize and understand information such as text, voice, images, and video, re-encoding the client information into text information. This text information is actually text information about user needs. The target scenario of the vehicle's interactive intelligent agent is determined based on user needs and the vehicle's electrical status information.

[0059] The first controller 2 is connected to the multimodal encoder 11. The first controller 2 can be a controller set in the cloud server 10 for controlling the vehicle interactive intelligent agent. It has a large model of the vehicle's electrical system inside, which can be used to process the text information into instructions that the vehicle controller can read and execute. Thus, the large model of the vehicle's electrical system can generate the target scene of the vehicle interactive intelligent agent based on the text information and the vehicle's electrical system status information.

[0060] In some embodiments, the vehicle electrical status information includes, but is not limited to, the status information of the air conditioner, cabin ambient lighting, seats, in-vehicle fragrance, windows, in-vehicle multimedia, virtual people and virtual pets, etc.

[0061] In some embodiments, the multimodal encoder 11 can be a Transformer encoder. The overall vehicle electrical system model is obtained by pre-training a general-purpose model and fine-tuning it. The general-purpose model can be chatGLM or LLaMA, etc., and the fine-tuning algorithm can be a Low Rank Adapter, etc. The database used for fine-tuning includes the design of the target scene and the corresponding control commands of the vehicle's interactive intelligent agent. It is understood that the general-purpose model and the fine-tuning algorithm can be selected as needed, and no specific restrictions are imposed here.

[0062] The cloud server 10 proposed in this embodiment includes a multimodal encoder 11 and a first controller 2. This means that the multimodal encoder 11 is fused with a large-scale vehicle electrical model, enabling the conversion of inputs in formats such as natural language text, voice, images, and video into instructions readable and executable by the vehicle's interactive intelligent agent. Therefore, the cloud server 10, through a unified cloud-based AI model, can be used to realize interaction between natural input and the vehicle's interactive intelligent agent.

[0063] Based on the functions of the cloud server 10 described above, the following describes a vehicle control method for a cloud server according to an embodiment of the present disclosure, with reference to Figures 2-5.

[0064] Figure 2 is a flowchart of a vehicle control method for a cloud server according to an embodiment of the present disclosure. The control method includes at least steps S1-S3, as detailed below.

[0065] S1 determines user needs based on client information.

[0066] The client information can include at least one of the following formats: voice, image, text, emoticons, and video. The cloud server has the ability to understand this information and thus obtain user needs. For example, if the cloud server receives a voice message from a customer saying "It's too hot," the cloud server can understand and analyze this voice message to determine that the user's need is to lower the cabin temperature.

[0067] The following steps S2 and S3 are described using the user's need to reduce cabin temperature as an example.

[0068] S2 determines the target scenario for the vehicle's interactive intelligent agent based on user needs and the vehicle's electrical status information.

[0069] In some embodiments, the multimodal encoder 11 may first generate text information about user needs based on user requirements. In some embodiments, the multimodal encoder may uniformly encode the multimodal input, i.e., the user needs, into text describing the design of the user needs. For example, if the user need to reduce the cabin temperature is encoded as "cooling down", and it can be confirmed that the user currently wants to stay in the vehicle, then the generated text information could be "reduce cabin temperature".

[0070] In some embodiments, the first controller 2 generates the target scene of the vehicle interactive intelligent agent based on the text information and the vehicle electrical status information.

[0071] The vehicle-wide interactive intelligent agent can include at least one of the following: air conditioning, ambient lighting, seats, in-vehicle fragrance, windows, in-vehicle multimedia, virtual humans, and virtual pets. The vehicle's electrical status information includes, but is not limited to, the status information of the air conditioning, ambient lighting, seats, in-vehicle fragrance, windows, in-vehicle multimedia, virtual humans, and virtual pets. In some embodiments, a target interactive intelligent agent can be selected based on text information. For example, when the text information is "reduce cabin temperature," the first controller 2 can select windows and the in-vehicle air conditioning as target interactive intelligent agents. For instance, if the current window is open and the ambient temperature is relatively high, then the execution target scenario for the vehicle-wide interactive intelligent agent can be generated as a scenario where the windows are closed, the air conditioning is on, and it operates in a cooling mode at a preset temperature.

[0072] S3 sends information about the target scenario to the vehicle's interactive intelligent agent.

[0073] The target scene information of the vehicle interactive intelligent agent includes the target scene of the vehicle interactive intelligent agent and the execution plan of the vehicle interactive intelligent agent to execute the target scene generated based on the target scene. The execution plan is the instruction that the vehicle controller can read and execute, that is, sending control instructions containing the target scene of the vehicle interactive intelligent agent and the corresponding execution plan. In some embodiments, the form of the control instruction can be text containing target actions or target states related to the vehicle interactive intelligent agent, such as sending text containing commands to close the windows, turn on the air conditioner and run it in cooling mode at a preset temperature.

[0074] The vehicle control method for cloud servers proposed in this disclosure takes into account user needs and determines the target scenario of the vehicle interactive intelligent agent based on user needs and vehicle electrical status information. This allows the system to provide users with a more accurate target scenario for the vehicle interactive intelligent agent, satisfying human emotional needs, thereby improving user experience and increasing user stickiness of the vehicle.

[0075] In some other embodiments of this disclosure, for step S2 above, after generating text information about user needs based on user needs, determining the target scenario of the vehicle interactive intelligent agent based on user needs and vehicle electrical status information, further includes: obtaining driving scenario reference information, and generating the target scenario of the vehicle interactive intelligent agent based on the text information, vehicle electrical status information and driving scenario reference information, wherein the driving scenario reference information includes at least one of user status information and vehicle environment information.

[0076] This includes, but is not limited to, user status information and user physiological status information. It also includes, but is not limited to, vehicle environmental information such as ambient temperature, humidity, and light intensity.

[0077] According to the vehicle control method for cloud servers proposed in this disclosure, the target scenario of the vehicle interactive intelligent agent is determined based on user needs and vehicle electrical status information, thereby providing users with a more accurate target scenario of the vehicle interactive intelligent agent. Furthermore, in addition to considering user needs, user status information and vehicle environmental information are also taken into account, which can fully consider the user's emotional and psychological needs as well as the vehicle's surrounding environment, thereby satisfying human emotional needs, improving user experience, and increasing user stickiness of the vehicle.

[0078] In some embodiments of this disclosure, as shown in FIG3, a flowchart of a vehicle control method for a cloud server according to another embodiment of this disclosure is presented. The control method further includes steps S4-S6, as detailed below.

[0079] S4: Obtain feedback information from the target interactive agent in the whole vehicle interactive agent after executing the target scenario.

[0080] In some embodiments, the feedback information includes execution status information and execution status judgment information of the target interactive agent executing the target scenario. It is understood that this feedback information is provided by the vehicle controller in the vehicle. After the target interactive agent in the vehicle interactive agent executes the execution scheme of the target scenario proposed in the above embodiments, the vehicle controller obtains the execution data of each target interactive agent, i.e., the execution status information, and judges each execution data to obtain the corresponding execution status judgment information. Then, it sends the execution status information and execution status judgment information of the target interactive agent executing the target scenario to the cloud server.

[0081] S5 generates interaction information for the corresponding target interactive agent based on the feedback information.

[0082] S6, send interactive information.

[0083] In some embodiments, generating interaction information for the corresponding target interactive agent based on feedback information may include generating interaction information based on the feedback information and stored historical information of the target interactive agent, wherein the historical information of the target interactive agent is determined based on the historical feedback information and historical state information of the target interactive agent. The interaction information is not limited to interactive text content, but also includes actions to be performed by various interactive agents, voice dialogues, and other content.

[0084] In some embodiments, users can "cultivate the personality" of target interactive agents within the interactive information processing system. This involves setting the historical information of the target interactive agents, i.e., setting the "personality" of each vehicle interactive agent. After receiving feedback information, the first controller sends it to the multimodal encoder. The multimodal encoder combines the "personality" settings and historical behaviors of each target interactive agent to generate interactive information for the current round corresponding to the feedback information, along with corresponding display methods such as voice, text, and emoticons, making the interactive information more engaging. This interactive information is then sent to the client for display. The interactive information is not limited to interactive text content but also includes actions and voice dialogues required by various interactive agents. Furthermore, each interactive agent can be configured with a unique voice. Users can directly understand the operational status of each target interactive agent based on the interactive information displayed on the client.

[0085] The vehicle control method for a cloud controller proposed in this disclosure improves the user experience by customizing the vehicle's "software" style. The output interactive information employs a fun, friendly, and personalized interaction method, satisfying human emotional needs and establishing an emotional connection between the user and the vehicle, thereby enhancing user experience and increasing user stickiness. Furthermore, human-vehicle interaction is achieved through user-familiar interaction methods, resulting in low learning costs, ease of use, and, compared to voice assistants, more private interaction, further improving the user experience.

[0086] In some embodiments, a flowchart of a vehicle control method for a cloud server according to an embodiment of the present disclosure can be understood in conjunction with Figures 4 and 5.

[0087] Figure 4 is a flowchart of a vehicle control method for a cloud server according to another embodiment of the present disclosure; Figure 5 is a schematic diagram of an information interaction platform in a client according to an embodiment of the present disclosure.

[0088] As shown in Figure 4, the control method includes at least steps S101-S106, as detailed below.

[0089] S101, the vehicle interaction intelligent agent continuously monitors the vehicle's electrical status information and sends it to the cloud server. Additionally, the vehicle interaction intelligent agent can continuously monitor driving scenario reference information and send it to the cloud server. This driving scenario reference information includes at least one of user status information and vehicle environment information. All three information—vehicle electrical status information, user status information, and vehicle environment information—can be sent to the cloud server wirelessly in JSON format, ensuring that the data recorded by the cloud server remains consistent with the data of the vehicle interaction intelligent agent.

[0090] S102, Receive client information and determine user needs based on the client information. In some embodiments, the user issues instructions on the information interaction platform (i.e., instant messaging software) in the client, such as a mobile terminal and / or in-vehicle interactive display terminal, for example, by showing a forest image and saying, "I'm so tired, I want to experience this feeling in the car." The client then sends the instructions to the cloud server via wireless communication.

[0091] S103, the multimodal encoder generates text information about user needs based on user requirements.

[0092] In some embodiments, the multimodal encoder uniformly encodes the multimodal input into text describing the target scene design, such as encoding the image as a forest, and confirming through text that the user currently wants to stay in the car, creating an atmosphere where there is no need to drive and relaxation is required; the atmosphere should be close to a forest, providing forest sounds, lighting effects, tree smells, a suitable temperature and isolating outside sounds, generating the design text "Set up an in-car forest scene".

[0093] S104, the first controller generates the target scene for the vehicle interactive intelligent agent based on text information and vehicle electrical status information. Furthermore, the first controller can also generate the target scene for the vehicle interactive intelligent agent based on text information, vehicle electrical status information, and driving scenario reference information. The driving scenario reference information includes at least one of user status information and vehicle environment information.

[0094] For example, based on user needs, in-vehicle temperature, and time, the target scenario for the vehicle's interactive intelligent agent is generated in JSON format: {"seat backrest": "0%"}{"interior lights": "2"}{"music": "search forest playlist"}{"windows": "100%"}{"fragrance": "1"}. Here, "1", "2", "0%", and "100%" can all be used to represent the degree of activation or operating level of some of the vehicle's interactive intelligent agents; no specific limitations are imposed here.

[0095] S105, the first controller sends information about the target scene of the vehicle's interactive intelligent agent.

[0096] In some embodiments, the target scene information of the vehicle interactive intelligent agent includes the target scene of the vehicle interactive intelligent agent and the execution scheme of the vehicle interactive intelligent agent based on the target scene. The first controller sends the generated execution scheme to the vehicle controller in the vehicle in the form of control commands via a wireless network. The vehicle controller can send commands to the in-vehicle multimedia via IPC cross-process communication, such as playing forest white noise music; and can also send commands to other vehicle interactive intelligent agents via CAN, CANFD, and ETH. For example, if the seat receives a command, it adjusts to a rest mode; the lights reduce brightness, turn on ambient lighting, close the windows, and set up a pine wood fragrance, and the execution status of the interactive intelligent agent is fed back to the first controller.

[0097] S106, the multimodal encoder receives feedback information from the first controller regarding the target interactive agent's execution of the target scenario within the vehicle's interactive intelligent agent. Based on this feedback information, it generates corresponding interaction information for the target interactive intelligent agent and sends it to the client. In some embodiments, the multimodal encoder combines the "personality" settings and historical behaviors of each vehicle interactive intelligent agent to generate feedback information for this round of interaction and corresponding display methods such as voice, text, and emoticons. This information is then displayed on an information interaction platform, such as a mobile terminal and / or an in-vehicle interactive display terminal, ultimately achieving the situation shown in Figure 5. Figure 5 is a schematic diagram of an information interaction platform in a client according to an embodiment of this disclosure.

[0098] Furthermore, when integrating the execution status of each target interactive intelligent agent, the vehicle controller further determines whether there is a fault and feeds the result back to the cloud server. For example, as shown in Figure 5, if the vehicle controller determines that the in-vehicle multimedia system failed to find the forest background sound due to network disconnection or permission issues, the cloud server will again control the audio equipment in the vehicle to emit the sound of flowing water according to the control method used in the above embodiment, and control the ambient lighting to add default dynamic effects, while other target interactive intelligent agents execute normally according to the original instructions.

[0099] As shown in Figure 5, the "speaker" sends a message saying, "I wanted to play a forest birdsong sound, but the guy responsible for the network seems to be offline, so I had to find a flowing water sound myself... I hope you like it." This embodiment of the disclosure provides a natural, fun, and anthropomorphic interaction method that intuitively visualizes the working status of all vehicle equipment. The interaction language between the platform and the user is simple and natural, allowing users to easily view the historical or real-time execution status of all vehicle equipment. The visibility of the functional status of each target device enhances user trust in the functionality. Furthermore, as shown in Figure 5, the vehicle control method of this embodiment anthropomorphizes individual vehicle equipment rather than the entire vehicle, making it more engaging; it also demonstrates the relationships between devices, providing more interactive information and reflecting the operating status of each device. In addition, children can learn about the vehicle's working status through conversation, which has a certain educational value.

[0100] Furthermore, as shown in Figure 5, in the embodiments of this disclosure, the intelligent control of generating the interaction information of the corresponding target interactive intelligent agent based on the feedback information is uniformly carried out through the cloud server, and does not require each target device to be equipped with independent interaction functions, which facilitates the intelligent and anthropomorphic interaction of various new functions.

[0101] In some embodiments of this disclosure, a control method for a vehicle is also proposed, wherein the executor of the control method is a vehicle controller and a vehicle interaction agent in the vehicle, the vehicle controller is connected to the vehicle interaction agent, and the vehicle controller is used to perform signal acquisition, processing, transmission and control functions.

[0102] The vehicle controller of this disclosure embodiment can be understood in conjunction with FIG6. FIG6 is a block diagram of a vehicle controller 100 according to an embodiment of this disclosure. The vehicle controller 100 is disposed in a vehicle and can be a BCM (body control module) in the vehicle, which is an electronic control unit used to control the body interaction intelligent system, that is, the whole vehicle interaction intelligent system.

[0103] In some embodiments, the vehicle controller 100 includes at least one processor 101 and a memory 102 connected to at least one processor 101. The memory 102 stores a computer program that, when executed by at least one processor 101, implements a vehicle control method for the vehicle controller 100.

[0104] In some embodiments, as shown in FIG7, it is a flowchart of a vehicle control method for a vehicle controller according to an embodiment of the present disclosure. The control method includes at least steps S10 and S20, as detailed below.

[0105] S10 receives information about the target scenario from the vehicle's interactive intelligent agent.

[0106] The vehicle controller can receive target actions or target status texts related to the vehicle interactive intelligent agent output by the cloud server via a wireless network. The wireless communication method can be HTTP, MQTT, Bluetooth, etc.

[0107] S20 controls the target interactive intelligent agent in the whole vehicle interactive intelligent agent to execute the target scenario based on the information of the target scenario.

[0108] Specifically, the vehicle controller converts information from the target scene, such as target actions or target status text, from the cloud server into execution control commands for the vehicle's interactive intelligent agent. These commands are then sent to the vehicle's interactive intelligent agent (excluding the in-vehicle multimedia system) via CAN, CANFD, LIN, or Ethernet. Furthermore, the vehicle controller can also send commands to the in-vehicle multimedia system via IPC cross-process communication.

[0109] According to the vehicle control method proposed in this disclosure, by controlling the target interactive intelligent agent in the whole vehicle interactive intelligent agent to execute the target scene, a suitable riding environment can be created for the user, satisfying human emotional needs, thereby improving user experience, increasing user stickiness of the vehicle, and enhancing the intelligence of the vehicle.

[0110] Furthermore, according to the vehicle controller 100 proposed in the embodiments of this disclosure, the memory 102 stores a computer program, which, when executed by at least one processor 101, implements the vehicle control method of the embodiments, enabling the target interactive intelligent agent to execute the target scenario, thereby creating a suitable riding environment for the user, satisfying human emotional needs, thereby improving user experience, increasing user stickiness of the vehicle, and enhancing the intelligence of the vehicle.

[0111] In some embodiments, as shown in FIG7, before receiving information about the target scene from the vehicle interactive intelligent agent, the vehicle control method for the vehicle controller in this embodiment of the present disclosure further includes step S30, as follows.

[0112] S30 sends the vehicle's electrical status information.

[0113] The vehicle electrical status information includes, but is not limited to, the status information of air conditioning, cabin ambient lighting, seats, in-vehicle fragrance, windows, in-vehicle multimedia, virtual people and virtual pets, etc.

[0114] In some other embodiments of this disclosure, before receiving the target scene information from the vehicle interaction agent, the control method further includes: sending driving scene reference information. The driving scene reference information includes at least one of user state information and vehicle environment information. The user state information includes, but is not limited to, user emotional information and user physiological state information. The vehicle environment information includes, but is not limited to, ambient temperature, humidity, and light intensity. The vehicle interaction agent can continuously detect its own state and driving scene reference information and upload them to the vehicle controller. The vehicle controller sends the state change information of each vehicle interaction agent and / or environmental state to the cloud server in JSON format via wireless communication, ensuring that the data recorded in the cloud server remains consistent with the data of the vehicle interaction agents.

[0115] In some other embodiments, as shown in FIG7, the vehicle control method for a vehicle controller according to the present disclosure further includes step S40, as follows.

[0116] S40 sends feedback information after the target interactive agent executes the target scenario.

[0117] The feedback information includes execution status information and execution status judgment information of the target interactive intelligent agent in executing the target scenario. After the target interactive intelligent agent in the whole vehicle interactive intelligent agent executes the execution scheme of the target scenario proposed in the above embodiment, the vehicle controller obtains the execution data of each target interactive intelligent agent, that is, the execution status information. When integrating the execution status of each target interactive intelligent agent, the vehicle controller further judges whether there is a fault, and then sends all the execution status information and execution status judgment information of the target interactive intelligent agent in executing the target scenario to the cloud server.

[0118] According to the vehicle control method for vehicle controller proposed in the embodiments of this disclosure, by obtaining the execution status information and execution status judgment information of the target interactive intelligent agent executing the target scenario, it is possible to further judge the execution status of the whole vehicle equipment, such as whether each target interactive intelligent agent is normal, execution order, risks, etc. If there is a fault, it is also convenient for the cloud server to generate a new target scenario or remedial solution based on the fault information, thereby providing intelligent services to users and enhancing users' trust in the function.

[0119] In some embodiments of this disclosure, a vehicle control method for a client is also proposed, wherein the executor of the control method is the client. The client 1000 of this disclosure embodiment can be understood in conjunction with Figures 5 and 8. Figure 8 is a block diagram of a client 1000 according to an embodiment of this disclosure, wherein the client 1000 includes at least one of a mobile terminal 1001 or an in-vehicle interactive display terminal 1002. In some embodiments, the mobile terminal 1001 includes mobile devices such as mobile phones and tablets. The in-vehicle interactive display terminal 1002 includes an in-vehicle central control screen, accessory screens, and rear-seat screens, etc.

[0120] The client 1000 is equipped with an information interaction platform as shown in Figure 5. The information interaction platform is used to send client information and / or receive and display the interaction information of the target interactive intelligent agent in the whole vehicle interactive intelligent agent.

[0121] Understandably, this information interaction platform is actually a unified instant messaging platform, similar to chat tools such as WeChat and QQ. It runs on mobile terminals and / or in-vehicle interactive display terminals, and can issue commands and receive system feedback in the form of sending and receiving chat information such as text, images, and voice.

[0122] According to the client 1000 proposed in this embodiment, by adopting a unified information interaction platform, it can provide users with a natural, interesting, human-like and intuitive way to visualize the working status of the whole vehicle equipment. It is more interesting and can not only show the relationship between the various devices, but also make it convenient for users to view the historical or real-time execution status of the whole vehicle equipment, such as whether each target interactive intelligent agent is normal, the execution order and risks, etc. The visibility of the functional status of each target device can enhance the user's trust in the function and simplify the user's memory burden of complex interactive commands.

[0123] In some embodiments of this disclosure, as shown in FIG9, it is a flowchart of a vehicle control method for a client according to an embodiment of this disclosure. The control method includes at least steps S100 and S200, as detailed below.

[0124] S100, send client information.

[0125] The client information may include at least one of the following formats: voice, image, text, emoticons, and video.

[0126] S200 receives and displays the interaction information of the target interactive agent in the whole vehicle interactive agent.

[0127] For example, as shown in Figure 5, "Little Smart" sends the message "Thank you for your hard work! I'll be right there for everyone"; "Light Bulb" sends the message "Want to take a break? I've turned off all the bright lights"; "Guard" sends the message "The car doors and windows are closed, so don't worry about being disturbed"; "Little Fragrance" sends the message "I think there's a faint fragrance in the forest, like this"; "Speaker" sends the message "I wanted to play a forest birdsong sound, but the guy in charge of the network seems to be offline, so I had to find a flowing water sound myself... I hope you like it," and so on.

[0128] For example, interactive intelligent agents such as virtual humans and virtual pets can use actions and voice dialogues to display interactive information, and each interactive intelligent agent can also be configured to have a unique voice. No specific limitations are set here.

[0129] According to the vehicle control method for the client proposed in the embodiments of this disclosure, the client provides a natural, interesting, and human-like interactive method that can intuitively visualize the working status of the vehicle equipment, making it more engaging. It can not only show the relationship between the various devices, but also make it convenient for users to view the historical or real-time execution status of the vehicle equipment, such as whether each target interactive intelligent agent is normal, the execution order, and risks. The visibility of the functional status of each target device can enhance the user's trust in the function.

[0130] In addition, children can learn about the vehicle's working status through interactive methods, which has a certain educational value.

[0131] In some embodiments of this disclosure, as shown in FIG10, a block diagram of a vehicle according to an embodiment of this disclosure is provided, wherein the vehicle 1 includes a vehicle interaction agent 200 and a vehicle controller 100 of the above embodiment, and the vehicle controller 100 is connected to the vehicle interaction agent 200.

[0132] As shown in Figure 11, the vehicle interactive intelligent agent 200 includes at least vehicle electrical components 300. It can be understood that the vehicle interactive intelligent agent 200 is an intelligent agent endowed with emotions and interaction capabilities. It can include hardware interactive devices or software interactive agents, where hardware interactive devices can include at least one of the vehicle electrical components 300, such as air conditioning, ambient lighting, seats, in-vehicle fragrance, windows, and in-vehicle multimedia systems. Software interactive agents can include virtual humans and virtual pets, etc., which are only examples here and are not specifically limited.

[0133] As shown in Figure 10, vehicle 1 also includes an in-vehicle interactive display terminal 1002, which includes an in-vehicle central control screen, accessory screens and rear screens, etc., for sending client information.

[0134] According to the vehicle 1 proposed in this embodiment, it can consider user needs in real time. When the vehicle controller 100 controls the vehicle interactive intelligent agent 200 to execute the target scenario, it can provide users with a more accurate and reasonable vehicle atmosphere, meet human emotional needs, promote the intelligent development of the vehicle, thereby improving the user's driving experience and increasing user stickiness.

[0135] In some embodiments of this disclosure, the interaction between the client 1000, cloud server 10, and vehicle 1 can be understood in conjunction with Figures 12-14. Figure 12 is a schematic diagram of the interaction scheme between the client 1000, cloud server 10, and vehicle 1 according to one embodiment of this disclosure; Figure 13 is a schematic diagram of the interaction scheme between the client 1000, cloud server 10, and vehicle 1 according to another embodiment of this disclosure; and Figure 14 is a schematic diagram of the interaction scheme between the client 1000, cloud server 10, and vehicle 1 according to yet another embodiment of this disclosure.

[0136] As shown in Figure 12, the client 1000 and the cloud server 10 exchange information, and the cloud server 10 and the vehicle 1 exchange information.

[0137] In some embodiments, as shown in FIG13, the cloud server 10 includes a multimodal encoder 11 and a first controller 2. Client information may include information such as voice, images, text, video, and emoticons. The client information sent by the client is acquired by the multimodal encoder 11, which re-encodes the client information into text information and sends it to the first controller 2. The first controller 2 sends the target scene information of the generated vehicle interactive intelligent agent to the vehicle controller 100 in JSON format via wireless communication. In addition, the vehicle controller 100 also sends the status information of the vehicle electrical system 300, vehicle environment information, and feedback information after the target interactive intelligent agent executes the target scene to the first controller 2 in the cloud server 10 in JSON format via wireless communication.

[0138] As shown in Figure 14, the client 1000 includes an in-vehicle interactive display terminal 1002 and a mobile terminal 1001. The vehicle interactive intelligent agent 200 may include air conditioning, cabin ambient lighting, seats, in-vehicle fragrance, windows, virtual people and virtual pets, etc. It can be understood that in practice, both the in-vehicle multimedia and the in-vehicle interactive display terminal 1002 can be the in-vehicle central control screen, accessory screens and rear screens. In addition to the display screen, the in-vehicle interactive display terminal 1002 may also include holographic display, 3D display and virtual display, etc. This is only an example and no specific limitation is made.

[0139] The in-vehicle multimedia and the in-vehicle interactive display terminal 1002 perform different functions. The in-vehicle interactive display terminal 1002 is mainly used to realize intelligent human-vehicle interaction, while the in-vehicle multimedia is used to play audio information, etc. Therefore, they are shown separately in Figure 14. The cloud server 10 interacts with the client 1000 and the vehicle controller 100 through MQTT, which is a wireless communication method. The vehicle controller 100 interacts with the in-vehicle multimedia through IPC cross-process communication, such as playing forest white noise music. The vehicle controller 100 can also interact with other vehicle interactive intelligent agents through CAN, CANFD, and ETH.

[0140] Based on the above, this embodiment of the disclosure, based on the architecture between client 1000, cloud server 10, and vehicle 01, provides a natural, engaging, and intuitively visual interaction method for monitoring the working status of the vehicle's equipment. The information interaction platform in the client has the function of issuing commands and receiving system feedback in the form of text, images, and voice. After issuing commands through the client, users can conveniently view the historical or real-time execution status of the vehicle's equipment in response to the execution strategy through the information interaction platform in the client, including but not limited to whether it is normal, the execution order, and risks. This intelligent and human-like interaction method simplifies the user's memory burden of complex interactive commands, increases engagement, and the visibility of the execution status enhances the user's trust and positive perception of the function, thereby increasing user stickiness.

[0141] In addition, children can learn about the vehicle's working status through interactive methods, which has a certain educational value.

[0142] Other configurations and operations of the client 1000, cloud server 10, and vehicle 1 according to embodiments of this disclosure are known to those skilled in the art and will not be described in detail here.

[0143] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example.

[0144] Although embodiments of this disclosure have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this disclosure, the scope of which is defined by the claims and their equivalents.

Claims

1. A control method of a vehicle for a cloud server, wherein, The control method comprises: determining user demand according to client information; determining a target scene of a vehicle interactive agent according to the user demand and vehicle electrical appliance state information; and sending information of the target scene of the vehicle interactive agent.

2. The control method according to claim 1, wherein Determining a target scene of a vehicle interactive agent according to the user demand and vehicle electrical appliance state information comprises: generating text information about the user demand according to the user demand; and generating the target scene of the vehicle interactive agent according to the text information and the vehicle electrical appliance state information.

3. The control method according to claim 2, wherein After the generating text information about the user demand according to the user demand, determining a target scene of a vehicle interactive agent according to the user demand and vehicle electrical appliance state information further comprises: obtaining driving scene reference information, and generating the target scene of the vehicle interactive agent according to the text information, the vehicle electrical appliance state information and the driving scene reference information, wherein the driving scene reference information comprises at least one of user state information and vehicle environment information.

4. The control method according to any one of claims 1-3, wherein, The control method further comprises: obtaining feedback information after a target interactive agent in the vehicle interactive agent executes the target scene; generating interactive information corresponding to the target interactive agent according to the feedback information; and sending the interactive information.

5. The control method according to claim 4, wherein Generating interactive information corresponding to the target interactive agent according to the feedback information comprises: generating the interactive information according to the feedback information and stored historical information of the target interactive agent, wherein the historical information of the target interactive agent is determined according to historical feedback information and historical state information of the target interactive agent.

6. The control method according to claim 4 or 5, wherein The feedback information comprises execution condition information and execution condition judgment information of the target interactive agent executing the target scene.

7. A control method of a vehicle in which For a vehicle, the control method comprises: receiving information of a target scene of a vehicle interactive agent; and controlling a target interactive agent in the vehicle interactive agent to execute the target scene according to the information of the target scene.

8. The control method according to claim 7, wherein Before the receiving information of a target scene of a vehicle interactive agent, the control method further comprises: sending vehicle electrical appliance state information.

9. The control method according to claim 7 or 8, wherein Before the receiving information of a target scene of a vehicle interactive agent, the control method further comprises: sending driving scene reference information, wherein the driving scene reference information comprises at least one of user state information and vehicle environment information.

10. The control method according to any one of claims 7-9, wherein, Further comprising: sending feedback information after the target interactive agent executes the target scene.

11. A control method of a vehicle in which For a client, the control method comprises: sending client information; and receiving interactive information of a target interactive agent in a vehicle interactive agent and displaying the interactive information.

12. A cloud server (10), wherein Comprise: a multi-modal encoder (11) for determining user demand in response to client (1000) information; and a first controller (2) connected with the multi-modal encoder (11) to implement the control method of the vehicle (1) according to any one of claims 1-6.

13. A vehicle controller (100), wherein Comprise: at least one processor (101); and a memory (102) connected with the at least one processor (101); the memory (102) stores a computer program, and the computer program is executed by the at least one processor (101) to implement the control method of the vehicle (1) according to any one of claims 7-10.

14. A client (1000), wherein The client (1000) comprises an information interaction platform, and the information interaction platform is used for sending client (1000) information, and / or receiving interaction information of a target interaction intelligent agent in the whole vehicle interaction intelligent agent (200) and displaying the interaction information.

15. The client (1000) according to claim 14, wherein The client (1000) comprises at least one of a mobile terminal (1001) or a vehicle-mounted interaction display terminal (1002).

16. A vehicle (1), wherein comprising: a whole vehicle interaction intelligent agent (200), wherein the whole vehicle interaction intelligent agent (200) at least comprises a whole vehicle electrical appliance (300); and the vehicle controller (100) according to claim 13, wherein the vehicle controller (100) is connected with the whole vehicle interaction intelligent agent (200).

17. The vehicle (1) according to claim 16, wherein further comprising: a vehicle-mounted interaction display terminal (1002) used for sending client (1000) information.

Images