A vehicle-mounted middleware and a control method, device, medium, product and vehicle thereof

Abstract

The present disclosure provides a vehicle-mounted middleware and a control method, device, medium, product and vehicle thereof, and relates to the technical field of voice control, and can improve the effectiveness of vehicle voice control. The vehicle-mounted middleware is in communication connection with a vehicle voice hub system; specifically, in response to a first instruction, the target running state of a target application is queried; the first instruction is issued to the vehicle-mounted middleware by the vehicle voice hub system in a case where the vehicle voice hub system receives a first vehicle voice instruction and determines that the target application in the first vehicle voice instruction is a state-sensitive application; the first instruction is used to indicate the feasibility of verifying the first vehicle voice instruction; the first vehicle voice instruction is used to indicate that the target application executes a target action; the feasibility is verified based on the target running state; in response to determining that the feasibility is feasible, a feasible instruction is returned to the vehicle voice hub system, so that the vehicle voice hub system executes the first vehicle voice instruction.

Description

Technical Field

[0001] This disclosure relates to the field of vehicle control technology, and more particularly to the field of voice control technology, specifically to an in-vehicle middleware and its control method, device, medium, product, and vehicle. Background Technology

[0002] Currently, with the development of intelligent vehicles, in-vehicle voice control centers have become an important entry point for human-machine interaction. However, existing in-vehicle voice control centers, after parsing voice control commands, generally rely on service status information maintained internally by the center to determine the validity of the commands. They do not consider whether the service status information is reliable and accurate, leading to the problem of erroneous execution of voice control commands. Summary of the Invention

[0003] This disclosure provides an in-vehicle middleware and its control method, device, medium, product, and vehicle, which can improve the execution accuracy of in-vehicle voice control commands.

[0004] In a first aspect, this application provides an in-vehicle middleware, which is communicatively connected to an in-vehicle voice hub system. The in-vehicle middleware is used to query the target operating status of a target application in response to a first instruction. The first instruction is issued to the in-vehicle middleware by the in-vehicle voice hub system after receiving a first in-vehicle voice instruction and determining that the target application in the first in-vehicle voice instruction is a state-sensitive application. The first instruction is used to instruct the verification of the feasibility of the first in-vehicle voice instruction. The first in-vehicle voice instruction is used to instruct the target application to perform a target action. A state-sensitive application refers to an application whose controllable functions change with the application's operating status. Feasibility is verified based on the target operating status. In response to determining that the feasibility is feasible, a feasible instruction is returned to the in-vehicle voice hub system to cause the in-vehicle voice hub system to execute the first in-vehicle voice instruction.

[0005] The vehicle-mounted middleware provided in this application can, upon receiving a vehicle-mounted voice command from the vehicle-mounted voice control system and determining that the target application to be controlled by the vehicle-mounted voice command is a state-sensitive application, issue a state query command to the vehicle-mounted middleware. The vehicle-mounted middleware queries the running status of the target application through the state query command, enabling it to obtain the actual application status of the target application more accurately and effectively. Based on this, the feasibility of the running status is verified. If the running status of the target application allows the execution of the vehicle-mounted voice command, feasibility feedback is sent to the vehicle-mounted voice control system, and the vehicle-mounted voice command is executed. This can bind the vehicle-mounted voice command to the running status of the target application, enabling the vehicle-mounted voice command to more accurately control the target application in that running status, reducing the probability of invalid commands or application miscontrol, and ensuring the accuracy of vehicle-mounted voice interaction.

[0006] In some possible implementations, the vehicle-mounted middleware is specifically used for: determining the target control function associated with the target action in the target application; and querying the target operating status; wherein the target operating status is the functional operating status of the target control function in the target application.

[0007] According to the above implementation method, by binding the target action with the associated control function of the action, when the target action needs to be executed, only the running status of the associated control function needs to be queried, so as to more quickly and accurately determine whether the target application can currently execute the target action.

[0008] In some possible implementations, the vehicle middleware is specifically used to: call the first mapping relationship dataset corresponding to the target application; wherein the first mapping relationship dataset is pre-written by the backend engineers to represent the correspondence between the control actions of the target application and the control functions associated with the control actions; and determine the control functions associated with the target actions in the first mapping relationship dataset as the target control functions.

[0009] According to the above implementation method, by displaying the mapping relationship between the control actions in each application and the control functions associated with the control actions, it is possible to extract the corresponding control functions associated with the target action more intuitively and effectively when the target action is received, so as to reduce the waste of resources consumed in matching actions and control functions.

[0010] In some possible implementations, the vehicle middleware is communicatively connected to the application manager of the vehicle operating system; the vehicle middleware is specifically used for: finding the application process of the target application based on the application manager; in response to determining that the application process cannot be found, forcibly setting the target running state to hibernation; in response to finding the application process, using the application process, sending a running state retrieval command to the target application so that the target application returns to the target running state.

[0011] According to the above implementation method, the application manager searches for the application process of the target application. If the application process is found, it indicates that the target application is in the application open state, and the actual application state of the target application is directly obtained to determine the target running state of the target application. If the application process is not found, it indicates that the target application is in the application closed state, and the target running state is forcibly set to hibernation. This allows for timely and effective status feedback even when no application process is found.

[0012] In some possible implementations, the vehicle-mounted middleware is specifically used to: determine a second mapping relationship dataset corresponding to the target operating state; wherein the second mapping relationship dataset is used to represent the operating state of the target application and the set of restricted control actions corresponding to the operating state; the restricted control actions are pre-written into the restricted control action set by backend engineers; in response to determining that the target action is not in the second mapping relationship dataset, the feasibility is determined to be feasible.

[0013] According to the above implementation method, by obtaining the set of restricted control actions that are limited in the target operating state, it is possible to determine the range of control functions that the target action cannot perform in the target operating state. Thus, when it is found that the target action is inconsistent with the control actions in the set of restricted control actions, the feasibility of the target action can be determined in a timely manner, thereby improving the efficiency of feasibility determination of the target action.

[0014] In some possible implementations, the restricted control actions in the restricted control action set satisfy at least one of the following: they are meaningless to implement in the target operating state; there is redundancy and / or switching conflict in the current control action corresponding to the target operating state; the target operating state satisfies the restriction conditions pre-set for the target action.

[0015] According to the above implementation method, if the target action has no practical significance in the target operating state, it indicates that the conditions for executing the target action cannot be met; if there is redundancy and / or switching conflict in the current control action corresponding to the target operating state, it indicates that the control action can be executed, but the execution result is consistent with the current target operating state, and there is no need to execute it; if the target operating state meets the pre-set restriction conditions for the target action, it indicates that the target operating state will trigger the interception rules or logical restrictions on the target action, because the target action cannot be executed; if the target action is not any of the above-mentioned restricted operation actions, it indicates that the target action can be effectively and accurately executed in the target application.

[0016] In some possible implementations, the vehicle-mounted middleware is further configured to modify the target mapping relationship dataset in response to a second instruction; wherein the second instruction is an instruction containing modification information issued by the vehicle-mounted voice hub system upon receiving a second vehicle-mounted voice instruction; the modification information is used to indicate the modified content of the target mapping relationship dataset; the target mapping relationship dataset is a first mapping relationship dataset and / or a second mapping relationship dataset.

[0017] According to the above implementation method, when there is a change in the correspondence between the control action and the control function associated with the control action, or the correspondence between the running state and the restricted control action corresponding to the running state, the mapping relationship dataset can be changed according to the received change instruction. This allows for a more convenient and effective change of the mapping relationship without reconstructing the underlying running logic, greatly reducing the maintenance cost and difficulty of the target relationship dataset.

[0018] Secondly, this application provides a control method for an in-vehicle middleware, wherein the in-vehicle middleware is communicatively connected to an in-vehicle voice hub. The control method for the in-vehicle middleware includes: the in-vehicle middleware responding to a first instruction to query the target operating state of a target application; wherein the first instruction is issued to the in-vehicle middleware by the in-vehicle voice hub system after receiving a first in-vehicle voice instruction and determining that the target application in the first in-vehicle voice instruction is a state-sensitive application; the first instruction is used to instruct the verification of the feasibility of the first in-vehicle voice instruction; the first in-vehicle voice instruction is used to instruct the target application to perform a target action; a state-sensitive application refers to an application whose controllable functions change with the application's operating state; verifying feasibility based on the target operating state; and, in response to determining that the feasibility is feasible, returning a feasible instruction to the in-vehicle voice hub system to cause the in-vehicle voice hub system to execute the first in-vehicle voice instruction.

[0019] The control method for in-vehicle middleware provided in this application can, when the in-vehicle voice central system receives an in-vehicle voice command and determines that the target application to be controlled by the in-vehicle voice command is a state-sensitive application, send a state query command to the in-vehicle middleware. The in-vehicle middleware queries the running status of the target application through the state query command, which can more accurately and effectively obtain the actual application status of the target application. Based on this, the feasibility of the running status is verified. If the running status of the target application can execute the in-vehicle voice command, the feasibility feedback is sent to the in-vehicle voice central system, and the in-vehicle voice command is executed. This can bind the in-vehicle voice command with the running status of the target application, so that the in-vehicle voice command can more accurately control the target application in this running status, reduce the probability of invalid commands or application miscontrol, and ensure the accuracy of in-vehicle voice interaction.

[0020] In some possible implementations, the above method further includes: determining the target control function associated with the target action in the target application; querying the target running status; wherein the target running status is the functional running status of the target control function in the target application.

[0021] In some possible implementations, the above method specifically includes: calling the first mapping relationship dataset corresponding to the target application; wherein the first mapping relationship dataset is pre-written by the backend engineers to represent the correspondence between the control actions of the target application and the control functions associated with the control actions; and determining the control functions associated with the target actions in the first mapping relationship dataset as the target control functions.

[0022] In some possible implementations, the vehicle middleware is communicatively connected to the application manager of the vehicle operating system; the above method specifically includes: searching for the application process of the target application based on the application manager; in response to determining that the application process cannot be found, forcibly setting the target running state to hibernation; in response to finding the application process, using the application process to send a running state retrieval instruction to the target application, so that the target application returns to the target running state.

[0023] In some possible implementations, the above method further includes: determining a second mapping relationship dataset corresponding to the target running state; wherein the second mapping relationship dataset is used to represent the running state of the target application and the set of restricted control actions corresponding to the running state; the restricted control actions are pre-written into the restricted control action set by backend engineers; in response to determining that the target action is not in the second mapping relationship dataset, the feasibility is determined to be feasible.

[0024] In some possible implementations, the restricted control actions in the restricted control action set satisfy at least one of the following: they are meaningless to implement in the target operating state; there is redundancy and / or switching conflict in the current control action corresponding to the target operating state; the target operating state satisfies the restriction conditions pre-set for the target action.

[0025] In some possible implementations, the above method further includes: modifying the target mapping relationship dataset in response to a second instruction; wherein the second instruction is an instruction containing modification information issued by the vehicle voice center system upon receiving a second vehicle voice instruction; the modification information is used to indicate the modification content of the target mapping relationship dataset; the target mapping relationship dataset is a first mapping relationship dataset and / or a second mapping relationship dataset.

[0026] Thirdly, this application provides a control device, including: a memory and a processor; the memory and the processor are coupled; the memory is used to store a computer program; and the processor executes the computer program to implement the control method of the vehicle middleware of any of the above embodiments.

[0027] Fourthly, this application provides a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the control method of the vehicle middleware in any of the above embodiments.

[0028] Fifthly, this application provides a computer program product, which includes computer program instructions that, when executed by a processor, implement the control method of the vehicle middleware described in any of the above embodiments.

[0029] Sixthly, this application provides a vehicle including the control device of any of the preceding embodiments; or the computer-readable storage medium of any of the preceding embodiments; or the computer program product of any of the preceding embodiments. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in this disclosure, the accompanying drawings used in some embodiments of this disclosure will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings.

[0031] Figure 1 This is a schematic diagram of the architecture of an in-vehicle terminal provided in an embodiment of this disclosure; Figure 2 This is a flowchart illustrating a control method for an in-vehicle middleware provided in an embodiment of this disclosure; Figure 3 This is a flowchart illustrating another control method for an in-vehicle middleware provided in an embodiment of this disclosure; Figure 4 This is a flowchart illustrating another control method for an in-vehicle middleware provided in this embodiment of the present disclosure; Figure 5 This is a schematic diagram of the structure of a control device provided in some embodiments of this disclosure. Detailed Implementation

[0032] The technical solutions of this disclosure will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0033] It should be noted that, in this disclosure, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this disclosure should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0034] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0035] In the description of this disclosure, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. "And / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "more than one" means two or more.

[0036] As mentioned in the background section, with the development of intelligent vehicles, in-vehicle voice control centers have become an important entry point for human-machine interaction. However, existing in-vehicle voice control centers, after parsing voice control commands, generally rely on the service status information maintained internally by the center to determine the validity of the commands. They do not consider whether the service status information is a reliable and genuine service status, leading to the problem of erroneous execution of voice control commands.

[0037] To address the aforementioned technical problems, this disclosure provides a control method for vehicle-mounted middleware, the idea of ​​which is as follows: An in-vehicle middleware is deployed in the vehicle terminal and connected to the in-vehicle voice hub system. When the in-vehicle middleware receives an in-vehicle voice command for a target application from the in-vehicle voice hub system, it queries the target application's operating status and verifies the feasibility of the in-vehicle voice command based on the target operating status. If the feasibility of the in-vehicle voice command is determined, the middleware controls the in-vehicle voice hub system to execute the in-vehicle voice command.

[0038] The vehicle-mounted terminal provided in the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0039] See Figure 1 This is a schematic diagram of the architecture of a vehicle provided in an embodiment of this disclosure. Figure 1 As shown, the vehicle includes an in-vehicle terminal 100, which includes: an in-vehicle middleware 110, an in-vehicle voice hub system 120, and a target application 130.

[0040] The aforementioned in-vehicle middleware is used to verify the feasibility of voice control commands for vehicles (especially intelligent driving vehicles) and to execute the commands. Vehicles can also be referred to as vehicles, mobile carriers, electric vehicles (EVs), hybrid electric vehicles (HEVs), plug-in hybrid electric vehicles (PHEVs), fuel cell vehicles (FCVs), autonomous vehicles, intelligent and connected vehicles (ICVs), and driverless vehicles, etc.

[0041] In this application, the vehicle can be a sedan, a sport utility vehicle (SUV), a truck, a special vehicle (such as an ambulance, fire truck, police car, etc.), a driverless taxi, a smart connected bus, an autonomous logistics vehicle, an electric truck, etc. Furthermore, this method is also applicable to various special-purpose vehicles, such as agricultural vehicles, mining vehicles, forestry vehicles, airport vehicles, and port vehicles. This application does not impose specific limitations in this regard.

[0042] The aforementioned vehicle-mounted terminal, also known as the vehicle dispatch and monitoring terminal (TCU terminal) or vehicle-mounted IoT terminal, is deployed in the vehicle and is the core front-end equipment in the vehicle monitoring and management system.

[0043] The aforementioned vehicle-mounted terminal integrates multiple functions such as positioning, communication, and vehicle driving record, and has the ability to schedule and process data for the entire vehicle's operations.

[0044] The vehicle voice hub system 120 is used to receive a first vehicle voice command, and when it is determined that the target application 130 in the first vehicle voice command is a state-sensitive application, it issues a first command to the vehicle middleware 110.

[0045] The aforementioned first instruction is used to indicate the feasibility of verifying the first in-vehicle voice instruction.

[0046] The aforementioned first in-vehicle voice command is used to instruct the target application to perform the target action.

[0047] The aforementioned state-sensitive applications refer to applications whose controllable functions change depending on the application's operating state.

[0048] The vehicle middleware 110 is used to receive a first instruction issued by the vehicle voice hub system 120, and in response to the first instruction, query the target operating status of the target application 130; and verify the feasibility of the first vehicle voice instruction based on the target operating status.

[0049] The aforementioned target operating state refers to the actual operating state of the target application, which includes standby state and on state, etc.

[0050] The vehicle middleware 110 is also used to return a feasible instruction to the vehicle voice hub system 120 in response to the feasibility of the first vehicle voice instruction.

[0051] The in-vehicle voice control system 120 is also used to execute a first in-vehicle voice command to control the target application 130 when the aforementioned feasible command is received.

[0052] It is understood that the application scenarios of the embodiments of this disclosure are not limited. The system architecture and business scenarios described in the embodiments of this disclosure are for the purpose of more clearly illustrating the technical solutions of the embodiments of this disclosure, and do not constitute a limitation on the technical solutions provided by the embodiments of this disclosure. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of this disclosure are also applicable to similar technical problems.

[0053] The following is a detailed description of the vehicle-mounted middleware provided in the embodiments of this disclosure. This vehicle-mounted middleware is... Figure 1 The vehicle-mounted middleware 110 shown.

[0054] This disclosure provides an in-vehicle middleware, which is used for: In response to the first instruction, query the target application's target running status.

[0055] The aforementioned first instruction is issued by the vehicle voice central system to the vehicle middleware when it receives the first vehicle voice instruction and determines that the target application in the first vehicle voice instruction is a state-sensitive application.

[0056] The aforementioned in-vehicle voice commands are obtained by converting the raw audio stream input by the driver or passenger through the in-vehicle microphone array into structured voice commands through an automatic speech recognition module.

[0057] As one feasible approach, when the target application is not a state-sensitive application, the target application can be directly controlled based on the first in-vehicle voice command.

[0058] In some possible implementations, querying the target operating status of the target application includes: determining the target control function associated with the target action in the target application; querying the target operating status; wherein the target operating status is the functional operating status of the target control function in the target application.

[0059] As one feasible method, querying the target application's target running status specifically includes: upon receiving the first instruction, sending a running status retrieval instruction to the target application through the vehicle middleware, so that the target application can feed back the actual target running status to the vehicle middleware, thereby obtaining the target application's target running status.

[0060] As another possible approach, querying the target application's target running status also includes: storing the historical application status of each application of the vehicle terminal based on the service status tracking module deployed inside the vehicle voice hub system, and obtaining the historical application status stored by the service status tracking module as the target running status of the target application upon receiving the first instruction.

[0061] In some possible implementations, determining the target control function associated with the target action includes: calling the first mapping relationship dataset corresponding to the target application; wherein the first mapping relationship dataset is pre-written by the backend engineers to represent the correspondence between the control actions of the target application and the control functions associated with the control actions; and determining the control functions associated with the target action in the first mapping relationship dataset as the target control functions.

[0062] The aforementioned target application refers to the specific software running in the in-vehicle terminal and about to be invoked by in-vehicle voice commands. For example, the target application could be an in-vehicle navigation system, a music playback application, or an air conditioning control panel.

[0063] The aforementioned first mapping dataset is a pre-built set that reflects the correspondence between the control actions associated with the user's in-vehicle voice control commands and the specific control functions that actually need to be executed in the target application.

[0064] As one possible approach, there is a one-to-one mapping between the target application's control actions and the associated control functions. For example, if the control action is to cool down to 25 degrees Celsius, the control function would be to adjust the air conditioner temperature to 25 degrees Celsius.

[0065] As another possible approach, there is a many-to-one mapping between the target application's control actions and the associated control functions. For example, if the control action is to cool down to 25 degrees Celsius or heat up to 25 degrees Celsius, the control function would be to adjust the air conditioner temperature to 25 degrees Celsius.

[0066] The above-mentioned control actions are obtained after parsing the in-vehicle voice commands, and can clearly reflect the control actions corresponding to the in-vehicle voice commands.

[0067] The aforementioned control functions are specific business functions bound to the target action within the target application. After associating the control action with the control function, it is possible to clearly and effectively issue specific and effective execution functions to the target application.

[0068] Feasibility was verified based on the target's operational status.

[0069] In some possible implementations, feasibility is verified based on the target operating state, including: Determine the second mapping relationship dataset corresponding to the target running state.

[0070] In response to the determination that the target action is not in the second mapping relationship dataset, the feasibility is determined to be feasible.

[0071] The second mapping relationship dataset is used to represent the running state of the target application and the set of restricted control actions corresponding to the running state.

[0072] Among them, the restricted control actions are pre-written into the restricted control action set by the back-end engineers.

[0073] As one possible implementation method, verifying feasibility based on the target operating state also includes: if the first in-vehicle voice command cannot be executed when the target application is in the target operating state, it indicates that the first in-vehicle voice command is not feasible. For example, if the air conditioner is off, the command to stop the airflow cannot be executed.

[0074] In some possible implementations, the restricted control actions in the restricted control action set satisfy at least one of the following: they are meaningless to implement in the target operating state; there is redundancy and / or switching conflict in the current control action corresponding to the target operating state; the target operating state satisfies the restriction conditions pre-set for the target action.

[0075] For example, sending a "pause playback" voice command to a music playback application that is not playing music has no practical significance when the application is in the target operating state.

[0076] As one feasible approach, when the control action is restricted to a state where it is meaningless to perform, the first in-vehicle voice command is not executed, and the user is given feedback of preset application status statements through the in-vehicle voice hub system, such as: "There is no music playing at the moment".

[0077] For example, if a car voice command to "pause playback" is sent to a music playback application that is in a paused state, there is a redundant operation with the current control action corresponding to the target operating state.

[0078] As one feasible approach, when the control action is restricted to a state where there is redundancy and / or switching conflict with the current control action corresponding to the target operating state, the first in-vehicle voice command is not executed. Instead, a preset clarifying statement, such as "Music is paused," is broadcast through the in-vehicle voice control system. This feedback proactively informs the user of the system's true status, transforming a potential interaction failure into a valid status confirmation.

[0079] As one feasible approach, when a service status tracking module is deployed in the in-vehicle voice control system, the target's operating status is fed back to the service status tracking module, and the target's operating status is marked as an authoritative status in the service status tracking module. Next, the feasibility of the first in-vehicle voice command is verified based on the authoritative status, and if the feasibility is deemed feasible, a feasible command is returned to the in-vehicle voice control system, enabling the in-vehicle voice control system to execute the first in-vehicle voice command.

[0080] In response to determining that the feasibility is feasible, a feasible instruction is returned to the vehicle voice center system so that the vehicle voice center system executes the first vehicle voice instruction.

[0081] As one feasible approach, in response to determining that the feasibility is infeasible, an infeasible instruction is returned to the in-vehicle voice control system, so that the in-vehicle voice control system can provide feedback to the user regarding the infeasibility of the current instruction.

[0082] As one feasible approach, while providing feedback to the user regarding the infeasibility of the current command based on the in-vehicle voice control system, it is also possible to display such information on the in-vehicle display screen.

[0083] In some embodiments, if no feasibility determination result is received within a preset time period, it is determined that the target application is in an abnormal state and cannot be used. Based on this, the first in-vehicle voice command is refused to be executed, and the user is informed of the temporary unavailability by the in-vehicle voice hub system, such as a voice prompt "Service is temporarily unavailable".

[0084] The vehicle-mounted middleware provided in this application can, upon receiving a vehicle-mounted voice command from the vehicle-mounted voice control system and determining that the target application to be controlled by the vehicle-mounted voice command is a state-sensitive application, issue a state query command to the vehicle-mounted middleware. The vehicle-mounted middleware queries the running status of the target application through the state query command, enabling it to obtain the actual application status of the target application more accurately and effectively. Based on this, the feasibility of the running status is verified. If the running status of the target application allows the execution of the vehicle-mounted voice command, feasibility feedback is sent to the vehicle-mounted voice control system, and the vehicle-mounted voice command is executed. This can bind the vehicle-mounted voice command to the running status of the target application, enabling the vehicle-mounted voice command to more accurately control the target application in that running status, reducing the probability of invalid commands or application miscontrol, and ensuring the accuracy of vehicle-mounted voice interaction.

[0085] To facilitate understanding, the following example illustrates the vehicle middleware.

[0086] For example, the vehicle middleware communicates with the application manager of the vehicle operating system.

[0087] When determining the target's runtime status based on the vehicle-mounted middleware, the following are included: Use the application manager to find the application process of the target application.

[0088] In response to the determination that the application process cannot be found, the target running state is forcibly set to sleep, and the target running state is returned.

[0089] In response to the discovery of the application process, the application process is used to send a runtime status retrieval command to the target application, so that the target application returns to the target runtime status.

[0090] The aforementioned application manager is deployed in the vehicle terminal and is a management module that supports functions such as installing, uninstalling, disabling, and backing up system applications and personal applications on the vehicle's infotainment system.

[0091] The aforementioned application process refers to a running application instance.

[0092] The target application mentioned above may include one or more application processes. For example, if the target application is a music playback application, the music playback application includes a main process, a background playback service process, a resource loading and processing process, a cross-application communication process, and a desktop widget process, etc.; among them, the main process refers to the interface that the user can see and interact with, which can display the song list, album art, lyrics, and respond to the user's click or swipe operations, etc.

[0093] In some embodiments, the inability to find the application process indicates that the target application has not started or has exited; the process name does not match the application name; permissions are restricted, the application cannot be executed, or the application does not exist.

[0094] For example, in the case of a change in the mapping relationship between the first mapping relationship dataset and / or the second mapping relationship dataset in the above embodiments, the vehicle middleware is further configured to: In response to the second instruction, the target mapping dataset is modified.

[0095] The second instruction is issued by the vehicle voice center system upon receiving the second vehicle voice instruction, and contains change information; the change information is used to indicate the changes to the target mapping relationship dataset; the target mapping relationship dataset is the first mapping relationship dataset and / or the second mapping relationship dataset.

[0096] The aforementioned changes include adding, deleting, and modifying mapping relationships.

[0097] The foregoing primarily describes the solutions of the embodiments of this disclosure from the perspective of the device. It is understood that, in order to achieve the aforementioned functions, the vehicle-mounted middleware includes at least one of the hardware structures and software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure.

[0098] This disclosure embodiment can divide the functional execution logic of the control method for the vehicle middleware according to the above-described device embodiment. For example, each functional execution logic can be divided into a step, or two or more functional execution logics can be integrated into one step. The integrated step can be implemented in hardware or software. It should be noted that the specific division of steps in this disclosure embodiment is illustrative and only represents one type of execution logic division. In actual implementation, there may be other division methods. The following description uses the division of each step according to each functional execution logic as an example.

[0099] Figure 2 This is a flowchart illustrating a control method for an in-vehicle middleware according to an embodiment of this disclosure. This control method is used to control the aforementioned in-vehicle middleware, such as... Figure 2As shown, the in-vehicle middleware is communicatively connected to the in-vehicle voice center, and the method includes the following steps: S201, The vehicle middleware responds to the first instruction and queries the target application's target running status.

[0100] The first instruction is issued by the vehicle voice hub system to the vehicle middleware when it receives the first vehicle voice command and determines that the target application in the first vehicle voice command is a state-sensitive application. The first instruction is used to verify the feasibility of the first vehicle voice command. The first vehicle voice command is used to instruct the target application to perform the target action. A state-sensitive application refers to an application whose controllable functions change with the application's operating state.

[0101] S202. Verify feasibility based on the target's operational status.

[0102] S203. In response to determining that the feasibility is feasible, a feasible instruction is returned to the vehicle voice center system so that the vehicle voice center system executes the first vehicle voice instruction.

[0103] The control method for in-vehicle middleware provided in this application can, when the in-vehicle voice central system receives an in-vehicle voice command and determines that the target application to be controlled by the in-vehicle voice command is a state-sensitive application, send a state query command to the in-vehicle middleware. The in-vehicle middleware queries the running status of the target application through the state query command, which can more accurately and effectively obtain the actual application status of the target application. Based on this, the feasibility of the running status is verified. If the running status of the target application can execute the in-vehicle voice command, the feasibility feedback is sent to the in-vehicle voice central system, and the in-vehicle voice command is executed. This can bind the in-vehicle voice command with the running status of the target application, so that the in-vehicle voice command can more accurately control the target application in this running status, reduce the probability of invalid commands or application miscontrol, and ensure the accuracy of in-vehicle voice interaction.

[0104] To facilitate understanding, the following example illustrates the vehicle middleware.

[0105] For example, Figure 3 This is a flowchart illustrating another control method for vehicle-mounted middleware provided in this embodiment; combined with Figure 2 and Figure 3 The control method for the vehicle-mounted middleware includes: S201-1, The vehicle-mounted middleware responds to the first instruction and determines the target control function associated with the target action in the target application.

[0106] S201-2, Query the target's operating status; whereby the target's operating status is the functional operating status of the target's control function in the target application.

[0107] In some possible implementations, the above method specifically includes: calling the first mapping relationship dataset corresponding to the target application; wherein the first mapping relationship dataset is pre-written by the backend engineers to represent the correspondence between the control actions of the target application and the control functions associated with the control actions; and determining the control functions associated with the target actions in the first mapping relationship dataset as the target control functions.

[0108] For example, Figure 4 This is a flowchart illustrating another control method for in-vehicle middleware provided in this disclosure embodiment; the in-vehicle middleware is communicatively connected to the application manager of the in-vehicle operating system; combined with Figure 2 and Figure 4 The control method for the vehicle-mounted middleware includes: S201-a, The vehicle middleware responds to the first instruction and searches for the application process of the target application based on the application manager; S201-b: In response to determining that the application process cannot be found, the target running state is forcibly set to hibernation.

[0109] S201-c: In response to finding the application process, use the application process to send a runtime status retrieval command to the target application so that the target application returns to the target runtime status.

[0110] In some possible implementations, the above method further includes: determining a second mapping relationship dataset corresponding to the target running state; wherein the second mapping relationship dataset is used to represent the running state of the target application and the set of restricted control actions corresponding to the running state; the restricted control actions are pre-written into the restricted control action set by backend engineers; in response to determining that the target action is not in the second mapping relationship dataset, the feasibility is determined to be feasible.

[0111] In some possible implementations, the restricted control actions in the restricted control action set satisfy at least one of the following: they are meaningless to implement in the target operating state; there is redundancy and / or switching conflict in the current control action corresponding to the target operating state; the target operating state satisfies the restriction conditions pre-set for the target action.

[0112] In some possible implementations, the above method further includes: modifying the target mapping relationship dataset in response to a second instruction; wherein the second instruction is an instruction containing modification information issued by the vehicle voice center system upon receiving a second vehicle voice instruction; the modification information is used to indicate the modification content of the target mapping relationship dataset; the target mapping relationship dataset is a first mapping relationship dataset and / or a second mapping relationship dataset.

[0113] When implementing the functions of the integrated modules described above in hardware, this disclosure provides a possible structure for the control device involved in the above embodiments. For example... Figure 5 As shown, the control device 500 includes a processor 502 and a bus 504. Optionally, the control device may also include a memory 501; alternatively, the control device 500 may also include a communication interface 503.

[0114] Processor 502 may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with embodiments of this disclosure. Processor 502 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with embodiments of this disclosure. Processor 502 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0115] Communication interface 503 is used to connect to other devices via a communication network. This communication network can be Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0116] The memory 501 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0117] In one possible implementation, the memory 501 can exist independently of the processor 502. The memory 501 can be connected to the processor 502 via a bus 504 and is used to store instructions or program code. When the processor 502 calls and executes the instructions or program code stored in the memory 501, it can implement the control method of the vehicle middleware provided in this embodiment. In another possible implementation, the memory 501 can also be integrated with the processor 502.

[0118] Bus 504 can be an extended industry standard architecture (EISA) bus, etc. Bus 504 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0119] Some embodiments of this disclosure provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium) storing computer program instructions that, when executed on a processor, cause the processor to perform a control method for an in-vehicle middleware as described in any of the above embodiments.

[0120] Exemplary examples of computer-readable storage media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes), optical discs (e.g., compact disks (CDs), digital versatile disks (DVDs), etc.), smart cards, and flash memory devices (e.g., erasable programmable read-only memory (EPROMs), cards, sticks, or key drives, etc.). The various computer-readable storage media described in this disclosure may represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0121] This disclosure provides a computer program product containing instructions that, when run on a processor, cause the processor to execute the vehicle middleware control method described in any of the above embodiments.

[0122] This disclosure provides a vehicle including the control device of any of the preceding embodiments; or the computer-readable storage medium of any of the preceding embodiments; or the computer program product of any of the preceding embodiments.

[0123] The vehicle provided in this application may be a passenger vehicle or a freight vehicle, and may also be an electric vehicle or a hybrid vehicle. This application does not limit the specific purpose or power type of the vehicle, and the choice can be made according to actual needs.

[0124] A vehicle consists of a body and wheels. The body is used for passengers and for carrying goods, while the wheels are mounted underneath the body to support it and allow it to roll on the road, thus enabling the vehicle to move.

[0125] In some possible examples, the vehicle is equipped with a control system, which typically adopts a layered distributed architecture. From the bottom layer to the top layer, it can be roughly divided into a perception layer, a control layer, a coordination layer, and an interaction layer. The layers communicate with each other through an in-vehicle network.

[0126] The perception layer mainly consists of various sensors distributed inside and outside the vehicle, including but not limited to external environment cameras, millimeter-wave radar, lidar, ultrasonic sensors, in-vehicle driver monitoring cameras, microphone arrays, and various vehicle status sensors (such as wheel speed sensors, inertial measurement units, temperature sensors, etc.). The perception layer is responsible for collecting multi-dimensional data such as the vehicle's own operating status, driver behavior, and external driving environment in real time.

[0127] The control layer consists of dozens to hundreds of electronic control units (ECUs), distributed across multiple functional domains including powertrain, chassis, body, intelligent driving, and infotainment. Each ECU embeds real-time control software that performs closed-loop control of the vehicle's actuators based on preset control strategies or upper-level commands, and generates corresponding alarm signals when abnormal conditions are detected. Typical ECUs include the engine control unit, transmission control unit, brake control unit, steering control unit, vehicle stability control unit, airbag control unit, intelligent driving domain controller, and in-vehicle infotainment unit.

[0128] The coordination layer typically exists in the form of a domain controller or a central computing platform, responsible for cross-domain data fusion, global state management, and collaborative decision-making. The coordination layer centrally processes and schedules the sensing data and control commands that were originally scattered across various functional domains, connecting downwards to various electronic control units and supporting human-machine interaction functions upwards.

[0129] The interaction layer mainly includes in-cabin display devices (such as instrument panel, central control screen, head-up display), voice interaction system, haptic feedback device, etc., which are responsible for presenting vehicle status, warning information and driving suggestions to the driver in the form of visual, auditory or tactile, while receiving the driver's touch, voice and other input commands.

[0130] Data transmission and interaction between different layers are achieved through the vehicle bus network. Common vehicle bus protocols include CAN (Controller Area Network), CAN FD (CAN with Flexible Data-Rate), LIN (Local Interconnect Network), FlexRay, and vehicle Ethernet, which supports high-bandwidth data transmission. Among these, CAN and CAN FD buses are widely used for communication in real-time control domains such as powertrain and chassis, while vehicle Ethernet is gradually being applied to high-bandwidth sensor data transmission in the intelligent driving domain and multimedia interaction scenarios in the cockpit domain.

[0131] In some possible examples, the intelligent driving domain controller, as the core of the in-vehicle computing platform, is equipped with a processor and memory to execute the control method of the in-vehicle middleware provided in the embodiments of this application.

[0132] For example, the control device of this application is arranged in the intelligent driving domain controller.

[0133] For example, the intelligent driving domain controller is equipped with multiple functional modules for executing the method steps of the control method of the vehicle middleware.

[0134] For example, an in-vehicle middleware is deployed in the intelligent driving domain controller.

[0135] The aforementioned vehicle middleware is used to query the target application's target running status in response to the first command issued by the vehicle voice hub system; The first instruction is issued by the vehicle voice hub system to the vehicle middleware when it receives the first vehicle voice command and determines that the target application in the first vehicle voice command is a state-sensitive application. The first instruction is used to verify the feasibility of the first vehicle voice command. The first vehicle voice command is used to instruct the target application to perform the target action. A state-sensitive application refers to an application whose controllable functions change with the application's operating state.

[0136] The vehicle-mounted middleware is also used to verify feasibility based on the target operating state; and in response to determining that the feasibility is feasible, it returns a feasible instruction to the vehicle-mounted voice hub system so that the vehicle-mounted voice hub system executes a first vehicle-mounted voice instruction to control the target application.

[0137] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any changes or substitutions within the technical scope disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A vehicle-mounted middleware, characterized in that, The vehicle-mounted middleware includes: The in-vehicle middleware is used to query the target operating status of a target application in response to a first instruction. The first instruction is issued to the in-vehicle middleware by the in-vehicle voice hub system after receiving a first in-vehicle voice instruction and determining that the target application in the first in-vehicle voice instruction is a state-sensitive application. The first instruction is used to instruct the verification of the feasibility of the first in-vehicle voice instruction. The first in-vehicle voice instruction is used to instruct the target application to perform a target action. The state-sensitive application refers to an application whose controllable functions change with the application's operating status. Verify the feasibility based on the target operating status; In response to determining that the feasibility is feasible, a feasible instruction is returned to the in-vehicle voice center system to cause the in-vehicle voice center system to execute the first in-vehicle voice instruction.

2. The vehicle-mounted middleware according to claim 1, characterized in that, The vehicle-mounted middleware is specifically used for: Determine the target control function associated with the target action in the target application; Query the target's operating status; wherein, the target's operating status is the functional operating status of the target control function in the target application.

3. The vehicle-mounted middleware according to claim 2, characterized in that, The vehicle-mounted middleware is specifically used for: The first mapping relationship dataset corresponding to the target application is invoked; wherein, the first mapping relationship dataset is pre-written by the backend engineers and is used to represent the correspondence between the control actions of the target application and the control functions associated with the control actions; The control functions associated with the target action in the first mapping relationship dataset are identified as the target control functions.

4. The vehicle-mounted middleware according to any one of claims 1-3, characterized in that, The vehicle-mounted middleware is communicatively connected to the application manager of the vehicle operating system; the vehicle-mounted middleware is specifically used for: The application manager is used to locate the application process of the target application; In response to determining that the application process cannot be found, the target running state is forcibly set to hibernation; In response to finding the application process, the application process is used to send a running status retrieval instruction to the target application, so that the target application returns to the target running status.

5. The vehicle-mounted middleware according to claim 4, characterized in that, The vehicle-mounted middleware is specifically used for: A second mapping relationship dataset corresponding to the target running state is determined; wherein, the second mapping relationship dataset is used to represent the running state of the target application and the set of restricted control actions corresponding to the running state; the restricted control actions are pre-written into the restricted control action set by backend engineers; In response to determining that the target action is not in the second mapping dataset, the feasibility is determined to be feasible.

6. The vehicle-mounted middleware according to claim 5, characterized in that, The restricted control actions in the set of restricted control actions satisfy at least one of the following: It is meaningless to implement under the target operating state; There is redundancy and / or switching conflict in the current control action corresponding to the target operating state; The target operating state satisfies the pre-set constraints for the target action.

7. The vehicle-mounted middleware according to claim 5, characterized in that, The vehicle-mounted middleware is also used to: modify the target mapping relationship dataset in response to the second instruction; Wherein, the second instruction is issued by the vehicle voice center system upon receiving the second vehicle voice instruction, and includes change information; the change information is used to indicate the changes to the target mapping relationship dataset; The target mapping relationship dataset is the first mapping relationship dataset and / or the second mapping relationship dataset.

8. A control method for vehicle-mounted middleware, characterized in that, The method includes: The vehicle-mounted middleware responds to the first instruction and queries the target application's target running status. Wherein, the first instruction is issued by the vehicle voice hub system to the vehicle middleware when it receives the first vehicle voice instruction and determines that the target application in the first vehicle voice instruction is a state-sensitive application; the first instruction is used to instruct the verification of the feasibility of the first vehicle voice instruction; the first vehicle voice instruction is used to instruct the target application to perform the target action; the state-sensitive application refers to an application whose controllable functions change with the application's running state. Verify the feasibility based on the target operating status; In response to determining that the feasibility is feasible, a feasible instruction is returned to the in-vehicle voice center system to cause the in-vehicle voice center system to execute the first in-vehicle voice instruction.

9. A control device, characterized in that, The control device includes: a memory and a processor; the memory and the processor are coupled; the memory is used to store instructions executable by the processor; when the processor executes the instructions, it performs the control method of the vehicle middleware as described in claim 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on a processor, cause the processor to perform the control method for the vehicle middleware as described in claim 8.

11. A computer program product, characterized in that, It includes a computer program; when the computer program is executed, it can implement the control method of the vehicle middleware as described in claim 8.

12. A vehicle, characterized in that, It includes the control device as claimed in claim 9; or the computer-readable storage medium as claimed in claim 10; or the computer program product as claimed in claim 11.

Images