Multi-mode-based multi-user process resource management method and device for in-vehicle infotainment system
By dynamically managing the occupant area of the vehicle's infotainment system using multimodal recognition technology, the problem of resource waste in multi-user scenarios is solved, achieving more efficient energy utilization and a smoother user experience.
Patent Information
- Application Number
- CN202510940405.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-09
- Publication Date
- 2025-11-21
AI Technical Summary
Existing in-vehicle infotainment systems suffer from resource waste in multi-user scenarios, especially when passengers are not in the riding area and system resources still need to be maintained, leading to increased energy consumption and decreased smoothness.
Through multimodal recognition technology, the system dynamically manages the riding and usage of passenger areas, dynamically starts and releases human-computer interaction processes and resource allocation, and optimizes multi-user process management strategies.
It reduces computing power and energy consumption, improves the smoothness of user cockpit interaction and energy efficiency, and implements the low-carbon green design concept.
Smart Images
Figure CN120994325A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of intelligent cockpit, in particular to a multi-modal based vehicle machine multi-user process resource management method, a multi-modal based vehicle machine multi-user process resource management device, an electronic device, a storage medium and a vehicle. BACKGROUND
[0002] With the continuous improvement of people's demand for cockpit entertainment, the current intelligent cockpit machine of the automotive industry generally supports multi-user system design with multiple screens and multiple peripherals as the main form of expression in different seating areas. Under the existing scheme, the vehicle-mounted system configures different default peripheral resources and system resources for the processes of different users. The concurrent scenario of multi-user also poses new challenges to the resource planning capabilities of the vehicle machine system such as computing power and memory. The current multi-user scheme generally adopts the following implementation form: the life cycle of the co-pilot, rear passengers and other occupant users in the multi-user is a static period, which depends on system power-on initialization, system power-off, hibernation and the life cycle of the main user.
[0003] The advantage of this design is that the co-pilot and rear passengers are always users who can respond to the operations of passengers in the corresponding seating area at any time. However, there is also a problem of waste of system resources. In the state where there is no passenger in the corresponding area, the system still needs to provide basic maintenance costs for the application processes and service processes of the corresponding users. Considering the demand for improvement of cockpit system smoothness and energy saving and other factors, there is room for optimization of the scheme.
[0004] Therefore, a multi-modal recognition based intelligent cockpit multi-user process and resource management scheme is needed, which optimizes the multi-user process management strategy of the existing vehicle machine system with the help of multi-modal recognition means; further, according to the actual seating and use of each occupant area, the life cycle management of multi-user processes and resources is dynamically performed; thereby improving the performance of the current vehicle machine system and energy consumption, improving the smoothness experience of user cockpit interaction, and implementing the low-carbon and green energy-saving design concept. SUMMARY
[0005] The purpose of the present application is to provide a multi-modal based vehicle machine multi-user process resource management method, a multi-modal based vehicle machine multi-user process resource management device, an electronic device, a storage medium and a vehicle, which at least solve one of the technical problems of how to use multi-modal recognition means to confirm the actual seating and use of each occupant area, and how to dynamically manage the life cycle of multi-user processes and resources.
[0006] The present application provides the following scheme:
[0007] According to one aspect of the present application, a multi-modal based vehicle machine multi-user process resource management method is provided, which comprises:
[0008] Based on the intelligent cockpit of the vehicle, multi-modal recognition information is obtained;
[0009] According to the multi-modal recognition information, the human-computer interaction state of the vehicle-mounted user is confirmed;
[0010] According to the human-computer interaction state of the vehicle-mounted user, the process based on serving the human-computer interaction of the vehicle-mounted user is controlled;
[0011] According to the control of the process based on serving the human-computer interaction of the vehicle-mounted user, the resource management based on realizing the process of serving the human-computer interaction of the vehicle-mounted user is controlled;
[0012] Among them, confirming the human-computer interaction state of the vehicle-mounted user includes confirming the generation of human-computer interaction demand, verifying the human-computer interaction condition and monitoring the human-computer interaction state.
[0013] Further, the generation of human-computer interaction demand includes:
[0014] Detecting whether the vehicle state is a parking state;
[0015] If yes, the state information of the door opening and the seat weighing is obtained;
[0016] According to the state information of the door opening and the seat weighing, the information of the human-computer interaction demand is generated.
[0017] Further, the verification of the human-computer interaction condition includes:
[0018] Detecting whether the vehicle state is a non-parking state;
[0019] If yes, according to the state information of the seat weighing, the information of the user's sitting state is obtained;
[0020] Obtaining the user's sitting state information of visual perception;
[0021] According to the user's sitting state information of visual perception, it is verified whether the condition of initiating human-computer interaction under the user's sitting state based on seat weighing is established;
[0022] If yes, the process of serving the human-computer interaction of the vehicle-mounted user corresponding to the sitting state is started and the resource of human-computer interaction is allocated.
[0023] Further, the monitoring of the human-computer interaction state includes:
[0024] Detecting whether the vehicle state is a non-parking state;
[0025] If yes, a man-machine interactive interface object for user switching is dynamically monitored according to the seat weight state information and the visually perceived user seating state information;
[0026] According to the dynamically monitored man-machine interactive interface object for user switching, the process of service vehicle user man-machine interaction is dynamically switched and the resource allocation of man-machine interaction is dynamically switched.
[0027] Further, it further comprises:
[0028] According to the user seating state, at least one man-machine interactive interface object is allocated;
[0029] According to the process of starting service vehicle user man-machine interaction and the allocation of man-machine interaction, each man-machine interactive interface object activates man-machine interaction with the user.
[0030] Further, it further comprises:
[0031] The process of starting service vehicle user man-machine interaction and the allocation of man-machine interaction comprises activating the man-machine interactive interface object in hibernation or starting the man-machine interactive interface object in shutdown state;
[0032] Starting the man-machine interactive interface object in shutdown state comprises lighting the screen of the man-machine interactive interface object, pulling up the screen desktop process, initializing the preset business process and providing the preset cabin interaction function.
[0033] According to the two aspects of the present application, a multi-modal based vehicle-machine multi-user process resource management device is provided, which comprises:
[0034] A multi-modal information acquisition module is configured to acquire multi-modal recognition information based on a vehicle intelligent cabin;
[0035] A man-machine interaction state module is configured to confirm the man-machine interaction state of a vehicle user according to the multi-modal recognition information;
[0036] An interaction process state module is configured to control the process of serving vehicle user man-machine interaction according to the man-machine interaction state of the vehicle user;
[0037] A process resource management module is configured to control the resource management for realizing the process of serving vehicle user man-machine interaction according to the control of the process of serving vehicle user man-machine interaction.
[0038] The confirmation of the man-machine interaction state of the vehicle user comprises confirming the generation of man-machine interaction demand, verifying the man-machine interaction condition and monitoring the man-machine interaction state.
[0039] According to three aspects of the present application, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus.
[0040] The memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the multi-modal based vehicle machine multi-user process resource management method.
[0041] According to four aspects of the present application, a computer readable storage medium is provided, which stores a computer program executable by an electronic device, and when the computer program runs on the electronic device, the electronic device executes the steps of the multi-modal based vehicle machine multi-user process resource management method.
[0042] According to five aspects of the present application, a vehicle is provided, comprising:
[0043] An electronic device is used to implement the steps of the multi-modal based vehicle machine multi-user process resource management method.
[0044] A processor runs a program, and when the program runs, the data output from the electronic device executes the steps of the multi-modal based vehicle machine multi-user process resource management method.
[0045] A storage medium is used to store a program, and when the program runs, the data output from the electronic device executes the steps of the multi-modal based vehicle machine multi-user process resource management method.
[0046] Through the above scheme, the following beneficial technical effects are obtained:
[0047] The present application confirms the actual occupancy and use of each passenger area through multi-modal recognition means, thereby making decisions on resource management and user processes, and reducing the waste of computing power resources and energy consumption.
[0048] The present application manages the life cycle of dynamic multi-user processes and resources, so that the human-machine interaction in the cabin is always in the state of serving user interaction needs.
[0049] The present application optimizes the multi-user process management strategy of the existing vehicle machine system, improves the performance and energy consumption of the current vehicle machine system, and improves the fluency experience of user cabin interaction, and implements the low-carbon and green energy-saving design concept. BRIEF DESCRIPTION OF DRAWINGS
[0050] Figure 1 is a flowchart of a multi-modal based vehicle machine multi-user process resource management method provided by one or more embodiments of the present application.
[0051] Figure 2is a structural diagram of a multi-modal based vehicle-machine multi-user process resource management device provided by one or more embodiments of the present application.
[0052] Figure 3 is a schematic diagram of a cockpit system starting step under multi-user provided by one specific embodiment of the present application.
[0053] Figure 4 is a schematic diagram of an intelligent cockpit function module relationship provided by one specific embodiment of the present application.
[0054] Figure 5 is an electronic device structural block diagram of a multi-modal based vehicle-machine multi-user process resource management method provided by one or more embodiments of the present application. DETAILED DESCRIPTION
[0055] The technical solutions of the present application will be described clearly and completely below in conjunction with the drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.
[0056] Figure 1 is a flowchart of a multi-modal based vehicle-machine multi-user process resource management method provided by one or more embodiments of the present application.
[0057] As shown in Figure 1 the multi-modal based vehicle-machine multi-user process resource management method includes:
[0058] Step S1, based on the vehicle intelligent cockpit, acquiring multi-modal recognition information;
[0059] Step S2, according to the multi-modal recognition information, confirming the human-computer interaction state of the vehicle-mounted user;
[0060] Step S3, according to the human-computer interaction state of the vehicle-mounted user, controlling the process based on serving the human-computer interaction of the vehicle-mounted user;
[0061] Step S4, according to the control of the process based on serving the human-computer interaction of the vehicle-mounted user, controlling the resource management based on realizing the process of serving the human-computer interaction of the vehicle-mounted user;
[0062] Step S5, confirming the human-computer interaction state of the vehicle-mounted user includes confirming the generation of the human-computer interaction demand of the vehicle-mounted user, verifying the human-computer interaction condition and monitoring the human-computer interaction state.
[0063] Specifically, in one specific embodiment, acquiring the multi-modal recognition information includes: CAN signal monitoring of the vehicle door switch and seat in place, confirming the vehicle getting on and off, the seat position, starting the camera vision passenger perception algorithm of the OMS / DMS (occupant monitoring system / driver monitoring system) service, confirming whether there is a change in the occupant riding state, etc. According to the control process based on the service vehicle-mounted user human-computer interaction, the resource management based on the implementation of the service vehicle-mounted user human-computer interaction process includes: vehicle starting, personnel entering the vehicle, and vehicle machine starting. When the vehicle machine enters the power-on and wake-up state, the user management service only performs default initialization on the driver user, and notifies the process management module, package management module, and window management module system services to allocate resources. Then, based on other passengers, if the user management service detects that an occupant in a certain seating area joins and enters a stable state (maintains the seating state for a period of time, or enters a driving state, etc.), the initialization and resource allocation of the user corresponding to the area are dynamically triggered. For example, the screen of the corresponding area is turned on, the default desktop process is pulled up, the necessary business processes are initialized, and the cabin interaction function is provided for the passengers in the corresponding area.
[0064] In the embodiment, the generation of the human-computer interaction demand includes:
[0065] detecting whether the vehicle state is a parking state;
[0066] if yes, acquiring the state information of the door opening and the seat weight;
[0067] generating the information of the human-computer interaction demand according to the state information of the door opening and the seat weight.
[0068] Specifically, in one specific embodiment, in the vehicle parking state, after the user management service monitors the door switch action and the seat weight signal change, the direction source and specific information of the signal are judged to confirm whether there is a passenger on the seat, thereby forming the demand of human-computer interaction and the source of human-computer interaction demand on the human-computer interaction interface object corresponding to the seat.
[0069] In the embodiment, the verification of the human-computer interaction condition includes:
[0070] detecting whether the vehicle state is a non-parking state;
[0071] if yes, acquiring the information of the user riding state according to the state information of the seat weight;
[0072] acquiring the user riding state information of visual perception;
[0073] verifying whether the condition of initiating human-computer interaction under the user riding state based on the seat weight is established according to the user riding state information of visual perception;
[0074] If yes, the process of the service vehicle user human-computer interaction is started and the human-computer interaction resources are allocated for the user in the corresponding riding state.
[0075] Specifically, in one specific embodiment, during the driving of the vehicle, the camera visual passenger sensing algorithm of the OMS / DMS (occupant monitoring system / driver monitoring system) service is started to confirm whether there is a change in the occupant riding state; if the occupant remains in the current seat without change and the state is stable, the condition for initiating the human-computer interaction is met, the process of the service vehicle user human-computer interaction is started and the human-computer interaction resources are allocated.
[0076] In this embodiment, the monitoring of the human-computer interaction state includes:
[0077] Detecting whether the vehicle state is a non-parking state;
[0078] If yes, the human-computer interaction interface object switched by the user is dynamically monitored according to the seat weighing state information and the visual sensing user riding state information;
[0079] According to the dynamic monitoring of the human-computer interaction interface object switched by the user, the process of the service vehicle user human-computer interaction is dynamically switched and the human-computer interaction resource allocation is dynamically switched.
[0080] Specifically, in one specific embodiment, the intelligent cabin includes multiple screens (human-computer interaction interface objects), and one person can interact with one of the multiple screens. With respect to the screen interaction online, if the user management service detects that the occupant in a certain riding area leaves and enters a stable state, the process and resources of the corresponding user of the area (the seat entertainment screen) are dynamically released, such as closing the screen, notifying the application process to end the business and exit, releasing the corresponding storage, process, window and other resources of the system and exiting the user active state.
[0081] In this embodiment, it also includes:
[0082] At least one human-computer interaction interface object is allocated according to the user riding state;
[0083] According to the process of the service vehicle user human-computer interaction and the allocation of the human-computer interaction resources, each human-computer interaction interface object activates the human-computer interaction with the user.
[0084] Specifically, in one specific embodiment, the intelligent cabin includes a central control screen, a co-driver screen, a ceiling screen, and an entertainment screen corresponding to each passenger. For example, the ceiling screen is equivalent to a public screen for the whole vehicle, and a passenger can interact with the entertainment screen of the seat or the ceiling screen. The change of the human-computer interaction interface object is equivalent to the change of the user interaction pairing with the central control screen, the co-driver screen, the ceiling screen, and the entertainment screen corresponding to each passenger. For example, although the rear passengers have their own entertainment screens, they can interact with the adjacent entertainment screen across the position. Through the camera vision passenger sensing algorithm of the OMS / DMS service, the selection and interaction of the occupant and the human-computer interaction interface object are captured, thereby starting the process of the service vehicle user human-computer interaction and allocating the human-computer interaction resources, and activating the human-computer interaction with the user.
[0085] In the embodiment, it also includes:
[0086] Starting the process of the service vehicle user human-computer interaction and allocating the human-computer interaction resources includes activating the dormant human-computer interaction interface object or starting the human-computer interaction interface object in the off state.
[0087] Starting the human-computer interaction interface object in the off state includes lighting the screen of the human-computer interaction interface object, pulling up the screen desktop process, initializing the preset business process, and providing the preset cabin interaction function.
[0088] Specifically, the temporary personnel converts the human-computer interaction interface object, and the original human-computer interaction interface object service process or part of the service process can be temporarily dormant, such as screen off. When the personnel returns to the original human-computer interaction interface object, the screen off can be quickly started. When the personnel changes the seat position to convert the human-computer interaction interface object, the human-computer interaction interface object on the original empty seat can be turned off, including ending the business of the corresponding notification application process and exiting, releasing the corresponding storage, process, window, and other resources of the system, and exiting the user active state. The state of the personnel converting the human-computer interaction interface object can be started or turned off.
[0089] Figure 2 It is a structure diagram of a multi-modal based vehicle-machine multi-user process resource management device provided by one or more embodiments of the present application.
[0090] As shown in Figure 2 The multi-modal based vehicle-machine multi-user process resource management device includes a multi-modal information acquisition module, a human-computer interaction state module, an interaction process state module, and a process resource management module.
[0091] The multi-modal information acquisition module is configured to acquire multi-modal recognition information based on a vehicle intelligent cabin.
[0092] The human-machine interaction status module is used to confirm the human-machine interaction status of the vehicle user based on multimodal recognition information.
[0093] The interaction process status module is used to control the process of serving the human-machine interaction of the vehicle user based on the human-machine interaction status of the vehicle user.
[0094] The process resource management module is used to control the resource management of the process that implements the human-machine interaction of the in-vehicle user based on the control of the process.
[0095] Confirming the human-machine interaction status of in-vehicle users includes confirming the generation of human-machine interaction needs, the verification of human-machine interaction conditions, and the monitoring of human-machine interaction status.
[0096] It is worth noting that although this system / device only discloses the multi-modal information acquisition module, the human-computer interaction status module, the interaction process status module, and the process resource management module, it does not mean that this device is limited to the above-mentioned basic functional modules. On the contrary, what this invention intends to express is that, based on the above-mentioned basic functional modules, those skilled in the art can arbitrarily add one or more functional modules in combination with the prior art to form an infinite number of embodiments or technical solutions. That is to say, this system / device is open rather than closed. It should not be assumed that the scope of protection of the claims of this invention is limited to the above-disclosed basic functional modules just because this embodiment only discloses a few basic functional modules.
[0097] In one specific embodiment, such as Figure 3 The cockpit system startup steps shown in the multi-user scenario are as follows: First, all human-machine interface objects are started. Then, based on the position status of passengers getting in and out of the vehicle, or the selection and matching status between passengers and human-machine interface objects, unused human-machine interface objects (entertainment screen, passenger screen, etc.) are turned off.
[0098] This embodiment helps reduce response latency, ensuring that passengers at the start of boarding are in a state with abundant interactive resources.
[0099] In another specific embodiment, a multi-user process management method for intelligent cockpits based on multimodal recognition is disclosed. The goal is to optimize the multi-user process management strategy of existing in-vehicle systems by leveraging multimodal recognition. Based on the actual passenger and usage patterns in each passenger area, the lifecycle management of multi-user processes and resources is dynamically performed. This improves the performance and energy consumption of current in-vehicle systems, enhances the smoothness of user cockpit interaction, and simultaneously implements a low-carbon, green, and energy-saving design concept.
[0100] Specifically, it includes:
[0101] Assume that the current passenger users, user IDs, and display screens have the corresponding relationships shown in Table 1:
[0102] Table 1
[0103] User name User number Ride area Screen Driver user user10 Main driver area Center screen Co-driver user user11 Co-driver area Co-driver screen Rear user user12 Rear area Curtain screen
[0104] Step one: In the user management service module of the car machine system, the CAN signal listening of the car door switch and the seat in place is added.
[0105] Step two: When the car machine enters the power-on and wake-up state, the user management service only initializes the driver user by default, and notifies the process management module, package management module and window management module and other system services to allocate resources.
[0106] Step three: After the user management service listens to the car door switch and seat load signal, the orientation source and specific information of the signal are judged. If there is a possibility of passenger change, the camera visual passenger sensing algorithm of the OMS / DMS (passenger monitoring system / driver monitoring system) service is started to confirm whether there is a passenger seating state change.
[0107] Step four: If the user management service detects that a passenger in a certain seating area joins and enters a stable state (maintains seating for a period of time, or enters a driving state, etc.), the initialization and resource allocation of the user corresponding to the area are dynamically triggered. The corresponding area screen is turned on, the default desktop process is pulled up, the necessary business process is initialized, and the cabin interaction function is provided for the corresponding area passenger.
[0108] Step five: If the user management service detects that a passenger in a certain seating area leaves and enters a stable state (maintains leaving for a period of time, or enters a driving state, etc.).
[0109] The process and resources of the user corresponding to the area are dynamically released, such as turning off the screen, notifying the application process to end the business and exit, releasing the corresponding storage, process, window and other resources of the system and exiting the user active state.
[0110] The above technical solution includes multi-user application process, system resource management, and the module relationship of the whole vehicle door, seat domain controller and visual sensing service, as shown in the intelligent cabin function module relationship. Figure 4
[0111] Based on the above embodiment, a further embodiment of an application scenario is disclosed:
[0112] Suppose a vehicle is equipped with a car machine system that implements the present solution. The vehicle owner starts the vehicle, thereby triggering the power-on of the car machine system, and the process begins.
[0113] 1. The car machine system initializes each service of the system, including user management service and process management, package management, window management, storage management, signal service, OMS / DMS visual algorithm service, and lights up the main driver screen.
[0114] 2. The user management service performs default initialization and resource allocation for the main driver user user10, including notifying the process management, package management, window management, storage management, and other services to allocate corresponding resources. At the same time, the main driver desktop application process is pulled up, and the application is notified to start the necessary main driver application process.
[0115] 3. The user management service registers signal service listening, listens to the door signals and seat in-place signals of each occupant area, and the driving state signals.
[0116] 4. Assuming that the car owner's family enters the co-driver and sits down at this time. The door area controller will send the door opening and closing signal, and the car machine system signal service will call the listening callback of the user management service after receiving it. Immediately afterwards, the seat area controller will send the seat in-place signal to the car machine system. The user management service receives two groups of signals in succession, and can preliminarily judge that the co-driver has an occupant, and then starts the DMS visual service algorithm to accurately judge whether there is a person in the co-driver.
[0117] 5. The DMS visual service algorithm starts the camera and performs visual data processing to determine that there is a person in the co-driver, and then notifies the user management service of the judgment result.
[0118] 6. The user management service waits for a reasonable period of time, while monitoring the driving state signal. If there is no new door signal and seat in-place signal change within that period of time, or the vehicle enters the driving state signal is received. Then start the initialization and resource allocation of the co-driver user user11, including notifying the process management, package management, window management, storage management, and other services to allocate corresponding resources. At the same time, the co-driver desktop application process is pulled up, and the application is notified to start the necessary co-driver application process. The co-driver screen is turned on. At this point, the car machine system in the co-driver occupant area is ready to serve the co-driver user.
[0119] 7. Assuming that the car owner sends the car owner's family to the destination, and the car owner's family gets off the car. Similar to the above steps, the user management service will receive the co-driver in-place signal and door opening and closing signal change callback, start the DMS visual algorithm service to judge that the co-driver has no passenger, and after waiting for a reasonable period of time, the user management service will execute the user11 related application process end notification, turn off the co-driver screen, notify the process management, package management, window management, storage management, and other services to release the related system resources, and end the life cycle of the user11 related process and resources.
[0120] 8、Assuming that the car owner then receives a hitchhiking order and picks up a passenger, the passenger opens the door and sits in the rear area, following the process described earlier, the user management service will initialize user12 after the multi-modal condition is determined, allocate relevant system resources, notify the rear application to start, and open the rear curtain screen. At this point, the rear passenger area infotainment system is ready to serve the rear passenger.
[0121] 9、After the hitchhiking passenger arrives at the destination and gets off, following the process described earlier, the user management service will close the rear curtain screen after the multi-modal condition is determined, release the user12 system resources, notify the rear application to end the process, and thus end the life cycle of user12.
[0122] In the above implementation process, through multi-modal dynamic user management, the infotainment system has at most two passenger area user processes and resource concurrency at the same time, compared with traditional static user management, greatly improving the performance and energy consumption of the infotainment system.
[0123] Figure 5 is an electronic device structure block diagram provided by one or more embodiments of the application based on a multi-modal infotainment system multi-user process resource management method.
[0124] As shown in Figure 5 , the present application provides an electronic device, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus;
[0125] The memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of a multi-modal infotainment system multi-user process resource management method.
[0126] The present application also provides a computer readable storage medium storing a computer program executable by an electronic device, which, when the computer program runs on the electronic device, causes the electronic device to execute the steps of a multi-modal infotainment system multi-user process resource management method.
[0127] The present application also provides a vehicle, comprising:
[0128] An electronic device for implementing the steps of a multi-modal infotainment system multi-user process resource management method;
[0129] A processor, the processor running a program, when the program runs, the data output from the electronic device executes the steps of a multi-modal infotainment system multi-user process resource management method;
[0130] A storage medium for storing a program, the program executing the steps of a multi-modal infotainment system multi-user process resource management method when the data output from the electronic device runs.
[0131] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus can be divided into an address bus, a data bus, a control bus, and the like. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.
[0132] The electronic device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory. The operating system can be any one or more computer operating systems that implement electronic device control through processes, such as a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system, or a windows operating system. In embodiments of the present application, the electronic device can be a handheld device such as a smartphone or a tablet computer, or an electronic device such as a desktop computer or a portable computer, and is not particularly limited in embodiments of the present application.
[0133] The execution subject of the electronic device control in embodiments of the present application can be an electronic device, or a functional module capable of calling and executing a program in the electronic device. The electronic device can obtain a firmware corresponding to the storage medium, and the firmware corresponding to the storage medium is provided by a supplier. The firmware corresponding to different storage media can be the same or different, and is not limited herein. After the electronic device obtains the firmware corresponding to the storage medium, the electronic device can write the firmware corresponding to the storage medium into the storage medium, specifically, burn the firmware corresponding to the storage medium into the storage medium. The process of burning the firmware into the storage medium can be implemented by using existing technology, and is not described in detail in embodiments of the present application.
[0134] The electronic device can also obtain a reset command corresponding to the storage medium, and the reset command corresponding to the storage medium is provided by a supplier. The reset command corresponding to different storage media can be the same or different, and is not limited herein.
[0135] At this time, the storage medium of the electronic device is the storage medium on which the corresponding firmware is written, and the electronic device can respond to the reset command corresponding to the storage medium in the storage medium on which the corresponding firmware is written, so that the electronic device resets the storage medium on which the corresponding firmware is written according to the reset command corresponding to the storage medium. The process of resetting the storage medium according to the reset command can be implemented in the prior art, and will not be described in detail in the embodiments of the present application.
[0136] For the convenience of description, the above apparatus is described as various units and modules in terms of functions. Of course, the functions of the units and modules can be implemented in one or more software and / or hardware in the implementation of the present application.
[0137] Those skilled in the art can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs. It should also be understood that terms such as those defined in a general dictionary should be understood in the context of the present application and should not be interpreted in an idealized or overly formal sense unless specifically defined.
[0138] For the convenience of description, the above apparatus is described as various units and modules in terms of functions. Of course, the functions of the units and modules can be implemented in one or more software and / or hardware in the implementation of the present application.
[0139] From the above description of the embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and the necessary general hardware platform. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server or a network device, etc.) execute the methods described in the various embodiments or some parts of the embodiments of the present application.
[0140] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A multimodal vehicle-mounted multi-user process resource management method, characterized in that, The multimodal vehicle-mounted multi-user process resource management method includes: Based on the vehicle's intelligent cockpit, acquire multimodal recognition information; Based on multimodal recognition information, the human-machine interaction status of the in-vehicle user is confirmed; Control the process of human-machine interaction based on the human-machine interaction status of the in-vehicle user; Based on the process of controlling the human-machine interaction of the in-vehicle user, control is based on the resource management that enables the human-machine interaction of the in-vehicle user. Confirming the human-machine interaction status of in-vehicle users includes confirming the generation of human-machine interaction needs, the verification of human-machine interaction conditions, and the monitoring of human-machine interaction status.
2. The vehicle-mounted multi-user process resource management method based on multimodal operation according to claim 1, characterized in that, The generation of the human-computer interaction requirements includes: Check if the vehicle is in a parked state; If yes, then obtain the status information of the car door opening and seat weighing; Based on the status information of the car door opening and seat weighing, information for human-computer interaction needs is generated.
3. The vehicle-mounted multi-user process resource management method based on multimodal operation according to claim 1, characterized in that, The verification of the human-computer interaction conditions includes: Check if the vehicle is in a non-parked state; If so, then obtain the user's riding status information based on the seat weighing status information; Acquire visually perceived information about the user's riding status; Based on the visual perception of the user's riding status information, verify whether the conditions for initiating human-computer interaction are met under the user's riding status based on seat weight. If so, then for the user in the riding state, start the process of serving the in-vehicle user human-computer interaction and allocate human-computer interaction resources.
4. The vehicle-mounted multi-user process resource management method based on multimodal operation according to claim 1, characterized in that, The monitoring of the human-computer interaction status includes: Check if the vehicle is in a non-parked state; If so, then based on the seat weighing status information and the visually perceived user riding status information, dynamically monitor the human-computer interaction interface objects switched by the user. Based on the dynamic monitoring of the human-machine interface objects switched by the user, the process of serving the vehicle user's human-machine interaction is dynamically switched and the resource allocation of the human-machine interaction is dynamically switched.
5. The in-vehicle multi-user process resource management method based on multimodal operation according to any one of claims 1 to 4, characterized in that, Also includes: For each user's riding status, at least one human-computer interaction interface object should be assigned; Each human-computer interaction interface object activates human-computer interaction with the user based on the progress of the in-vehicle user human-computer interaction service and the allocation of human-computer interaction resources.
6. The vehicle-mounted multi-user process resource management method based on multimodal operation according to claim 5, characterized in that, Also includes: Starting the process of serving the in-vehicle user human-machine interaction and allocating human-machine interaction resources includes activating a dormant human-machine interaction interface object or starting a human-machine interaction interface object that is in a closed state. Starting a closed human-computer interaction interface object includes turning on the screen of the human-computer interaction interface object, launching the screen desktop process, initializing the preset business process, and providing the preset cockpit interaction functions.
7. A vehicle-mounted multi-user process resource management device based on multimodal operation, characterized in that, The multimodal vehicle-mounted multi-user process resource management device includes: The multimodal information acquisition module is used to acquire multimodal recognition information based on the vehicle's intelligent cockpit; The human-machine interaction status module is used to confirm the human-machine interaction status of the vehicle user based on multimodal recognition information; The interaction process status module is used to control the process of serving the human-machine interaction of the vehicle user based on the human-machine interaction status of the vehicle user. The process resource management module is used to control the resource management of the process that implements the human-machine interaction service for in-vehicle users, based on the control of the process. Confirming the human-machine interaction status of in-vehicle users includes confirming the generation of human-machine interaction needs, the verification of human-machine interaction conditions, and the monitoring of human-machine interaction status.
8. An electronic device, characterized in that, include: The processor, communication interface, memory, and communication bus are connected, with the processor, communication interface, and memory communicating with each other via the communication bus. The memory stores a computer program that, when executed by a processor, causes the processor to perform the steps of the multimodal vehicle-mounted multi-user process resource management method as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The device stores a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of the multimodal vehicle-mounted multi-user process resource management method as described in any one of claims 1 to 6.
10. A vehicle, characterized in that, include: An electronic device for implementing the steps of the multimodal vehicle-mounted multi-user process resource management method as described in any one of claims 1 to 6; The processor runs a program, and when the program runs, it executes the steps of the multimodal vehicle-mounted multi-user process resource management method as described in any one of claims 1 to 6 from data output by the electronic device. A storage medium for storing a program that, when running, performs the steps of the multimodal vehicle-mounted multi-user process resource management method as described in any one of claims 1 to 6 on data output from an electronic device.