Interaction method, apparatus and vehicle

Abstract

The application discloses an interaction method, device and vehicle, and the method comprises the following steps: acquiring first indication information, wherein the first indication information indicates that first type information of a first application program is viewed, and the first application program is any application program deployed in a first terminal; viewing the first application program according to the first indication information; and when the first type information is detected, controlling a sound production device associated with the first terminal to prompt first information, wherein the first information is used for prompting the first type information. The technical solution can be applied to the field of human-computer interaction related to intelligent vehicles such as electric vehicles and new energy vehicles, and helps to improve the timeliness of information acquisition of a user on an application program under the premise of ensuring driving safety, so that the driving and riding experience and the driving safety are taken into account.

Description

Technical Field

[0001] This application relates to the field of human-computer interaction, and more specifically, to an interaction method, device, and vehicle. Background Technology

[0002] With the development of vehicle intelligence, people have higher and higher requirements for driving experience and convenience of car use, resulting in more and more applications installed in the car's infotainment system, which has enriched the user's driving experience to a certain extent.

[0003] However, under the current technological conditions, the way in-vehicle systems push messages is not intelligent, which will affect the user experience of drivers and passengers, and even affect driving safety.

[0004] Therefore, an interactive solution that can balance driving experience and driving safety is urgently needed to be developed. Summary of the Invention

[0005] This application provides an interaction method, device, and vehicle that helps improve the timeliness of a user's access to information from an application while ensuring driving safety, thereby balancing driving experience and driving safety.

[0006] Firstly, an interaction method is provided that can be applied to terminals such as vehicles, for example, it can be executed by chips or circuits installed in the vehicle.

[0007] The method includes: obtaining first instruction information, the first instruction information instructing the viewing of a first type of information of a first application, the first application being any application deployed in a first terminal; viewing the first application according to the first instruction information; and when the first type of information is detected, controlling a sound-emitting device associated with the first terminal to prompt first information, the first information being used to prompt the first type of information.

[0008] The process of viewing the first application based on the first instruction information may include: watching the first application for an extended period of time or periodically checking the first application to determine whether the first application contains the first type of information.

[0009] In some implementations, the first instruction information may instruct the first application to protect or safeguard the first type of information. The aforementioned "protection" or "guarding" can be understood as: viewing or detecting the first application to determine whether the first type of information exists. The aforementioned "viewing" may be viewing for a long time until the first type of information is detected, and the aforementioned "detection" may be periodic detection until the first type of information is detected.

[0010] By the technical solution, for the information that the user wants to pay attention to, the user does not need to frequently refresh the application, and the time of the user can be saved. In addition, the voice device pushes the related information of the application program that the user pays attention to to the user, which helps to minimize the influence of the first information push on the attention concentration of the user, and when the user is driving, the timeliness of the user obtaining information can be improved under the premise of ensuring driving safety, so as to balance the driving experience and driving safety.

[0011] With reference to the first aspect, in some implementations of the first aspect, the first indication information is obtained by: obtaining a first voice instruction of the first user, the first voice instruction indicating to view the first type of information; and generating the first indication information according to the first voice instruction.

[0012] In some implementations, the first voice instruction can also indicate the first application program.

[0013] In the technical solution, the user can be assisted in viewing the information of a certain type in the application program that the user pays attention to in response to the voice instruction of the user, the user can be helped to avoid manually clicking the application program to view the information, the hands of the user can be freed, the user can be helped to concentrate on driving during driving, and the convenience and safety of the user during driving can be improved.

[0014] With reference to the first aspect, in some implementations of the first aspect, the method further includes: determining the first terminal according to the position information and / or the identity information of the first user, the position information indicating a position of the first user in a cabin of a vehicle, the vehicle including the first terminal; and determining the first application program according to the first terminal and the first voice instruction.

[0015] In this implementation, the identity information includes biological feature information and / or account information of the user, and the identity information is used to determine the first terminal that logs in the account of the first user.

[0016] By the technical solution, the target terminal can be automatically determined according to the position of the user in the cabin of the vehicle or the identity information, the terminal on which the first application program is deployed does not need to be specified by the user through a voice instruction or the like, the reach path of the user triggering the viewing of the first type of information can be shortened, and the interaction steps can be simplified, thereby improving the use experience of the user.

[0017] With reference to the first aspect, in some implementations of the first aspect, the method further includes: determining the first application program according to the first voice instruction; and determining the first terminal according to the first application program and the position information and / or the identity information of the first user, the position information indicating a position of the first user in a cabin of a vehicle, the vehicle including the first terminal.

[0018] In the implementation, the identity information includes biometric information and / or account information of the user, and the identity information is used to determine a first terminal on which a first application program of a first user is deployed.

[0019] According to the technical solution, the first application program can be automatically determined according to the voice instruction of the user, and then the target terminal on which the first application program is deployed is determined. In this way, the user can trigger the process of taking care of the first type of information through the fuzzy voice instruction, which helps to reduce the learning cost of the user learning to use the application solution, shorten the triggering path of the user triggering the process of taking care of the first type of information, and thus improve the user experience.

[0020] In combination with the first aspect, in some implementations of the first aspect, the viewing of the first application program according to the first indication information includes: viewing a first interface of the first terminal according to the first indication information, the first interface being an interface that is virtually displayed in a virtual screen of the first terminal, and the first interface being used to carry content displayed when the first application program runs.

[0021] In the technical solution, whether the first type of information is detected is determined through the interface of the first application program displayed in the virtual screen. On the one hand, based on the background virtual screen operation, the frequent refreshing process is not presented to the user, which can avoid distracting the user's attention. On the other hand, the completeness of the first type of information obtained (the message notification obtained through the system application program interface cannot guarantee the completeness of the first type of information) can be improved, thereby improving the detection rate of the first type of information. On the other hand, it is helpful to expand the application range of the application solution to any application. As long as the first type of information can be displayed through the interface of the first application program, the first type of information can be taken care of through the application solution.

[0022] In combination with the first aspect, in some implementations of the first aspect, the viewing of the first application program according to the first indication information includes: inputting the first indication information into a first machine learning model, so that the first machine learning model outputs a first operation instruction, the first operation instruction being used to instruct to open a first interface; in response to the first operation instruction, the first interface is opened; at least one frame of image of the first interface is obtained; the at least one frame of image and the first indication information are input into a second machine learning model, so that the second machine learning model outputs interface understanding information of each frame of image in the at least one frame of image, the interface understanding information indicating whether the first type of information is detected.

[0023] In some implementations, the machine learning model can include a deep learning model based on one or more of a Transformer neural network, a convolutional neural network, a recurrent neural network, or can also include a vision large language model (VLLM), or can also include other types of machine learning models.

[0024] In actual implementation, the functions of the first machine learning model and the second machine learning model can be implemented by the same machine learning model.

[0025] Through the above technical solutions, the processing speed of the indication information obtained based on the user voice instruction can be improved, and the understanding speed and accuracy of the interface content in the virtual screen can be improved, so that the user can realize the care of the first type of information without performing a touch screen operation, which helps to improve the sense of technology that the user experiences in the process, thereby improving the user's use experience.

[0026] In combination with the first aspect, in some implementations of the first aspect, the method further includes: obtaining a second voice instruction, the second voice instruction indicating to perform a first operation on the first type of information; and performing, according to the second voice instruction, a second operation associated with the first operation on the first interface.

[0027] Through the above technical solutions, the first operation of the first type of information can be automatically implemented by the user without feeling based on the user's voice instruction, which avoids too much distraction of the user's attention, and if the user is in the process of driving, helps to improve the user's driving convenience and safety.

[0028] In combination with the first aspect, in some implementations of the first aspect, performing, according to the second voice instruction, a second operation associated with the first operation on the first interface includes: inputting second indication information to a third machine learning model to make the third machine learning model output a second operation instruction, the second indication information being obtained based on the second voice instruction, the second operation instruction being used to indicate at least one set of operation information, the operations indicated by the at least one set of operation information constituting the second operation, each set of operation information in the at least one set of operation information including an operation event type and an operation position, the operation position indicating a coordinate in the first interface, and the operation event type indicating an operation performed in the first interface; and performing the operation indicated by the second operation instruction in the first interface.

[0029] In some implementations, the functions of the third machine learning model and the aforementioned first machine learning model and / or the second machine learning model can be implemented by the same machine learning model.

[0030] By the technical solution, the first operation on the first type of information can be completed without the user performing a touch screen operation, which helps to improve the convenience of the user.

[0031] With reference to the first aspect, in some implementations of the first aspect, the first application program is a communication application program, the first type of information is information received through the first application program, and the first operation and the second operation are operations for replying to the first type of information.

[0032] In some implementations, the communication application program includes an instant messaging application program, or can also include an application program such as an email.

[0033] With reference to the first aspect, in some implementations of the first aspect, the first application program is a video application program, the first type of information is update information of a first video played through the first application program, and the first operation and the second operation are operations for controlling the playing of the first video.

[0034] In some implementations, the video application program can include a long video application program, in which case the first video can include a video related to a television series or a movie; or the video application program can also include a short video application program, in which case the first video can be a video updated by a certain blogger.

[0035] With reference to the first aspect, in some implementations of the first aspect, the second operation is further used to control the visual interface of the first terminal to display a running interface of the first application program.

[0036] The visual interface refers to an interface that can be viewed by the user.

[0037] By the technical solution, the first terminal can play a video that the user expects to see without the user manually operating the first terminal, which helps to improve the convenience of the user using the first terminal.

[0038] With reference to the first aspect, in some implementations of the first aspect, the first information is further used to inquire whether to perform a further operation on the first type of information.

[0039] In the technical solution, the further operation can be an automatically suggested further operation, and the user is prompted by the first information to suggest an operation on the first type of information, which helps to help the user optimize the further operation, thereby further improving the convenience of the user.

[0040] With reference to the first aspect, in some implementations of the first aspect, the first terminal is applied in a vehicle, and the method further includes: obtaining display content of a visual interface of the first terminal; and controlling a sound producing device associated with the first terminal to prompt the first information, including: when the visual interface does not display a running interface of the safety-critical application, controlling the sound producing device to prompt the first information.

[0041] In the above technical solution, when the visual interface does not display the safety-critical application, it can be preliminarily judged that the vehicle is not currently in a manual driving state or a man-machine co-driving (i.e., manual and intelligent driving system collaborative vehicle control) state, at this time, controlling the sound producing device to prompt the first information has less impact on the driver, even if it can distract the driver's attention, it will not affect the driving safety of the vehicle, and it is helpful to ensure the timeliness of the user obtaining the information of interest.

[0042] With reference to the first aspect, in some implementations of the first aspect, the first terminal is applied in a vehicle, and the method further includes: obtaining speed information, the speed information indicating a driving speed of the vehicle; and controlling a sound producing device associated with the first terminal to prompt the first information, including: when the driving speed is less than or equal to a speed threshold, controlling the sound producing device to prompt the first information.

[0043] In some implementations, the speed threshold can be one of 3 m / s to 5 m / s, or the speed threshold can also be other values.

[0044] In the above technical solution, when the vehicle speed is slow or the vehicle is in a parking state, the prompted first information may even distract the driver's attention, but has little or even no impact on the driving safety of the vehicle, and can ensure that the user obtains the information of interest in time.

[0045] With reference to the first aspect, in some implementations of the first aspect, the first terminal is applied in a vehicle, the first indication information is triggered by a first user, and the first user is a user located at a driving position of the vehicle, and the method further includes: obtaining driving mode information, the driving mode information indicating a driving mode in which the vehicle is located; and controlling a sound producing device associated with the first terminal to prompt the first information, including: when the vehicle is not in a first driving mode, controlling the sound producing device to prompt the first information; wherein the first driving mode is a mode in which the operation of the driver affects the driving safety of the vehicle.

[0046] In the above technical solution, when the vehicle is in a driving mode that does not require human participation, the prompted first information may even distract the driver's attention, but will not affect the driving safety of the vehicle, so as to help improve the timeliness of the user obtaining information while ensuring the driving safety of the vehicle.

[0047] In conjunction with the first aspect, in some implementations of the first aspect, the first terminal is applied in a vehicle, and the sound-emitting device associated with the first terminal is controlled to prompt the first information, including: when the first terminal is associated with a non-safety-critical screen and the first type of information is detected, the sound-emitting device is controlled to prompt the first information.

[0048] The first terminal is associated with a non-safety-critical screen, which can be understood as: the display screen included in the first terminal is a non-safety-critical screen.

[0049] In this implementation, the first terminal can be determined based on a mapping relationship between the display screen and the cabin area, wherein the mapping relationship indicates the mapping relationship between the area where the user is located in the vehicle's cabin and the display screen included in the terminal. For example, when the first user is in the driver's seat area, the aforementioned mapping relationship indicates that the central control screen is the display screen of the first terminal deploying the first application; as another example, when the first user is in the passenger seat area, the aforementioned mapping relationship indicates that the passenger entertainment screen is the display screen of the first terminal deploying the first application.

[0050] In the above technical solution, when the first terminal is a non-safety-critical screen, it can be determined that the user triggering the first instruction information is not the driver. In this case, the target person of the first information is not the driver, so the impact of the first information on the driver is generally small. Therefore, it helps to ensure that the user triggering the first instruction information receives the first information in a timely manner while ensuring vehicle safety. In addition, the first information can be broadcast through a sound device in the area where the user triggering the first instruction information is located. This reduces the impact of the first information on the driver and also ensures the timeliness of the user's access to the first information.

[0051] In conjunction with the first aspect, in some implementations of the first aspect, the first indication information is triggered by the first user, and the method further includes: obtaining user status information, the user status information indicating the current status of the first user; controlling the sound-emitting device associated with the first terminal to prompt the first information, including: controlling the sound-emitting device to prompt the first information when the user status information indicates that the first user is in a non-sleep state.

[0052] By employing the above technical solutions, the likelihood of users being startled or disturbed when receiving the first information prompt can be reduced, thereby improving the user's comfort when using the first terminal.

[0053] In a second aspect, an interaction apparatus is provided, which comprises an acquisition unit and a processing unit, wherein the acquisition unit is configured to acquire first indication information, the first indication information indicating that first type information of a first application is to be viewed, the first application being any application deployed in a first terminal; and the processing unit is configured to view the first application according to the first indication information, and control a sound generating apparatus associated with the first terminal to prompt first information when the first type information is detected, the first information being used to prompt the first type information.

[0054] With reference to the second aspect, in some implementations of the second aspect, the acquisition unit acquiring the first indication information can be refined as: the acquisition unit acquires a first voice instruction of a first user, the first voice instruction indicating that the first type information is to be viewed; and the processing unit generates the first indication information according to the first voice instruction.

[0055] With reference to the second aspect, in some implementations of the second aspect, the processing unit is further configured to determine the first terminal according to position information and / or identity information of the first user, the position information indicating a position of the first user in a cabin of a vehicle, the vehicle comprising the first terminal, and determine the first application according to the first terminal and the first voice instruction.

[0056] With reference to the second aspect, in some implementations of the second aspect, the processing unit is further configured to determine the first application according to the first voice instruction, and determine the first terminal according to the first application and the position information and / or the identity information of the first user, the position information indicating a position of the first user in a cabin of a vehicle, the vehicle comprising the first terminal.

[0057] With reference to the second aspect, in some implementations of the second aspect, the processing unit is further configured to view a first interface of the first terminal according to the first indication information, the first interface being an interface that is virtually displayed in a virtual screen of the first terminal, and the first interface being used to carry content displayed when the first application runs.

[0058] With reference to the second aspect, in some implementations of the second aspect, the processing unit further comprises: inputting the first indication information to a first machine learning model, so that the first machine learning model outputs a first operation instruction, the first operation instruction being used to instruct to open the first interface; in response to the first operation instruction, opening the first interface; controlling the acquisition unit to acquire at least one frame of image of the first interface; inputting the at least one frame of image and the first indication information to a second machine learning model, so that the second machine learning model outputs interface understanding information of each frame of image in the at least one frame of image, the interface understanding information indicating whether the first type information is detected.

[0059] With reference to the second aspect, in some implementations of the second aspect, the obtaining unit is further configured to: obtain a second voice instruction, the second voice instruction indicating to perform a first operation on the first type of information; and the processing unit is further configured to: perform, according to the second voice instruction, a second operation associated with the first operation on the first interface.

[0060] With reference to the second aspect, in some implementations of the second aspect, the processing unit is further configured to: input, to a third machine learning model, second indication information to cause the third machine learning model to output a second operation instruction, the second indication information being obtained based on the second voice instruction, the second operation instruction being used to indicate at least one set of operation information, the operations indicated by the at least one set of operation information constituting the second operation, each set of operation information in the at least one set of operation information including an operation event type and an operation position, the operation position indicating a coordinate in the first interface, and the operation event type indicating an operation performed in the first interface; and perform, in the first interface, the operations indicated by the second operation instruction.

[0061] With reference to the second aspect, in some implementations of the second aspect, the first application program is a communication application program, the first type of information is information received through the first application program, and the first operation and the second operation are operations for replying to the first type of information.

[0062] With reference to the second aspect, in some implementations of the second aspect, the first application program is a video application program, the first type of information is update information of a first video played through the first application program, and the first operation and the second operation are operations for controlling playing of the first video.

[0063] With reference to the second aspect, in some implementations of the second aspect, the second operation is further used to control the visual interface of the first terminal to display a running interface of the first application program.

[0064] With reference to the second aspect, in some implementations of the second aspect, the first information is further used to inquire whether to perform a further operation on the first type of information.

[0065] With reference to the second aspect, in some implementations of the second aspect, the first terminal is applied in a vehicle, the obtaining unit is further configured to: obtain display content of a visual interface of the first terminal; and the processing unit is configured to: control the sound producing device to prompt the first information when the visual interface does not display a running interface of a safety-critical application program.

[0066] With reference to the second aspect, in some implementations of the second aspect, the first terminal is applied in a vehicle, the obtaining unit is further configured to: obtain speed information indicating a driving speed of the vehicle; and the processing unit is further configured to: control the sound producing device to prompt the first information when the driving speed is less than or equal to a speed threshold.

[0067] In some implementations of the second aspect, the first terminal is applied in a vehicle, the first indication information is triggered by a first user, and the first user is a user located at a driving position of the vehicle. The obtaining unit is further configured to obtain driving mode information, the driving mode information indicating a driving mode in which the vehicle is located. The processing unit is configured to control the sound emitting device to prompt the first information when the vehicle is not in a first driving mode. The first driving mode is a mode in which an operation of a driver affects driving safety of the vehicle.

[0068] In some implementations of the second aspect, the first terminal is applied in a vehicle, and the processing unit is configured to control the sound emitting device to prompt the first information when the first terminal is associated with a non-safety-critical screen and the first type of information is detected.

[0069] In some implementations of the second aspect, the first indication information is triggered by a first user. The obtaining unit is further configured to obtain user state information, the user state information indicating a current state of the first user. The processing unit is configured to control the sound emitting device to prompt the first information when the user state information indicates that the first user is in a non-sleep state.

[0070] In a third aspect, an interaction apparatus is provided. The apparatus includes a processor configured to execute a computer program stored in a memory to cause the apparatus to perform the method in any possible implementation of the first aspect.

[0071] In some implementations of the third aspect, the apparatus further includes the memory.

[0072] In a fourth aspect, a computer program product is provided. The computer program product includes computer program codes. When the computer program codes are executed on a computer or a processor, the computer or the processor performs the method in any possible implementation of the first aspect.

[0073] It should be noted that the computer program codes can be stored in whole or in part on a storage medium. The storage medium can be packaged together with the processor or packaged separately from the processor.

[0074] In a fifth aspect, a computer readable storage medium is provided. The computer readable medium stores instructions. When the instructions are executed by a processor, the processor implements the method in any possible implementation of the first aspect.

[0075] In a sixth aspect, a chip is provided. The chip includes a circuit configured to perform the method in any possible implementation of the first aspect.

[0076] In a seventh aspect, a vehicle is provided, which comprises the apparatus in any possible implementation manner of the second aspect or the third aspect, or the vehicle comprises the computer readable storage medium in any possible implementation manner of the fifth aspect, or the vehicle comprises the chip in any possible implementation manner of the sixth aspect, or the vehicle is loaded with the computer program product in any possible implementation manner of the fourth aspect.

[0077] In connection with the seventh aspect, in some implementations of the seventh aspect, the vehicle can include a vehicle in a broad sense, for example, can be a vehicle (such as a commercial vehicle, a passenger vehicle, a motorcycle, a flying vehicle, a train, etc.), an industrial vehicle (such as a forklift, a trailer, a tractor, etc.), an engineering vehicle (such as an excavator, a bulldozer, a crane, etc.), an agricultural device (such as a mower, a harvester, etc.), a recreational device, a toy vehicle, etc. In actual implementation, the vehicle can also be a road vehicle, a water vehicle, an air vehicle, an industrial device, an agricultural device, or an entertainment device, or other intelligent driving devices.

[0078] The beneficial effects not described in detail in the second aspect to the seventh aspect can be referred to the description in the first aspect, which will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0079] Figure 1 is a functional schematic block diagram of a vehicle provided by an embodiment of the present application;

[0080] Figure 2 is a schematic block diagram of an interaction system architecture provided by an embodiment of the present application;

[0081] Figure 3 is a schematic flowchart of an interaction method provided by an embodiment of the present application;

[0082] Figure 4 is a schematic diagram of an application scenario related to an embodiment of the present application;

[0083] Figure 5 is another schematic diagram of an application scenario related to an embodiment of the present application;

[0084] Figure 6 is still another schematic flowchart of an interaction method provided by an embodiment of the present application;

[0085] Figure 7 is a schematic block diagram of a control device provided by an embodiment of the present application;

[0086] Figure 8 is still another schematic block diagram of a control device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0087] The technical solutions in the present application will be described below with reference to the drawings.

[0088] Figure 1 is a functional block diagram of a vehicle provided by an embodiment of the present application. As shown in Figure 1 the vehicle 100 can include a perception system 120, an interaction system 130, and a computing platform 150, wherein the perception system 120 can include a plurality of sensors for sensing the surrounding environment information of the vehicle 100. For example, the perception system 120 can include a positioning system, which can be a global positioning system (GPS), a Beidou system, or other positioning systems. For another example, the perception system 120 can further include one or more of an inertial measurement unit (IMU), a laser radar, a millimeter wave radar, an ultrasonic radar, and a camera.

[0089] The camera can include one or more cameras disposed in the vehicle cabin for capturing images inside or outside the cabin, such as the camera of a driver monitor system (DMS), the camera of a cabin monitor system (CMS), and the camera of a dashcam. The camera for capturing images inside and outside the cabin can be the same camera or different cameras. In addition, the perception system 120 can further include one or more pressure sensors and sound wave sensors disposed in the cabin for monitoring whether there is a user inside the cabin and the position of the user.

[0090] The interaction system 130 comprises a device for receiving user instructions and a prompting device for prompting information to the user. The device for receiving user instructions can comprise at least one of: a sound receiving device, such as a microphone, a microphone, a microphone, etc., for receiving voice instructions of the user; or a device for receiving instructions input by the user through the screen, such as a human machine interface (HMI), etc.; or a camera device for receiving instructions such as human body gestures of the user, such as a camera in the cockpit, etc. The prompting device can include but is not limited to: a sound emitting device, a display device. More specifically, the sound emitting device can include a speaker, a sound device, etc. The display device is mainly divided into two categories: the first category is a vehicle-mounted display screen; the second category is a projection display screen, such as a head-up display (HUD). The vehicle-mounted display screen is a physical display screen and is an important part of the vehicle-mounted information entertainment system. It should be noted that the vehicle-mounted display screen can include an HMI. The head-up display, also known as a head-up display system, is mainly used to display driving information such as speed, navigation, etc. on a display device in front of the user (such as a windshield) to reduce the user's visual transfer time, avoid pupil changes caused by the user's visual transfer, and improve driving safety and comfort.

[0091] Some or all of the functionality of the vehicle 100 can be controlled by the computing platform 150. The computing platform 150 can include processors 151-15n, which are circuits that have the capability to process signals. In one implementation, the processors can be circuits that have the capability to fetch and execute instructions, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a kind of microprocessor), or a digital signal processor (DSP), etc. In another implementation, the processors can be circuits that implement certain functions through a fixed or reconfigurable logical relationship of hardware circuits, such as an application-specific integrated circuit (ASIC) or a programmable logic device (PLD) implemented hardware circuit, such as a field programmable gate array (FPGA). In a reconfigurable hardware circuit, the processor loads a configuration document to implement the configuration of the hardware circuit, which can be understood as the process of the processor loading instructions to implement related functions. In addition, the processor can also be a hardware circuit designed for artificial intelligence, which can be understood as a kind of ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc. In addition, the computing platform 150 can also include a memory for storing instructions, and some or all of the processors 151-15n can call the instructions in the memory to implement corresponding functions.

[0092] The following will be described in combination with Figure 2 The roles of the perception system 120, the interaction system 130, and the computing platform 150 in the present application will be specifically explained. Figure 2 A schematic block diagram of an interaction system architecture provided by an embodiment of the present application is shown. Specifically, Figure 2 The system shown includes an interaction module 210, a speech processing module 220, and an application daemon module 230. In some implementations, the system also includes a prompt timing determination module 240 and / or a perception module 250. The functions of the interaction module 210 can be implemented by Figure 1The functions of one or more devices in the illustrated interaction system 130, the speech processing module 220, the application daemon module 230, and the prompt opportunity determination module 240 can be respectively implemented by Figure 1 The functions of one or more processors in the illustrated computing platform 150, in some implementations, the application daemon module 230 can also be partially or fully implemented by processors in a cloud server associated with the vehicle; the functions in the perception module 250 can be implemented by one or more sensors in the perception system 120. The roles of the modules are described as follows (I) to (V):

[0093] (I) The interaction module 210 is configured to collect a voice instruction in the vehicle cabin and send the voice instruction to the speech processing module 220; in addition, the interaction module 210 can also prompt information to the user in the cabin according to the indication of the application daemon module 230 or the prompt opportunity determination module 240.

[0094] (II) The speech processing module 220 can process the voice instruction by using voice recognition algorithm, etc., convert the information in the form of voice into text information, and input the text information to the application daemon module 230.

[0095] (III) The application daemon module 230 includes a target application determination module 231, an application analysis and daemon module 232, and an interaction instruction processing module 233. Among them, the target application determination module 231 is configured to determine the target application program that needs to be daemoned (or monitored), the display device on which the target application program is deployed, and the type of information that needs to be daemoned for the target application program according to the text information. In some implementations, the target application determination module can determine the display device on which the target application program is deployed according to the information from the perception module 250. For example, according to the in-cabin perception information collected by the perception module 250, the position of the user who issues the voice instruction in the cabin is determined, and then the display device on which the target application program is deployed is determined according to the position of the user in the cabin.

[0096] The application analysis and daemon module 232 is configured to open a virtual interface of the target application through the virtual screen according to the information from the target application determination module 231, and determine whether there is information that needs to be daemoned in the virtual interface. In an example, when the information that needs to be daemoned is detected, the application analysis and daemon module 232 can control the interaction module 210 to perform information prompting, which is used to prompt the user that the relevant information has been detected. In another example, when the information that needs to be daemoned is detected, the application analysis and daemon module 232 can activate the prompt opportunity determination module 240 to make the prompt opportunity determination module 240 determine whether information prompting can be performed, and when the prompt opportunity determination module 240 determines that information prompting can be performed, the application analysis and daemon module 232 can control the interaction module 210 to perform information prompting. In yet another example, when the information that needs to be daemoned is detected, the application analysis and daemon module 232 sends the information that needs to be prompted to the interaction module 210, and activates the prompt opportunity determination module 240 to make the prompt opportunity determination module 240 determine whether information prompting can be performed, and when the prompt opportunity determination module 240 determines that information prompting can be performed, the application analysis and daemon module 232 indicates to the interaction module 210 that information prompting can be performed, and then the interaction module 210 prompts the information from the application analysis and daemon module 232.

[0097] In actual implementation, when there is no virtual screen in the system, a virtual screen can be created first, and then the virtual interface of the target application is opened through the virtual screen.

[0098] In some implementations, when the target application determined by the target application determination module 231 is a communication application and the type of the information that needs to be daemoned is information sent by a certain contact person, the application analysis and daemon module 232 can also detect information from the contact person in real time through the application programming interface (API) of the communication application.

[0099] In some implementations, after the interaction module 210 performs information prompting, a voice instruction of the user for the aforementioned prompted information can be received, which indicates to perform operation 1 for the target application. After the voice processing module 220 processes the aforementioned voice instruction, the obtained text information can be input to the interaction instruction processing module 233, and then the interaction instruction processing module 233 can perform operation 2 in the virtual interface in response to the aforementioned text information, which can include at least one sub-operation to implement the operation 1 requested by the user.

[0100] In actual implementation, the functions of the aforementioned application analysis and daemon module 232 and the interaction instruction processing module 233 can be implemented based on a machine learning model. For example:

[0101] The application analysis and daemon module 232 can include a processing module, and the application analysis and daemon module 232 includes a machine learning model, or the application analysis and daemon module 232 is associated with a machine learning model. When the application analysis and daemon module 232 needs to view the target application through the virtual interface of the target application, the machine learning model can output an operation instruction 1 based on the text information from the speech processing module 220. Taking an example of information A indicating the daemon application 1 by the text information, the aforementioned operation instruction 1 can indicate to open the interface including the information A in the application 1 through the virtual screen. Further, the processing module in the application analysis and daemon module 232 opens the virtual screen in response to the operation instruction 1, and opens the application 1 and the interface including the information A in the virtual screen, so that the content displayed by the virtual screen includes the interface associated with the information A. Further, the processing module in the application analysis and daemon module 232 sends the image of the interface displayed by the virtual screen acquired periodically to the machine learning model, and the machine learning model understands and analyzes the image, and determines whether there is content related to the information A in the virtual interface based on an image analysis algorithm and / or a text recognition algorithm. Exemplarily, the frequency of acquiring the image of the interface displayed by the virtual screen can be determined based on the specific application scenario, and the aforementioned frequency can be dynamically adjusted. In some implementations, the machine learning model can also determine an operation interaction suggestion based on the specific content of the information A, the operation interaction suggestion indicating the next operation suggested to the user, and further, the processing module in the application analysis and daemon module 232 can control the interaction module 210 to prompt the aforementioned next operation suggested to the user. For example, taking an example of the target application being a video application and the information needing to be daemoned being whether a certain TV series is updated, when the aforementioned TV series is detected to be updated, the operation interaction suggestion indicated to the user can be “play the updated part of the TV series”.

[0102] The interaction instruction processing module 233 can include a processing module, and the interaction instruction processing module 233 includes a machine learning model, or the interaction instruction processing module 233 is associated with a machine learning model. When the text information indicating operation 1 performed on the target application program is received, the machine learning model can output operation instruction 2 based on the text information indicating operation 1 performed on the target application program, and the operation instruction 2 can indicate at least one set of operation information, each set of operation information including an operation event type and an operation position, wherein the operation event type indicates an operation (such as clicking, sliding, etc.) performed in the virtual interface, and the operation position indicates the coordinates of the relevant operation in the virtual interface. Taking the operation 1 indicating that the contact A replies to the message X as an example, the operation instruction 2 can indicate that the message X is input in the message bar of the conversation box of the contact A in the target application program and sent. Exemplarily, the operation instruction 2 can at least include: 1) operation information indicating clicking (operation event type) the position (operation position) where the conversation box of the contact A in the target application program is located; 2) operation information indicating clicking (operation event type) the position (operation position) where each letter in the virtual keyboard in the virtual screen is located, so as to form the message X in the message bar; or operation information indicating directly inputting the message X (operation event type) in the message bar (operation position); 3) operation information indicating clicking (operation event type) the position (operation position) where the control for sending the message in the conversation box of the contact A is located. Further, the processing module in the interaction instruction processing module 233 opens the virtual screen in response to the operation instruction 2, and opens the conversation box of the contact A in the target application program in the virtual screen, and inputs the message X in the message bar of the conversation box and sends it.

[0103] The machine learning model involved in the foregoing application analysis and daemon module 232 and interaction instruction processing module 233 can be the same machine learning model, or can be different machine learning models.

[0104] (Four) The prompt opportunity determination module 240 is configured to determine whether the information prompt can be controlled by the interaction module 210 according to at least one of the following: the application program running in the visual interface of the display device deploying the target application program, the type of the display device deploying the target application program, the position of the user issuing the voice instruction in the cabin, the state of the user issuing the voice instruction, and the running state of the vehicle. It should be noted that when it is determined that the current opportunity does not meet the information prompt condition, the prompt opportunity determination module 240 continues to monitor until the state of the cabin and / or the state of the user meet the information prompt condition, and then the information prompt is performed.

[0105] (Five) The perception module 250 is configured to collect perception information in the cabin, and the perception information is used to determine the presence of personnel in the cabin and / or the source of the voice instruction in the cabin.

[0106] It should be understood that the above modules are only an example, and in actual applications, the above modules can be added or deleted according to actual needs. For example, Figure 2 In the system architecture shown in FIG. 2, the voice processing module 220 and the application daemon module 230 can be combined into one module; for another example, the application daemon module 230 and the prompt opportunity determination module 240 can be combined into one module. In some implementations, the perception module 250 and the part of the interaction module 210 for receiving user instructions can be combined into one module.

[0107] The above introduces the control system architecture provided by the embodiments of the present application, and the following introduces in detail the control method provided by the embodiments of the present application based on the system architecture. Figure 2 The interaction system shown in FIG. 2 implements the control method provided by the embodiments of the present application.

[0108] Figure 3 A schematic flowchart of the control method provided by the embodiments of the present application is shown, which can be executed by the vehicle 100 shown in FIG. 1, or also can be executed by the system shown in FIG. 2. Specifically, the method 300 can include the following or all steps: Figure 1 Figure 2 The method 300 can include the following or all steps:

[0109] S301, obtaining a voice instruction 1, the voice instruction 1 at least indicating the I-class information associated with a care display device 1.

[0110] In some implementations, the voice instruction 1 at least indicating the I-class information associated with the care display device 1 can be understood as: the voice instruction 1 explicitly indicating the I-class information and explicitly or implicitly indicating the display device 1. Wherein, the voice instruction 1 explicitly indicating the I-class information means that the voice instruction 1 includes the “I-class information” text; the voice instruction 1 implicitly indicating the display device 1 means that the voice instruction 1 does not include the “display device 1” text. When the voice instruction 1 does not include the text indicating the display device, the display device 1 can be determined based on the position and / or identity information of the user issuing the voice instruction 1 in the cabin, etc.

[0111] Exemplarily, the cabin of the vehicle can include multiple display devices, and the same or different application programs can be deployed in each display device. For example, at least one of the following display devices can be arranged inside the cabin of the vehicle: a central control screen, a co-driver entertainment screen, a main driver headrest rear screen, a co-driver headrest rear screen, a display screen arranged at the top of the cabin (or a second-row entertainment screen), and an instrument screen. The display device 1 can be one of the foregoing multiple display devices.

[0112] ​In an example, the voice instruction 1 can be a voice instruction issued by a user 1 in the cabin. Further, the display device 1 can be determined based on a position of the user 1 in the cabin and a mapping relationship between a region where the user 1 is located in the cabin and the display device. For example, when the user 1 is located at the main driver seat, the display device 1 can be a center control screen and / or a co-driver entertainment screen; when the user 1 is located at the co-driver seat, the display device 1 can be a co-driver entertainment screen and / or a center control screen; and when the user 1 is located in a second-row region in the cabin, the display device 1 can be a display screen located at a rear part of a main driver headrest or a rear part of a co-driver headrest, or a display screen located in the second-row region.

[0113] In another example, the voice instruction 1 can be a voice instruction issued by a user 1 in the cabin. Further, the display device 1 can be determined based on identity information of the user 1, and the display device 1 can be a display device in which a system account of the user 1 is logged in among the display devices in the cabin; or the display device 1 can be a display device in which a target application program logged in an application account of the user 1 is deployed among the display devices. The identity information of the user 1 can be determined based on biological characteristics such as a face, a body feature, a voice feature, etc. of the user. In this case, if the user 1 is located at the co-driver seat and a display device (e.g., a center control screen) in a main driver region logs in the system account of the user 1, the display device in the main driver region can be determined as the display device 1. Alternatively, if the user 1 is located at the co-driver seat and the display device (e.g., the center control screen) in the main driver region is deployed with the target application program logged in the application account of the user 1, the display device in the main driver region can be determined as the display device 1.

[0114] The system account refers to an account used to log in a vehicle system and use related services of the vehicle system, and the application account refers to an account used to log in an application program and use related services of the application program.

[0115] In some implementations, the voice instruction 1 can only indicate the class I information, and the target application program can be determined based on the class I information. For example, the target application program can be determined based on the attribute of the class I information. For example, the attribute of the class I information is video information, and the target application program can be determined as a video application program. In the case where a plurality of video application programs are deployed in the vehicle, the video application program with the highest usage frequency of the user can be determined as the target application program. For another example, the attribute of the class I information is instant message, and the target application program can be determined as an instant messaging application program. In addition, when the attribute of the class I information can match a plurality of application programs, the types of the plurality of application programs can be the same or different. In this case, the plurality of application programs can all be regarded as the target application program, or one of the plurality of application programs that is used by the user 1 most recently in the time period 1 can be determined as the target application program. For example, the class I information is information from a certain contact person, and two instant messaging application programs, such as application program a and application program b, can be matched based on the class I information. In this case, the application program a and the application program b can all be regarded as the target application program, or the application program a can be regarded as the target application program if the user 1 has used the application program a in the time period 1 and has not used the application program b.

[0116] For example, the time period 1 can be a time period between 5 minutes and 30 minutes, or the time period 1 can also be other time periods. In addition, the end time of the time period 1 can be the time when the voice instruction 1 is obtained, or the end time of the time period 1 can be earlier than the time when the voice instruction is obtained, and the time difference between the end time of the time period 1 and the time when the voice instruction is obtained is less than a certain threshold value (such as one of 5 seconds and 10 seconds).

[0117] In some implementations, the voice instruction 1 can also indicate the target application program and the class I information, so that the target application program can be determined based on the voice instruction 1.

[0118] It should be noted that when the voice instruction 1 explicitly indicates the target application program but does not explicitly indicate the display device 1, the display device 1 can also be determined in combination with the target application program. At this time, the display device 1 can be a display device on which the application account of the user 1 is logged in.

[0119] S302, generating indication information 1 according to the voice instruction 1, the indication information 1 indicating the application program 1 that needs to be cared and the class I information associated with the application program 1.

[0120] For example, the application program 1 can be the target application program in the foregoing embodiments, and the specific implementation of determining the target application program can be referred to the description in S301, which will not be described here.

[0121] For example, the indication information 1 can include the operation instruction 1 in the foregoing embodiments. For a more specific implementation of generating the indication information 1, refer to the description in the third item of the corresponding part. Figure 2 For details, refer to the description in the third item of the corresponding part.

[0122] In addition, the application program 1 is an application program deployed in the display device 1.

[0123] S303, according to the indication information 1, display the virtual interface of the application program 1.

[0124] For example, according to the indication information 1, open the virtual screen corresponding to the display device 1, and open the virtual interface of the application program 1 in the virtual screen, and then display the information in the interface.

[0125] S304, determine whether the I-class information is detected in the virtual interface.

[0126] For details of determining whether the I-class information is detected in the virtual interface, refer to the description in the third item of the corresponding part. Figure 2 For details, refer to the description in the third item of the corresponding part.

[0127] Specifically, when the I-class information is detected, perform S305; otherwise, continue to perform S303 and S304.

[0128] S305, determine whether the prompting condition is met.

[0129] In some implementations, whether the prompting condition is met can be determined according to one or more of the following: the type of the display device 1, the type of the application program associated with the interface displayed by the visual interface of the display device 1, the running state of the vehicle, the speed of the vehicle, and the state of the user 1. Specifically, when at least one of the following conditions is met, it can be determined that the prompting condition is met:

[0130] 1) the type of the application program associated with the interface displayed by the visual interface of the display device 1 is a non-safety-critical application program;

[0131] 2) the display device 1 is a non-safety-critical screen;

[0132] 3) the vehicle is not in a manual driving state;

[0133] 4) the vehicle is not in a man-machine co-driving state;

[0134] 5) the speed of the vehicle is less than or equal to a speed threshold value, which can be one of 3 m / s to 10 m / s, or the speed threshold value can also be other numerical values;

[0135] 6) the user 1 is in a non-sleep state.

[0136] In some implementations, the safety-critical application refers to an application that affects the driving safety of the vehicle under manual control, such as a navigation application, an application for displaying a 360-degree surround view interface, and the like. The non-safety-critical application refers to an application that does not affect the driving safety of the vehicle under manual control, such as an intelligent driving application, an entertainment application, and the like. The intelligent driving application is an application for controlling an intelligent driving function, where the intelligent driving function can include, but is not limited to, the following functions that do not require driver control: automatic valet parking (AVP), intelligent navigation cruise assist (NCA). The safety-critical screen refers to a screen or display device that is easy to affect the driver and cause the driver to be distracted, such as an instrument screen, a central control screen, and the like. The non-safety-critical screen refers to a screen that is not easy to cause the driver to be distracted, or a screen that is not easy for the driver to observe or is far away from the driver, such as a screen near the rear passengers, a rear passenger entertainment screen, and the like.

[0137] For example, whether the user is in a sleep state can be determined based on facial information and / or body information of the occupant collected by a camera in the cabin, such as whether the user closes his eyes based on the facial information, whether the user lies down based on the body information, and the like. When the user closes his eyes and / or lies down, it is determined that the user is in a rest state.

[0138] More specifically, when it is determined that the prompting condition is met, S306 is performed; otherwise, S305 is repeatedly performed to determine whether the prompting condition is met at the current opportunity.

[0139] S306, controlling the prompting device to prompt information a, where the information a is used to prompt the class I information.

[0140] For example, the prompting device can be a sound emitting device, and the information a can be in the form of voice information. More specifically, the area where the sound emitting device for prompting the information a is located can be the same area as the area where the user 1 is located, or the area where the sound emitting device for prompting the information a is located can be the same area as the area where the display device 1 is located.

[0141] In one example, the class I information is an information sent by a certain contact, and the information a can be used to prompt that the information from the contact has been received. In another example, the class I information is an information related to a video update, and the information a can be used to prompt that the video has been updated. In yet another example, the class I information is a dynamic updated by a certain contact in a social application, and the information a can be used to prompt the dynamic updated by the contact.

[0142] S307, obtaining a voice instruction 2, where the voice instruction 2 indicates to perform an operation 1 on the class I information.

[0143] Exemplarily, if the first information is information sent by a contact, the operation 1 can be an operation for replying to the information of the contact; if the first information is information related to a video update, the operation 1 can be an operation for playing the video, or the operation 1 can also be an operation for controlling the video to start playing at a future time; if the first information is a dynamic updated by a contact, the operation 1 can be an operation for replying to the dynamic or liking the dynamic.

[0144] S308, performing the operation 2 associated with the operation 1 in the virtual interface.

[0145] It should be noted that the operation 1 can be understood as a purpose expected to be achieved by the user, and the operation 2 is a series of operations that need to be performed in the virtual interface in order to achieve the aforementioned purpose.

[0146] In an example, taking the operation 1 as an operation for replying to information of a contact as an example, the operation 2 can include the following part or all operations: an operation of opening an application 1 in the virtual screen so that the virtual screen displays a virtual interface of the application 1 when the application 1 is running, an operation of opening a dialogue box for communication with the contact in the application 1 displayed in the virtual interface, an operation of inputting the aforementioned information in an information bar of the dialogue box, and an operation of clicking a virtual button for sending the aforementioned information.

[0147] In another example, taking the operation 1 as an operation for controlling a video to play as an example, the operation 2 can include the following part or all operations: an operation of opening an application 1 in the virtual screen so that the virtual screen displays a virtual interface of the application 1 when the application 1 is running, and an operation of clicking the aforementioned video in the application 1 displayed in the virtual interface so that the video is played.

[0148] In another example, taking the operation 1 as an operation for liking a dynamic of a contact as an example, the operation 2 can include the following part or all operations: an operation of opening an application 1 in the virtual screen so that the virtual screen displays a virtual interface of the application 1 when the application 1 is running, an operation of sliding the virtual interface so that the dynamic of the aforementioned contact in the application 1 displayed in the virtual interface is displayed, and an operation of clicking a virtual button for liking the dynamic.

[0149] In some implementations, the voice instruction 2 can be processed based on a machine learning model to generate the operation instruction 2 in the aforementioned examples, and then the operation 2 is performed based on the operation instruction 2. For more specific implementations of generating the operation instruction 2, reference can be made to the description in the third part of the corresponding part. Figure 2 For the description in the third part of the corresponding part, details are not repeated here.

[0150] It should be noted that in actual implementation, only the operation 1 or the operation 2 can be performed. Figure 3Some steps in the method flow shown, for example, when it is determined in S304 that the class I information is detected, S305 is skipped and S306 is directly executed; for another example, after S306 is executed, S307 and S308 can also not be executed.

[0151] To facilitate understanding of the interactive scheme provided by the embodiments of the present application, the following will be described in combination with Figure 4 and Figure 5 The specific application scenarios of the interactive scheme will be described in detail below. The processing actions (such as control, detection, etc.) or steps involved in each of the following application scenarios can be executed by Figure 1 the computing platform 150 shown, or can also be executed by the interactive system described above.

[0152] Figure 4 An application scenario of the interactive scheme provided by the embodiments of the present application is shown, as shown in Figure 4 The scenario can involve the following flow:

[0153] 1) If the voice instruction 401 of the user in the main driving area is detected: "Help me follow application 1, and AA's information reminds me", S302 to S304 are executed, and it is determined whether the information from the contact AA is received in the application 1. In this case, the information from AA can be regarded as an example of class I information. At this time, the display device deployed for the application 1 can be a central control screen or other screen. More specific implementations of the display device deployed for the application 1 can be referred to the description in S301, which will not be described here.

[0154] 2) When the information from AA is detected, S305 is executed to determine whether the current opportunity meets the prompt condition. If the central control screen of the vehicle displays a navigation interface as shown in the upper right of Figure 4 , it can be determined that the vehicle is driving under manual control, and thus the information prompt is temporarily not given to the user.

[0155] 3) When it is detected that the vehicle is in a navigation pause state, for example, when the element 402 is detected in the navigation interface, the element 402 indicates that the traffic light at the nearest intersection of the vehicle is red and the remaining time of the red light is 35 seconds, it can be determined that the vehicle is in a navigation pause state, and it is further determined that the current opportunity meets the prompt condition, and the sound emitting device can be controlled to play the voice information 403. The voice information 403 can be regarded as an example of the information a in the foregoing embodiment. Exemplarily, the voice information 403 can indicate "AA sent a message, asking you how to report the materials".

[0156] In actual implementation, whether the current opportunity meets the prompt condition can also be determined based on other manners described in S305.

[0157] Exemplarily, the sounding device for sounding the voice information 403 can include a sounding device located in the main driver area.

[0158] 4) If the voice instruction 404 of the user in the main driver area is detected after the prompt voice information 403, that is, "just say ready and send the BB file to him", S307 and S308 can be executed to reply the relevant information (such as "the file is ready") to AA and send the BB file. At this time, the operation of replying the aforementioned relevant information to AA and sending the BB file can be regarded as an example of the aforementioned operation 1.

[0159] In actual implementation, based on the interaction scheme provided in the present application, it can also be determined whether there is information mentioning the user in the care group chat a in response to the user's instruction, and whether there is information mentioning the user in the interface of the target application program of the group chat a is detected based on S302 to S304. Further, when the mentioned information of the user is detected in the aforementioned interface, the user is prompted with prompt information indicating the information. For example, AA1 in the group chat a mentions the user to handle XX things, and AA2 in the group chat a sends a file and asks everyone in the group to learn it, and the prompt information can be the voice instruction as follows: "AA1 in the group chat a asks you to handle XX things, and AA2 sends a file to ask everyone to learn it". Further, if the voice instruction "reply AA1 in the group, say received, and download the file sent by AA2, remind me in the evening" indicating the next operation is detected, S307 and S308 are executed to reply the relevant information (such as "received") to AA1, download the file from AA2, and set the reminder information associated with the aforementioned file.

[0160] Figure 5 Another application scenario of the interaction scheme provided by the embodiments of the present application is shown, for example, Figure 5 As shown, the scenario can involve the following process:

[0161] 1) If the voice instruction 501 of the user in the co-driver area is detected, that is, "CC updates tell me", S302 to S304 are executed to detect whether the information of the CC update is detected in the target application program. In this case, the information of the CC update can be regarded as an example of the I-class information. The aforementioned CC can be a serialized TV series, or it can also be a video uploaded by a blogger in a short video platform, or the CC can also be other information. At this time, since the display device in the co-driver area is more convenient for the user in the co-driver area to watch the related video, the display device deploying the target application program can be the co-driver entertainment screen, that is, the display device deploying the target application program is based on the mapping relationship between the area where the user is located in the cabin and the display device.

[0162] 2) When the CC update is detected, S305 is performed to determine whether the current timing meets the prompting condition. Exemplarily, if the eyes of the user in the front passenger area are detected to be in the closed state when the CC update is detected, it can be determined that the user is in a resting state, and thus the information prompting is temporarily not performed.

[0163] 3) When the eyes of the user in the front passenger area are detected to be in the open state, it can be determined that the current timing meets the prompting condition, and thus the voice information 502 can be played by the sound emitting device, which can be regarded as an example of the information a in the foregoing embodiment. Exemplarily, the voice information 502 indicates “The CC has been updated. Do you want me to play it for you?”.

[0164] In actual implementation, the current timing whether meets the prompting condition can also be determined based on other manners described in S305.

[0165] Exemplarily, the sound emitting device playing the voice information 502 can include a sound emitting device located in the front passenger area.

[0166] 4) If the voice instruction 503 “Play it” of the user in the front passenger area is detected after the voice information 502 is played, S307 and S308 can be performed to control the front passenger entertainment screen to display the interface shown in the lower left of FIG. 5 to update the playing of the CC video. Figure 5

[0167] In actual implementation, if the length of the inserted advertisement in the video watched by the user using the display device 1 is too long, based on the interactive method provided in the present application, the application program playing the video can be controlled to switch to the background running in response to the instruction of the user, and after the playing of the advertisement is ended, the application program playing the video can be switched to the foreground running, so as to enable the user to continue watching the video.

[0168] In addition to the application scenarios shown in Figure 4 and Figure 5 , the interactive scheme provided in the embodiments of the present application can also be applied to other application programs, for example, the scheme of the present application can be applied to a social application program or other application programs. Taking the scenario in which the scheme is applied to a social application program as an example, the scenario can involve the following flow:

[0169] 1) When the voice instruction of the user “Tell me the new dynamics in the friend circle of application 2” is detected, S302 to S304 are performed to detect whether there is a new dynamic in the friend circle of application 2, and in this case, the updated dynamic in the friend circle can be regarded as an example of the I-type information.

[0170] 2) When the new dynamic in the friend circle (for example, friend A and friend B update the dynamics respectively) is detected, S305 is performed to determine whether the current timing meets the prompting condition.​

[0171] 3) When the current timing meets the prompting conditions, the following voice message can be displayed: "Recently, friend A posted... update, friend B posted... update." The "..." part can be the specific content of the update.

[0172] 4) If, after the aforementioned voice information is displayed, a further voice command from the user is detected: "Give A's post a like", then S307 and S308 can be executed to like A's post.

[0173] It should be understood that the aforementioned application scenarios are merely illustrative examples. In actual implementation, the interaction scheme based on this application can also protect other types of applications. This application will not exhaustively list the types of applications here.

[0174] It should be noted that the collection, acquisition, and access of user-related data involved in this application comply with the local laws and regulations of the region where this solution is used. Therefore, the aforementioned collection, acquisition, and access of user-related data (such as biometric information) is carried out with the user's authorization.

[0175] Figure 6 This illustration shows a schematic flowchart of an interaction method provided in an embodiment of this application. The method 1000 can be... Figure 1 The method is executed by the vehicle 100, for example by the computing platform 150 within the vehicle 100, or it can also be executed by... Figure 2 The system shown executes this method. The method includes:

[0176] S1010, Obtain first instruction information, the first instruction information instructs to view a first type of information of a first application, the first application being any application deployed in the first terminal.

[0177] For example, the first application may include the target application in method 300, and the first type of information may include the type I information in method 300. When this solution is applied to a vehicle, the first terminal may include a display device deployed in the vehicle, or the first terminal may also include a display screen deployed in the vehicle and a processor controlling the display screen; further, the first terminal and the sound-emitting device may be associated through a processor or control device such as a computing platform of the vehicle. When this solution is applied to electronic devices such as mobile phones and tablets, the first terminal may be the electronic device itself, and further, the sound-emitting device associated with the first terminal may be a speaker or other sound-emitting device of the electronic device.

[0178] In some implementations, S1010 can be further refined as follows: obtaining a first voice command from a first user, the first voice command instructing the viewing of a first type of information; and generating first instruction information based on the first voice command.

[0179] For example, the first user can include user 1 in the method 300, the first voice instruction can include voice instruction 1 in the method 300, and the first instruction information can include information in a text form converted based on the first voice instruction. For a specific implementation of generating the first instruction information based on the first voice instruction, reference can be made to the description in the first item of the corresponding part. Figure 2 For the description in the second item of the corresponding part, details are not repeated here.

[0180] In some implementations, the method further includes: determining the first terminal according to the position information and / or the identity information of the first user, wherein the position information indicates a position of the first user in a cabin of a vehicle, and the vehicle includes the first terminal; and determining the first application program according to the first terminal and the first voice instruction.

[0181] For example, when the present solution is applied in a vehicle, the first terminal can include display device 1 in the method 300, and for a more specific implementation of determining the first terminal, reference can be made to the description in S301, which is not repeated here.

[0182] For example, determining the first application program according to the first terminal and the first voice instruction can be understood as: based on the first voice instruction, determining, as a target application program, the first application program deployed in the first terminal on the basis of determining the first terminal.

[0183] In some implementations, the method further includes: determining the first application program according to the first voice instruction; and determining the first terminal according to the first application program and the position information and / or the identity information of the first user, wherein the position information indicates a position of the first user in a cabin of a vehicle, and the vehicle includes the first terminal.

[0184] For example, for a more specific implementation of determining the first application program and a more specific implementation of determining the first terminal based on the first application program, reference can be made to the description in S301, which is not repeated here.

[0185] S1020, viewing the first application program according to the first instruction information.

[0186] In some implementations, S1020 can be refined as: viewing the first application program according to the first instruction information, including: viewing a first interface of the first terminal according to the first instruction information, the first interface being an interface virtually displayed in a virtual screen of the first terminal and being used to carry content displayed when the first application program runs.

[0187] For example, the first interface can include the interface displayed in the virtual screen in the foregoing embodiments.

[0188] In some implementations, S1020 can be refined as: inputting the first indication information into the first machine learning model, so that the first machine learning model outputs a first operation instruction, the first operation instruction being used to instruct to open the first interface; in response to the first operation instruction, opening the first interface; obtaining at least one frame of image of the first interface; inputting the at least one frame of image and the first indication information into the second machine learning model, so that the second machine learning model outputs interface understanding information of each frame of image in the at least one frame of image, the interface understanding information indicating whether the first type of information is detected.

[0189] For example, the first operation instruction can include the operation instruction 1 in the foregoing embodiments, and the first machine learning model and the second machine learning model can include Figure 2 For the machine learning model involved in the corresponding part, more detailed processing steps associated with the first machine learning model and the second machine learning model can be referred to Figure 2 The description in the third item of the corresponding part will not be repeated here.

[0190] S1030, when the first type of information is detected, controlling the sound generating device associated with the first terminal to prompt the first information, the first information being used to prompt the first type of information.

[0191] In some implementations, when the first terminal is applied in a vehicle, the sound generating device associated with the first terminal can be determined based on a position of a display screen included in the first terminal in the cabin, for example, the sound generating device associated with the first terminal and the display screen included in the first terminal can be located in the same area in the cabin; or the sound generating device associated with the first terminal can be determined based on a position of a user triggering the first indication information in the cabin, for example, the sound generating device associated with the first terminal and the user triggering the first indication information can be located in the same area in the cabin.

[0192] In some implementations, the method further includes: obtaining a second voice instruction, the second voice instruction indicating to execute a first operation on the first type of information; and executing a second operation associated with the first operation on the first interface according to the second voice instruction.

[0193] For example, the first operation can include the operation 1 in the foregoing embodiments, and the second operation can include the operation 2 in the foregoing embodiments.

[0194] The second operation associated with the first operation is performed on the first interface according to the second voice instruction, including: inputting the second instruction information into the third machine learning model, so that the third machine learning model outputs a second operation instruction, the second instruction information is obtained based on the second voice instruction, the second operation instruction is used to indicate at least one set of operation information, the operation indicated by the at least one set of operation information constitutes the second operation, each set of operation information in the at least one set of operation information includes an operation event type and an operation position, the operation position indicates a coordinate in the first interface, and the operation event type indicates an operation performed in the first interface; and the operation indicated by the second operation instruction is performed in the first interface.

[0195] Exemplarily, the second operation instruction can include the operation instruction 2 in the foregoing embodiments, and the more detailed processing steps associated with the third machine learning model can refer to Figure 2 The description in the third part of the corresponding part will not be repeated here.

[0196] In an example, the first application program is a communication application program, the first type of information is information received through the first application program, and the first operation and the second operation are operations for replying to the first type of information.

[0197] Exemplarily, the communication application program includes Figure 4 Taking the application 1 as an example, the first type of information can include information from a contact AA, and further, the first operation and the second operation can be operations for replying to the information related to the contact AA; or the first type of information can also include information mentioning the user in a group chat a, and further, the first operation and the second operation can be operations for replying to the information of the related contact, or the first operation and the second operation can also be other operations, such as saving a file, setting a reminder, and further operations performed on the first type of information.

[0198] In another example, the first application program is a video application program, the first type of information is update information of a first video played through the first application program, and the first operation and the second operation are operations for controlling the playing of the first video.

[0199] Exemplarily, the video application program includes Figure 5 Taking the target application program as an example, the first type of information can include information updated by CC, and further, the first operation and the second operation can be operations for controlling the playing of the part updated by CC; or the first type of information can also include information that an advertisement has been played, and further, the first operation and the second operation can be operations for controlling the target application program to run in the foreground and play the video.

[0200] For the latter example, the second operation is further used to control the visual interface of the first terminal to display a running interface of the first application program.

[0201] In some implementations, the first information is further used to inquire whether to perform a further operation for the first type of information.

[0202] In some implementations, the first information is prompted by the sound-emitting device when the state of the vehicle and / or the state of the user meets a prompting condition. Wherein, determining whether the state of the vehicle and / or the state of the user meets the prompting condition can comprise:

[0203] Implementation one: if the first terminal is applied in the vehicle, the display content of the visual interface of the first terminal is acquired; further, when the visual interface does not display the running interface of the safety-critical application, the first information is prompted by the sound-emitting device.

[0204] Wherein, the specific connotation of the safety-critical application can refer to the description in S305, which will not be repeated here.

[0205] Implementation two: if the first terminal is applied in the vehicle, the speed information is acquired, the speed information indicating the driving speed of the vehicle; further, when the driving speed is less than or equal to a speed threshold, the first information is prompted by the sound-emitting device.

[0206] In some implementations, when the visual interface displays the running interface of the safety-critical application, if the driving speed of the vehicle is less than or equal to the speed threshold, the first information can also be prompted by the sound-emitting device.

[0207] Implementation three: if the first terminal is applied in the vehicle, the first indication information is triggered by the first user, and the first user is the user located in the driving seat of the vehicle, the driving mode information is acquired, the driving mode information indicating the driving mode of the vehicle; further, when the vehicle is not in the first driving mode, the first information is prompted by the sound-emitting device; wherein, the first driving mode refers to the mode in which the operation of the driver affects the driving safety of the vehicle.

[0208] Exemplarily, the first driving mode can be the mode of manually driving the vehicle, or the first driving mode can also be the mode of jointly controlling the vehicle by the manual driving system and the intelligent driving system.

[0209] Implementation four: if the first terminal is applied in the vehicle, the first terminal is associated with a non-safety-critical screen, and the first type of information is detected, the first information is prompted by the sound-emitting device.

[0210] Implementation five: if the first indication information is triggered by the first user, the user state information is acquired, the user state information indicating the current state of the first user; further, when the user state information indicates that the first user is in a non-sleep state, the first information is prompted by the sound-emitting device.

[0211] In actual implementation, if the user triggering the first indication information is a user in the main driving area, it can be determined whether the vehicle state and / or the user state satisfies the prompting condition based on one or more of the implementation manner one, the implementation manner two and the implementation manner three. If the user triggering the first indication information is a user in an area other than the main driving area, it can be determined whether the vehicle state and / or the user state satisfies the prompting condition based on the implementation manner four and / or the implementation manner five. In some scenarios, if the user triggering the first indication information is a user in an area other than the main driving area, in order to avoid the voice broadcast by the sound emitting device from affecting the driver, the sound emitting device in the area other than the main driving area can be used for voice broadcast; or, when the user triggering the first indication information is a user in an area other than the main driving area, in order to avoid the voice broadcast by the sound emitting device from affecting the driver, it can also be determined whether the vehicle state and / or the user state satisfies the prompting condition based on one or more of the implementation manner one, the implementation manner two and the implementation manner three.

[0212] The interaction method provided by the embodiments of the present application can save the time of the user without refreshing the application frequently for the information that the user wants to focus on. In addition, the sound emitting device pushes the related information of the application program that the user focuses on to the user, which can minimize the influence of the first information pushing on the attention concentration degree of the user, and helps to improve the timeliness of information acquisition of the user on the premise of ensuring driving safety when the user is driving, thereby balancing the driving experience and driving safety.

[0213] In various embodiments of the present application, the terms and / or descriptions of various embodiments are consistent and can be mutually referred to if there is no special description and logical conflict. The technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0214] The method provided by the embodiments of the present application is described in detail above. The following will be described with reference to the Figures 1 to 6 The method provided by the embodiments of the present application is described in detail above. The following will be described with reference to the Figure 7 and Figure 8 The device provided by the embodiments of the present application is described in detail. It should be understood that the description of the device embodiments corresponds to the description of the method embodiments, and therefore, the content not described in detail can be referred to the method embodiments described above. For brevity, the description is not repeated here.

[0215] Figure 7A schematic block diagram of the interaction apparatus provided by the embodiments of the present application is shown. The apparatus 2000 can include units for performing the embodiments of the foregoing method. Each unit in the apparatus 2000 is configured to implement the corresponding process of the embodiments of the method described above. The apparatus 2000 includes a processing unit 2020, which can be configured to implement the corresponding processing function. In some embodiments, the apparatus 2000 can further include an obtaining unit 2010, which can be configured to implement the corresponding data obtaining or transceiving function.

[0216] Optionally, the apparatus 2000 further includes a storage unit, which can be configured to store instructions and / or data. The processing unit 2020 can read the instructions and / or data in the storage unit, so that the apparatus implements the related actions in the foregoing various method embodiments.

[0217] It should be understood that the specific process by which each unit performs the corresponding steps described above has been described in detail in the foregoing method embodiments, and thus will not be described here again for brevity.

[0218] It should also be understood that the apparatus 2000 here is in the form of functional units. The term “module” or “unit” here can refer to an application-specific ASIC, an electronic circuit, a processor (for example, a shared processor, a dedicated processor, or a group of processors) and a memory for executing one or more software or firmware programs, and a combination of logical circuits and / or other suitable components supporting the described functions.

[0219] The apparatus of each of the above solutions has the function of implementing the corresponding steps in the foregoing method. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions; for example, the obtaining unit 2010 can be replaced by a transceiver, and other units such as the processing unit can be replaced by a processor, for performing the related processing operations in the various method embodiments.

[0220] For example, the obtaining unit 2010 and the processing unit 2020 can also be arranged in the vehicle 100 shown. Figure 1 For example, the operations performed by the obtaining unit 2010 and the processing unit 2020 described above can be performed by one processor, or can also be performed by different processors.

[0221] In the specific implementation process, each unit in the above apparatus can be integrated completely or partially, or can also be independently implemented. In one implementation, these units are integrated together to be implemented in the form of a system on a chip (SoC).

[0222] Figure 8 is another schematic block diagram of the interaction apparatus provided by the embodiments of the present application.Figure 8 The apparatus 2100 shown can include a processor 2110, a transceiver 2120, and a memory 2130. The processor 2110, the transceiver 2120, and the memory 2130 are connected through internal connection paths. The memory 2130 is configured to store instructions, and the processor 2110 is configured to execute the instructions stored in the memory 2130 to implement the methods in the above embodiments. Alternatively, the memory 2130 can be coupled to the processor 2110 through an interface or integrated with the processor 2110.

[0223] It should be noted that the transceiver 2120 can include, but is not limited to, a transceiving device such as an input / output interface to enable communication between the apparatus 2100 and other devices or communication networks.

[0224] The memory 2130 can be a volatile memory and / or a non-volatile memory. The non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM). For example, the RAM can be used as an external cache. By way of example and not limitation, the RAM includes the following types: static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0225] The transceiver 2120 uses a transceiving device such as, but not limited to, a transceiver to enable communication between the apparatus 2100 and other devices or communication networks to receive / send data / information for implementing the methods in the above embodiments.

[0226] The embodiment of the present application further provides an intelligent driving device, which comprises the device 2000 or the device 2100 in the above embodiment.

[0227] The embodiment of the present application further provides a computer program product, which comprises computer program codes, and when the computer program codes are run on a computer, the computer is caused to implement the method in the above embodiment of the present application.

[0228] The embodiment of the present application further provides a computer readable storage medium, which stores computer instructions, and when the computer instructions are run on a computer, the computer is caused to implement the method in the above embodiment of the present application.

[0229] The embodiment of the present application further provides a chip, which comprises a circuit, and is used for implementing the method in the above embodiment of the present application.

[0230] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, and will not be described here.

[0231] In the description of the embodiments of the present application, unless otherwise specified, " / " represents the meaning of or, for example, A / B can represent A or B; "and / or" herein is a description of the association relationship of the associated object, which means that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In the present application, "at least one" means one or more, and "multiple" means two or more. "At least one of the following" or similar expressions means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b, or c can represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, and c can be single or multiple.

[0232] In the embodiments of the present application, the prefix words such as "first", "second" are only used to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects. The use of ordinal words such as ordinal words in the embodiments of the present application does not constitute a limitation on the described objects, and the description of the described objects should refer to the description of the context in the claims or embodiments, and should not constitute redundant limitations because of the use of such prefix words.

[0233] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the embodiments of the device described above are merely schematic, and the division of the units is merely logical function division. There can be other division manners in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0234] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place or can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0235] In addition, each functional unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0236] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An interaction method, characterized in that, The method comprises: obtaining a first voice instruction of a first user, the first voice instruction indicating to view a first type of information; generating first indication information according to the first voice instruction, the first indication information indicating to view the first type of information of a first application program, the first application program being any application program deployed in a first terminal; inputting the first indication information into a first machine learning model, so that the first machine learning model outputs a first operation instruction, the first operation instruction being used to instruct to open a first interface, the first interface being used to carry content displayed when the first application program runs; opening the first interface in response to the first operation instruction; obtaining at least one frame of image of the first interface; inputting the at least one frame of image and the first indication information into a second machine learning model, so that the second machine learning model outputs interface understanding information of each frame of image in the at least one frame of image, the interface understanding information indicating whether the first type of information is detected; when the first type of information is detected, controlling a sound producing device associated with the first terminal to prompt first information, the first information being used to prompt the first type of information.

2. The method of claim 1, wherein, The method further comprises: determining the first terminal according to position information and / or identity information of the first user; wherein the position information indicates a position of the first user in a cabin of a vehicle, the vehicle comprising the first terminal; determining the first application program according to the first terminal and the first voice instruction.

3. The method of claim 1, wherein, The method further comprises: determining the first application program according to the first voice instruction; determining the first terminal according to the first application program and position information and / or identity information of the first user; wherein the position information indicates a position of the first user in a cabin of a vehicle, the vehicle comprising the first terminal.

4. The method according to any one of claims 1 to 3, characterized in that, The first interface is an interface that is virtually displayed in a virtual screen of the first terminal.

5. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: obtaining a second voice instruction, the second voice instruction indicating to perform a first operation on the first type of information; performing a second operation associated with the first operation in the first interface according to the second voice instruction.

6. The method of claim 5, wherein, The performing a second operation associated with the first operation in the first interface according to the second voice instruction comprises: inputting second indication information into a third machine learning model, so that the third machine learning model outputs a second operation instruction, the second indication information being obtained based on the second voice instruction, the second operation instruction being used to instruct at least one set of operation information, the at least one set of operation information indicating the second operation, each set of operation information in the at least one set of operation information comprising an operation event type and an operation position, the operation position indicating a coordinate in the first interface, the operation event type indicating an operation performed in the first interface; performing the operation indicated by the second operation instruction in the first interface.

7. The method of claim 5, wherein, The first application is a communication application, the first type of information is information received through the first application, and the first operation and the second operation are operations for replying to the first type of information.

8. The method of claim 5, wherein, The first application is a video application, the first type of information is update information of a first video played through the first application, and the first operation and the second operation are operations for controlling playing of the first video.

9. The method of claim 5, wherein, The second operation is further used to control the visual interface of the first terminal to display a running interface of the first application.

10. The method of claim 5, wherein, The first information is further used to inquire whether to perform a further operation on the first type of information.

11. The method according to any one of claims 1 to 3, characterized in that, The first terminal is applied to a vehicle, and the method further includes: obtaining display content of a visual interface of the first terminal; The control of the sound-producing device associated with the first terminal to prompt the first information includes: when the visual interface does not display a running interface of a safety-critical application, the sound-producing device is controlled to prompt the first information.

12. The method of any one of claims 1 to 3, wherein, The first terminal is applied to a vehicle, and the method further includes: obtaining speed information, the speed information indicating a driving speed of the vehicle; The control of the sound-producing device associated with the first terminal to prompt the first information includes: when the driving speed is less than or equal to a speed threshold, the sound-producing device is controlled to prompt the first information.

13. The method according to any one of claims 1 to 3, characterized in that, The first terminal is applied to a vehicle, the first indication information is triggered by a first user, and the first user is a user located at a driving position of the vehicle, and the method further includes: obtaining driving mode information, the driving mode information indicating a driving mode in which the vehicle is located; The control of the sound-producing device associated with the first terminal to prompt the first information includes: when the vehicle is not in a first driving mode, the sound-producing device is controlled to prompt the first information; The first driving mode refers to a mode in which an operation of a driver affects driving safety of the vehicle.

14. The method of any one of claims 1 to 3, wherein, The first terminal is applied to a vehicle, and the control of the sound-producing device associated with the first terminal to prompt the first information includes: when the first terminal is associated with a non-safety-critical screen and the first type of information is detected, the sound-producing device is controlled to prompt the first information.

15. The method of any one of claims 1 to 3, wherein, The first indication information is triggered by a first user, and the method further includes: obtaining user state information, the user state information indicating a current state of the first user; The control of the sound-producing device associated with the first terminal to prompt the first information includes: when the user state information indicates that the first user is in a non-sleep state, the sound-producing device is controlled to prompt the first information.

16. An interactive device, characterized by The apparatus includes an obtaining unit and a processing unit, and the obtaining unit and the processing unit are configured to perform the method of any one of claims 1 to 15.

17. An interactive device, characterized by The apparatus includes: a processor configured to execute a computer program stored in a memory, so that the apparatus performs the method of any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that, The apparatus has instructions stored thereon, and the instructions are executed by a processor to implement the method of any one of claims 1 to 15.

19. A chip, characterized by The chip comprises circuitry for carrying out the method of any one of claims 1 to 15.

20. A computer program product, characterised in that, The computer program product comprises computer program code which, when run by a processor, implements the method of any one of claims 1 to 15.

21. A vehicle characterized by The vehicle comprises the apparatus of claim 16 or 17, or the computer readable storage medium of claim 18, or the chip of claim 19, or the vehicle is loaded with the computer program product of claim 20.

Images