Voice interaction method and device, terminal and storage medium

Abstract

The invention provides a voice interaction method and device, a terminal and a storage medium, and belongs to the technical field of terminals. The invention provides a method for triggering voice interaction without wakeup words, which comprises the following steps: predicting a service with a voice interaction intention of a user according to a target event capable of triggering voice interaction, and if a first service expressed by a voice instruction is related to the target event, executing the first service. The tedious operation that a user needs to speak the wake-up word frequently in the voice interaction process is omitted, so that the problem that the voice interaction efficiency is too low due to the fact that the wake-up word is speaked frequently is solved, operation is more convenient, the voice interaction process is more natural and personified, and therefore the user experience of voice interaction is improved.

Description

[0001] This application is a divisional application. The application number of the original application is 201910755150.6, and the original application date is August 15, 2019. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to a voice interaction method, device, terminal and storage medium. Background Art

[0003] With the development of terminal technology, more and more terminals support voice interaction functions. Users can interact with the terminal by speaking, thereby freeing their hands and improving the efficiency of human-computer interaction.

[0004] At present, the process of voice interaction usually includes: when the user wants to interact with the terminal by voice, first, the user says the wake-up word, the terminal will collect the voice command, and determine whether the voice command contains the wake-up word. If the voice command contains the wake-up word, the terminal will switch from the standby state to the working state, that is, the terminal is awakened; after that, the user says the business that needs to be processed by the terminal, and the terminal collects the voice command again, determines the business that needs to be processed according to the voice command, and processes the business. In an exemplary scenario, suppose the wake-up word of a certain vehicle terminal is "Hello, Xiaohua", and the user turns on the radio while driving. If he wants the vehicle terminal to play a song, he needs to say "Hello, Xiaohua", then the terminal is awakened, and the user says "Please help play the song of "Oriental Music Radio", then the vehicle terminal will automatically adjust the radio to "Oriental Music Radio" and play the song of "Oriental Music Radio".

[0005] When the above method is used for voice interaction, the user needs to say the wake-up word first to wake up the terminal to process the business, which makes the operation more cumbersome and inefficient. Summary of the invention

[0006] The embodiments of the present application provide a voice interaction method, device, terminal and storage medium, which can solve the technical problems of cumbersome operation and low efficiency of voice interaction in related technologies. The technical solution is as follows:

[0007] In a first aspect, a voice interaction method is provided, the method comprising: determining that a target event is detected, the target event being an event that can trigger voice interaction; querying a mapping relationship based on the target event to obtain a service type set, the service type set including one or more target service types; collecting voice instructions; obtaining a first service corresponding to the semantic information based on semantic information corresponding to the voice instructions; if the service type of the first service is any target service type in the service type set, executing the first service according to the voice instruction.

[0008] This embodiment provides a method for triggering voice interaction without a wake-up word. According to the target event that can trigger the voice interaction, a set of business types with which the user has the intention of voice interaction is predicted. If the business type of the first business expressed by the voice command is the target business type in the business type set, the first business is executed. By eliminating the tedious operation of the user needing to frequently say the wake-up word during the voice interaction process, the problem of low voice interaction efficiency caused by frequently saying the wake-up word is solved, the operation is made more convenient, and the process of voice interaction is made more natural and humanized, thereby improving the user experience of voice interaction.

[0009] Optionally, the determining that the target event occurs includes: determining that a first operation of a user is detected;

[0010] The method of querying the mapping relationship according to the target event to obtain a business type set includes: querying the mapping relationship according to the first operation to obtain the business type set, wherein the target business types included in the business type set are business types corresponding to one or more target second operations, and the one or more target second operations are continuity operations associated with the first operation.

[0011] Generally speaking, if a user performs any operation, he or she will continuously perform the next operation, and therefore will have the intention to perform voice interaction for the business corresponding to the next operation. This optional method fully utilizes the continuity of operations and maps the operation currently performed by the user to the business type corresponding to the next operation to be performed with a certain probability. This allows the user to accurately predict which business type he or she wants to perform voice interaction for when performing the operation, thereby ensuring the accuracy of the target business type.

[0012] Optionally, the determining that the target event occurs includes: receiving a notification message from an operating system or an application;

[0013] The querying of the mapping relationship according to the target event to obtain a business type set includes:

[0014] According to the notification message, the mapping relationship is queried to obtain the service type set, wherein the target service type included in the service type set is message viewing or message processing corresponding to the notification message.

[0015] Generally speaking, if the terminal receives a notification message, the user will have the need to view the notification message or process the notification message, and will therefore have a voice interaction intention for message viewing or message processing. This optional method fully takes into account the user's need to view or process the notification message, and maps the event of receiving the notification message to the service type of message viewing or message processing. This allows the user to accurately predict which service type they want to perform voice interaction for when receiving a notification message, thereby ensuring the accuracy of the target service type.

[0016] Optionally, the notification message includes at least one of an incoming call notification, a text message, an instant messaging message, and an alarm message, and querying the mapping relationship according to the notification message to obtain the service type set includes at least one of the following:

[0017] According to the incoming call notification, query the mapping relationship to obtain the service type set, wherein the target service type included in the service type set is answering the incoming call;

[0018] According to the short message or instant messaging message, query the mapping relationship to obtain the service type set, wherein the target service type included in the service type set is message viewing or message replying;

[0019] According to the alarm message, the mapping relationship is queried to obtain the service type set, where the target service type included in the service type set is fault handling or information query.

[0020] Optionally, the determining that the target event occurs includes: determining that a current environmental parameter satisfies a first condition;

[0021] The querying of the mapping relationship according to the target event to obtain the business type set includes: querying the mapping relationship according to the environmental parameter to obtain the business type set, wherein the target business type included in the business type set is the adjustment of the environmental parameter.

[0022] Generally speaking, the environment will affect the user's perception, and the user will have the need to cope with the environment. For example, if a certain environmental parameter changes, the user will have the need to adjust the environmental parameter, and thus will have a voice interaction intention to adjust the environmental parameter. This optional method fully takes the user's need to cope with the environment into consideration, and maps the event that the environmental parameter meets the first condition to the business type of adjusting the environmental parameter. In this way, when the environmental parameter meets the first condition, it can accurately predict which business type the user wants to perform voice interaction for, thereby ensuring the accuracy of the target business type.

[0023] Optionally, the determining that the target event occurs includes: determining that the progress of the current business satisfies a second condition;

[0024] The querying of the mapping relationship according to the target event to obtain the business type set includes: querying the mapping relationship according to the current business to obtain the business type set, wherein the target business type included in the business type set is the business type of the current business.

[0025] Generally speaking, the progress of the current business will affect the user's perception, and the user will have the need to cope with the current business. For example, if the current business is about to end, the user will usually want to re-execute the current business, stop executing the current business, or adjust the current business. Through this optional method, the user's need to cope with business changes is fully taken into consideration, and the event that the progress of the current business meets the second condition will be mapped to the business type of the current business. In this way, when the progress of the current business meets the second condition, it can be accurately predicted for which business type the user wants to perform voice interaction, thereby ensuring the accuracy of the target business type.

[0026] Combined with the above-mentioned multiple optional methods, the target event can cover multiple modes, and the target event of any mode can trigger the voice interaction function of the corresponding business type, so that the wake-up function without wake-up words can be supported in various application scenarios, expanding the scope of application.

[0027] Optionally, the process of establishing the mapping relationship includes:

[0028] According to the historical records, a historical business executed in association with the historical target event is obtained, and the business type of the historical business and the historical target event are written into the mapping relationship;

[0029] Through this optional method, since the personal behavior pattern of the same user has regularity, under normal circumstances, the voice interaction intention generated by the user after the current target event will be the same or similar to the voice interaction intention generated after the historical target event occurred in the historical time with a high probability. Then, the business to be executed after the current target event will also be the same or similar to the business to be executed after the historical target event occurred with a high probability. Therefore, by combining historical records to predict the target business type targeted by the current voice interaction intention, the accuracy of the target business type can be improved.

[0030] Optionally, the process of establishing the mapping relationship includes: calling a machine learning model, inputting a sample target event into the machine learning model, outputting a business type, writing the output business type and the sample target event into the mapping relationship, and the machine learning model is used to predict the business type based on the event.

[0031] Through this optional method, a large number of samples can be used for pre-training to obtain a machine learning model, so that the machine learning model can learn the mapping relationship between events and business types. Therefore, a more accurate mapping relationship can be constructed through the machine learning model.

[0032] Optionally, after obtaining the first service corresponding to the semantic information according to the semantic information corresponding to the voice instruction, the method further includes: if the service type of the first service is different from each target service type in the service type set, writing the service type of the first service into the mapping relationship.

[0033] Through this optional method, the next time the target event is detected, after querying the mapping relationship, the business type set obtained will include the business type of the first business. Then, when the user expresses the business type of the first business through voice instructions, the first business will be executed in response to the voice instructions. In this way, if during the historical operation of the terminal, whenever event X is detected, the user expresses the intention to conduct voice interaction with business type Y through voice, event X and business type Y can be added to the mapping relationship. Then, on the one hand, as the voice interaction process is executed, the correlation between events and business types can be excavated, and the business types corresponding to events and semantic information can be supplemented and improved. On the other hand, as the business expands, the terminal can add new events and new business types to the mapping relationship to improve the scalability and timeliness of the mapping relationship.

[0034] Optionally, querying the mapping relationship according to the target event to obtain a business type set includes: querying the mapping relationship according to the target event to obtain the business type set and the probability corresponding to each target business type in the business type set, the probability indicating the possibility of executing a business of the corresponding target business type; if the business type of the first business is any target business type in the business type set, before executing the first business according to the voice instruction, the method also includes: filtering out the target business types whose probabilities do not meet the probability threshold from the business type set.

[0035] Optionally, after obtaining the first service corresponding to the semantic information according to the semantic information corresponding to the voice instruction, the method further includes: updating the probability in the mapping relationship according to the semantic information corresponding to the voice instruction.

[0036] Through this optional method, whenever the target event occurs, the probability can be dynamically adjusted according to the semantic information expressed by the user this time, so that in a self-learning manner, by evaluating the correctness of the predicted business type, the probability can be continuously corrected through iteration, so that the probability of each business type in the mapping relationship can be continuously optimized with the occurrence of the target event and the semantics expressed by the user, gradually approaching the user's personal behavior habits, ensuring that the mapping relationship is more accurate.

[0037] Optionally, updating the probability in the mapping relationship according to the semantic information corresponding to the voice instruction includes:

[0038] If the service type of the first service is any target service type in the service type set, increase the probability corresponding to the service type of the first service in the mapping relationship;

[0039] Through this optional method, if the target event X corresponds to the target business type Y, and each time the target event X is detected, the user always requests to execute the target business type Y service through voice, indicating that the target business type Y happens to be the business type that the user has the voice interaction intention after the target event X occurs. Then, through this optional method, the probability of the target business type Y will continue to increase. Therefore, when the target event is detected again, the probability of the target business type Y will meet the probability threshold, so that the target business type Y will be screened out. Then, if the voice command expresses the semantic information of the target business type Y, the terminal will respond to the voice command to process the service.

[0040] Optionally, updating the probability in the mapping relationship according to the semantic information corresponding to the voice instruction includes:

[0041] If the service type corresponding to the semantic information is different from each of the one or more target service types, reducing the probability corresponding to the target event and the one or more target service types in the mapping relationship;

[0042] Through this optional method, if the target event X corresponds to the target service type Y, and each time the target event X is detected, the user does not request to execute the service of the target service type Y through voice, indicating that the target service type Y is not the service type with which the user has the intention of voice interaction after the target event X occurs, then through this optional method, the probability of the target service type Y will continue to decrease. Therefore, when the target event X is detected again subsequently, the probability of the target service type Y will not meet the probability threshold, so that the target service type Y will be filtered out, and the terminal will not process the service of the target service type Y, thereby avoiding false wake-up.

[0043] Optionally, updating the probability in the mapping relationship according to the semantic information corresponding to the voice instruction includes: if the semantic information includes a wake-up word, increasing the probability corresponding to the business type of the first business in the mapping relationship.

[0044] Through this optional method, if the target event X corresponds to the target business type Y, and each time the target event X is detected, the user says the wake-up word, indicating that the target business type Y happens to be the business type for which the user has the intention of voice interaction after the target event X occurs, then through this optional method, the probability of the target business type Y will continue to increase, so when the target event is detected again subsequently, the probability of the target business type Y will meet the probability threshold, so that the target business type Y will be screened out, then if the voice command expresses the semantic information of the target business type Y, the terminal will respond to the voice command to process the business.

[0045] Optionally, after obtaining the first service corresponding to the semantic information according to the semantic information corresponding to the voice instruction, the method further includes: if the service type of the first service is different from each target service type in the service type set, discarding the voice instruction.

[0046] Through this optional method, if the service type set does not include the service type of the first service, it indicates that the service type expressed by the user's voice is outside the range of the predicted service type, the terminal predicts the service type incorrectly, and the user has no intention of voice interaction. The terminal may not respond to the voice command but discard the voice command, thereby avoiding false wake-up caused by service processing based on the voice command and saving the cache space occupied by the voice command.

[0047] In a second aspect, a voice interaction device is provided, wherein the voice interaction device has the function of implementing the voice interaction in the first aspect or any optional manner of the first aspect. The device includes at least one module, and the at least one module is used to implement the voice interaction method provided in the first aspect or any optional manner of the first aspect.

[0048] In a third aspect, a terminal is provided, comprising one or more processors and one or more memories, wherein at least one instruction is stored in the one or more memories, and the instruction is loaded and executed by the one or more processors to implement the voice interaction method provided in the first aspect or any optional method of the first aspect.

[0049] In a fourth aspect, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the instruction is loaded and executed by a processor to implement the voice interaction method provided by the first aspect or any optional method of the first aspect.

[0050] In a fifth aspect, a computer program product is provided, the computer program product comprising: a computer program code, when the computer program code is executed by a terminal, the terminal executes the voice interaction method provided by the above-mentioned first aspect or any optional method of the first aspect.

[0051] In a sixth aspect, a chip is provided, comprising a processor for calling and executing instructions stored in a memory from the memory, so that a terminal equipped with the chip executes the voice interaction method provided in the first aspect or any optional method of the first aspect.

[0052] In the seventh aspect, another chip is provided, comprising: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the voice interaction method provided by the above-mentioned first aspect or any optional method of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a schematic diagram of an implementation environment of a voice interaction method provided in an embodiment of the present application;

[0054] Figure 2 is a schematic diagram of the structure of a terminal 100 provided in an embodiment of the present application;

[0055] Figure 3 is a functional architecture diagram of a terminal 100 provided in an embodiment of the present application;

[0056] Figure 4 is a flow chart of a voice interaction method provided in an embodiment of the present application;

[0057] Figure 5 It is a software architecture diagram of a voice interaction system provided in an embodiment of the present application;

[0058] Figure 6 is a structural schematic diagram of a vehicle-mounted terminal provided in an embodiment of the present application;

[0059] Figure 7 It is a structural diagram of a voice interaction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0061] The following explains the terms used in this application.

[0062] Business type: a general term for a type of business, which can also be called a business field. For example, business types may include message viewing, message processing, adjusting environmental parameters, navigation, schedule consultation, air conditioning, radio, music, vehicle control, mileage inquiry, Q&A consultation, games, system settings, vehicle control, charging, repair and maintenance, communications, etc. Message viewing may include viewing text messages, viewing instant messaging messages of instant messaging applications, and viewing push messages of resource recommendation applications. Message processing may include answering calls, message replies, fault handling, information query, etc. Adjusting environmental parameters may include adjusting dust concentration, humidity, light, noise intensity, and temperature.

[0063] Service: is a specific transaction to be executed. For example, a service of the message viewing service type can be: viewing the conversation message X sent by user A, viewing the latest group announcement Y in the group chat, viewing the discount message released by the shopping application today, etc. A service of the environmental parameter adjustment service type can be: adjusting the temperature to 25° through the air conditioner, a service of the music service type can be: playing the latest song Z of singer B, a service of the navigation service type can be: navigating to the F community of E Road, D District, City C, a service of the answering call service type can be: answering the call of the caller ZZ, a service of the message reply service type can be: replying to the contact G with a text message "I am driving, reply later", and a service of the radio service type can be: opening the "Oriental Music Radio".

[0064] The following is an exemplary description of the implementation environment.

[0065] Figure 1 1 is a schematic diagram of an implementation environment of a voice interaction method provided in an embodiment of the present application. The implementation environment includes: a terminal 100 and a voice interaction platform 200.

[0066] The terminal 100 is connected to the voice interaction platform 200 via a wireless network or a wired network. The terminal 100 may be at least one of a smart phone, a smart speaker, a robot, a smart car, a vehicle-mounted terminal, a home appliance, a game console, a desktop computer, a tablet computer, an e-book reader, a smart TV, an MP3 (moving picture experts group audio layer III, dynamic image expert compression standard audio layer 3) player or an MP4 (moving picture experts group audio layer IV, dynamic image expert compression standard audio layer 4) player and a laptop portable computer. The terminal 100 is installed and runs an application that supports voice interaction. The application may be a voice assistant, an intelligent question and answer application, etc. Exemplarily, the terminal 100 is a terminal used by a user, and a user account is logged in to the application running in the terminal 100.

[0067] The voice interaction platform 200 includes at least one of a server, multiple servers, a cloud computing platform, and a virtualization center. The voice interaction platform 200 is used to provide background services for applications that support voice interaction. For example, the voice interaction platform can construct the mapping relationship provided in the following method embodiment, and send the mapping relationship to the terminal 100, so that the terminal 100 performs voice interaction based on the mapping relationship.

[0068] Optionally, the voice interaction platform 200 includes: a voice interaction server 201 and a database 202. The voice interaction server 201 is used to provide background services related to voice interaction. The voice interaction server 201 can be one or more. When there are multiple voice interaction servers 201, there are at least two voice interaction servers 201 for providing different services, and / or there are at least two voice interaction servers 201 for providing the same service, such as providing the same service in a load balancing manner, which is not limited in this embodiment of the present application. The database 202 can be used to store mapping relationships. In addition, the database 202 can store sample events and sample business types, so that the voice interaction server 201 reads sample events and sample business types from the database 202, and trains a machine learning model based on the sample events and sample business types, and establishes a mapping relationship through the machine learning model.

[0069] The terminal 100 may generally refer to one of a plurality of terminals, and this embodiment is only illustrated by the terminal 100. Those skilled in the art may know that the number of the above-mentioned terminals 100 may be more or less. For example, the above-mentioned terminal 100 may be only one, or the above-mentioned terminal 100 may be dozens or hundreds, or a greater number, in which case the above-mentioned voice interaction system also includes other terminals. The embodiment of the present application does not limit the number and device type of the terminal 100.

[0070] The following is an exemplary description of the hardware structure of the terminal.

[0071] Figure 2 It is a structural diagram of a terminal 100 provided in an embodiment of the present application.

[0072] The terminal 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0073] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the terminal 100. In other embodiments of the present application, the terminal 100 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0074] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0075] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0076] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0077] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0078] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the terminal 100.

[0079] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0080] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0081] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0082] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the terminal 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the terminal 100.

[0083] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0084] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the terminal 100, and can also be used to transmit data between the terminal 100 and peripheral devices. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other terminals, such as AR devices, etc.

[0085] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the terminal 100. In other embodiments of the present application, the terminal 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0086] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the terminal 100. While the charging management module 140 is charging the battery 142, it may also power the terminal through the power management module 141.

[0087] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0088] The wireless communication function of the terminal 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0089] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0090] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied on the terminal 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0091] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0092] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the terminal 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0093] In some embodiments, the antenna 1 of the terminal 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the terminal 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0094] The terminal 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0095] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0096] The terminal 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor.

[0097] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0098] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0099] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the terminal 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0100] Video codecs are used to compress or decompress digital videos. Terminal 100 may support one or more video codecs. Thus, terminal 100 may play or record videos in various coding formats, such as moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0101] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the terminal 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0102] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0103] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the terminal 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the terminal 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0104] The terminal 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor.

[0105] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0106] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The terminal 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0107] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the terminal 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the ear.

[0108] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can make a sound by putting his mouth close to microphone 170C to input the sound signal into microphone 170C. Terminal 100 may be provided with at least one microphone 170C. In other embodiments, terminal 100 may be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, terminal 100 may also be provided with three, four or more microphones 170C to realize collection of sound signals, noise reduction, identification of sound source, realization of directional recording function, etc.

[0109] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130, or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0110] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The terminal 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the terminal 100 detects the touch operation intensity according to the pressure sensor 180A. The terminal 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0111] The gyroscope sensor 180B can be used to determine the motion posture of the terminal 100. In some embodiments, the angular velocity of the terminal 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. Exemplarily, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the terminal 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the terminal 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0112] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the terminal 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0113] The magnetic sensor 180D includes a Hall sensor. The terminal 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the terminal 100 is a flip phone, the terminal 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover automatic unlocking and other features are set.

[0114] The acceleration sensor 180E can detect the magnitude of the acceleration of the terminal 100 in various directions (generally three axes). When the terminal 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the terminal posture and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0115] The distance sensor 180F is used to measure the distance. The terminal 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the terminal 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0116] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The terminal 100 emits infrared light outward through the light emitting diode. The terminal 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the terminal 100. When insufficient reflected light is detected, the terminal 100 can determine that there is no object near the terminal 100. The terminal 100 can use the proximity light sensor 180G to detect that the user holds the terminal 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0117] The ambient light sensor 180L is used to sense the ambient light brightness. The terminal 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the terminal 100 is in a pocket to prevent accidental touches.

[0118] The fingerprint sensor 180H is used to collect fingerprints. The terminal 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0119] The temperature sensor 180J is used to detect temperature. In some embodiments, the terminal 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the terminal 100 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the terminal 100 heats the battery 142 to avoid abnormal shutdown of the terminal 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the terminal 100 performs a boost on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0120] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the terminal 100, which is different from the position of the display screen 194.

[0121] The bone conduction sensor 180M can obtain vibration signals. In some embodiments, the bone conduction sensor 180M can obtain vibration signals of the vibrating bones of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure pulse signals. In some embodiments, the bone conduction sensor 180M can also be set in headphones to form bone conduction headphones. The audio module 170 can parse out voice commands based on the vibration signals of the vibrating bones of the vocal part obtained by the bone conduction sensor 180M to realize the voice function. The application processor can parse the heart rate information based on the blood pressure pulse signal obtained by the bone conduction sensor 180M to realize the heart rate detection function.

[0122] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The terminal 100 may receive key input and generate key signal input related to user settings and function control of the terminal 100.

[0123] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0124] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0125] The SIM card interface 195 is used to connect the SIM card. The SIM card can be connected to and separated from the terminal 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The terminal 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The terminal 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the terminal 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the terminal 100 and cannot be separated from the terminal 100. The software system of the terminal 100 can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture.

[0126] The software structure of the terminal 100 is illustrated by taking the Android system with a layered architecture as an example.

[0127] Figure 3 It is a functional architecture diagram of a terminal 100 provided in an embodiment of the present application.

[0128] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.

[0129] The application layer can include a series of application packages.

[0130] like Figure 3 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0131] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0132] like Figure 3 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0133] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0134] Content providers are used to store and retrieve data and make it accessible to applications. This data can include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0135] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0136] The phone manager is used to provide communication functions of the terminal 100, such as management of call status (including connection, disconnection, etc.).

[0137] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0138] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, the terminal vibrates, the indicator light flashes, etc.

[0139] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0140] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0141] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0142] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0143] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0144] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0145] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0146] A 2D graphics engine is a drawing engine for 2D drawings.

[0147] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0148] The following is an example of the workflow of the software and hardware of the terminal 100 in conjunction with the event detection scenario.

[0149] Taking the target event as a touch operation on the interface as an example, the process of detecting the trigger operation may include: when the touch sensor 180K receives the touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into an original input event (including touch coordinates, timestamp of the touch operation and other information). The original input event is stored in the kernel layer. The application framework layer obtains the original input event from the kernel layer, identifies the control corresponding to the input event, and thus detects that the touch operation has occurred. For example, if the touch operation is a single-click operation and the control corresponding to the single-click operation is the icon of a music application, the music application calls the interface of the application framework layer to start the music application, and then displays the interface of the music application through the display screen 194.

[0150] The following is an illustrative description of the application scenarios of the embodiments of the present application.

[0151] The embodiments of the present application can be applied to the scenario of waking up the terminal for voice interaction. In the related art, every time the user wants the terminal to perform a certain service, he needs to say the wake-up word of the terminal. By applying the method provided by the following method embodiment, the function of waking up without the wake-up word can be realized. Taking the wake-up word of the terminal as "Hello, Xiaohua" as an example, the following is a comparative description in combination with scenes one to eight:

[0152] Scenario 1: While driving, the user wants to play a song on the in-vehicle terminal.

[0153] In the related art, this scenario includes the following steps 1 to 7:

[0154] Step 1: The user says "Hello, Xiaohua".

[0155] Step 2: The vehicle terminal collects voice commands and determines that the voice commands contain "Hello, Xiaohua". The vehicle terminal is awakened and plays the voice "Are you there" to respond to the user's voice command. After that, if the vehicle terminal does not collect voice commands after a preset period of time, the vehicle terminal goes back to sleep.

[0156] Step 3: The user turns on the radio switch.

[0157] Step 4: The user says "Hello, Xiaohua".

[0158] Step 5: The vehicle terminal collects the voice command and determines that the voice command contains "Hello, Xiaohua". Then the vehicle terminal is reawakened and plays the voice "Are you there?" to respond to the user's voice command.

[0159] Step 6. The user says, "Please help play songs from Oriental Music Radio."

[0160] Step 7. The vehicle terminal collects the voice command and determines that the voice command includes "Hello, Xiaohua". Then the vehicle terminal will adjust the radio station to "Oriental Music Radio" and play songs from "Oriental Music Radio".

[0161] By applying the method provided in this embodiment, the scenario may include the following steps 1 to 4:

[0162] Step 1: The user turns on the radio switch.

[0163] Step 2: The radio switch sends a signal to the vehicle terminal, and the vehicle terminal determines that the radio switch is turned on and detects the operation. According to the operation of turning on the radio switch, the mapping relationship is queried and the target service type obtained is music, thereby predicting that the user has the intention to conduct voice interaction for the music service type.

[0164] Step 3: The user says, "Please help play songs from Oriental Music Radio."

[0165] Step 4: The vehicle terminal collects the voice command, obtains the service type corresponding to "help play songs from Oriental Music Radio", and determines that the predicted target service type is the same as the service type expressed by the user through voice, and then plays songs from Oriental Music Radio. In addition, if the content said by the user in step 3 is not related to music, the vehicle terminal may not respond to the content said by the user.

[0166] From the above description, it can be seen that by turning on the radio switch, the user can activate the wake-up word-free voice interaction function of the vehicle terminal for the music service type, thereby eliminating the need for the user to frequently say the wake-up word.

[0167] Scenario 2: While driving, the user wants to check instant messaging messages.

[0168] In the related art, this scenario includes the following steps 1 to 7:

[0169] Step 1: Application A running on a mobile phone receives an instant messaging message.

[0170] Step 2: The user says "Hello, Xiaohua".

[0171] Step 3: The terminal collects the voice command and determines that the voice command contains "Hello, Xiaohua", then the mobile phone is awakened and plays the voice "Are you there" to respond to the user's voice command; after that, if the mobile phone does not collect the voice command after a preset period of time, the mobile phone goes back to sleep.

[0172] Step 4: The user says "Hello, Xiaohua".

[0173] Step 5: The mobile phone collects the voice command and determines that the voice command contains "Hello, Xiaohua". Then the mobile phone is reawakened and plays the voice "Are you there?" to respond to the user's voice command.

[0174] Step 6: The user says, "Look what application A is saying."

[0175] Step 7: The mobile phone collects voice commands and determines that the voice commands include "see what application A is saying", then the mobile phone obtains the instant messaging message "let's go to eat hot pot at 7pm" received by application A, and plays the voice "let's go to eat hot pot at 7pm".

[0176] By applying the method provided in this embodiment, the scenario may include the following steps 1 to 4:

[0177] Step 1: Application A running on a mobile phone receives an instant messaging message.

[0178] Step 2: The mobile phone queries the mapping relationship based on the received instant messaging message and obtains that the target service type is application A, thereby predicting that the user has the intention to perform voice interaction for the service of application A type.

[0179] Step 3: The user says, "Look what application A is saying."

[0180] Step 4: The mobile phone collects voice commands and obtains the service type corresponding to "see what application A is saying" as application A. The mobile phone determines that the predicted target service type is the same as the service type expressed by the user through voice, and then the mobile phone obtains the instant messaging message "Let's go to eat hot pot at 7 pm" received by application A, and plays the voice "Let's go to eat hot pot at 7 pm". In addition, if the content said by the user in step 3 is irrelevant to application A, the mobile phone can not respond to the content said by the user, thereby avoiding false wake-up.

[0181] It can be seen from the above description that if an instant messaging message is received, the wake-up word-free voice interaction function of the mobile phone for the instant messaging application service can be activated, thereby eliminating the step of the user frequently saying the wake-up word.

[0182] Scenario 3: When a song is about to end, the user wants to continue playing another song.

[0183] In the related art, this scenario includes the following steps 1 to 5:

[0184] Step 1: Song A currently being played by the smart speaker is about to end.

[0185] Step 2: The user says "Hello, Xiaohua".

[0186] Step 3: The smart speaker collects voice commands and determines that the voice commands include "Hello, Xiaohua". Then the smart speaker is awakened and plays the voice "Are you there?" to respond to the user's voice commands.

[0187] Step 4: The user says "Put down song B".

[0188] Step 5: The smart speaker collects voice commands and determines that the voice commands include "Put down song B", then the smart speaker plays song B.

[0189] By applying the method provided in this embodiment, the scenario may include the following steps 1 to 3:

[0190] Step 1. When song A currently being played by the smart speaker is about to end, the smart speaker determines that the progress of the current service meets the conditions. The smart speaker queries the mapping relationship based on the current service music and obtains that the target service type is music, thereby predicting that the user has the intention to conduct voice interaction for the service type of music.

[0191] Step 2: The user says "Put down song B".

[0192] Step 3: The smart speaker collects the voice command and obtains that the service type corresponding to "Put down song B" is music. If the predicted target service type is the same as the service type expressed by the user through voice, the smart speaker plays song B. In addition, if the content said by the user in step 2 is not related to music, the smart speaker may not respond to the content said by the user.

[0193] From the above description, it can be seen that if the currently playing song is about to end, the mobile phone's wake-up word-free voice interaction function for the music service type can be activated, thereby eliminating the need for the user to frequently say the wake-up word.

[0194] Scenario 4: It suddenly rains while driving.

[0195] In the related art, this scenario includes the following steps 1 to 4:

[0196] Step 1: The user says "Hello, Xiaohua".

[0197] Step 2: The vehicle terminal collects the voice command and determines that the voice command contains "Hello, Xiaohua", then the vehicle terminal is awakened and plays the voice "Are you there" to respond to the user's voice command.

[0198] Step 3: The user says "Turn on the wipers".

[0199] Step 4: The vehicle-mounted terminal collects the voice command and determines that the voice command includes "turn on the wipers", then the vehicle-mounted terminal sends a signal to the controller of the car, and the controller controls the drive circuit of the wipers to drive the wipers to rotate.

[0200] By applying the method provided in this embodiment, the scenario may include the following steps 1 to 3:

[0201] Step 1: The vehicle terminal determines that rain is detected through the raindrop sensor, and then queries the mapping relationship based on the rain, and obtains that the target service type is wipers, thereby predicting that the user has the intention to perform voice interaction for the service type of wipers.

[0202] Step 2: The user says "Turn on the wipers".

[0203] Step 3: The vehicle terminal collects the voice command and obtains the service type corresponding to "turn on the wipers" as wipers. If the predicted target service type is determined to be the same as the service type expressed by the user through voice, the vehicle terminal sends a signal to the controller of the car, and the controller instructs the drive circuit of the wipers to drive the wipers to rotate. In addition, if the content said by the user in step 2 is not related to the wipers, the vehicle terminal may not respond to the content said by the user.

[0204] From the above description, it can be seen that if it rains, the wake-up word-free voice interaction function of the mobile phone for the wiper service type can be activated, thereby eliminating the need for the user to frequently say the wake-up word.

[0205] Scenario 5: The car is running low on fuel while driving.

[0206] In the related art, this scenario includes the following steps 1 to 4:

[0207] Step 1: The user says "Hello, Xiaohua".

[0208] Step 2: The vehicle terminal collects the voice command and determines that the voice command contains "Hello, Xiaohua", then the mobile phone is awakened and plays the voice "I'm here", thereby responding to the user's voice command.

[0209] Step 3: The user says “Where is the nearest gas station?”

[0210] Step 4: The vehicle terminal collects the voice command and determines that the voice command includes "where is the nearest gas station", then the vehicle terminal calls the interface with the navigation application to query the address of the gas station, and then plays the voice "the gas station address is on Road Y, District X".

[0211] By applying the method provided in this embodiment, the scenario may include the following steps 1 to 3:

[0212] Step 1: The vehicle terminal detects the fuel level of the car and determines that the fuel level is lower than a threshold. Then, the mapping relationship is queried based on the fuel level, and the service type set obtained includes gas stations and navigation, thereby predicting that the user has the intention to perform voice interaction for the two service types of gas stations or navigation.

[0213] Step 2: The user says “Where is the nearest gas station?”

[0214] Step 3: The vehicle terminal collects the voice command and finds that the service type corresponding to "where is the nearest gas station" is navigation. If the predicted target service type is the same as the service type expressed by the user through voice, the vehicle terminal calls the interface with the navigation application to query the gas station address, and then plays the voice "the gas station address is on Road Y, District X". In addition, if the content said by the user in step 2 is not related to navigation, the vehicle terminal may not respond to the content said by the user.

[0215] From the above description, it can be seen that if the car is low on fuel, the wake-up word-free voice interaction function of the vehicle terminal for the two types of services, gas station or navigation, can be activated, thereby eliminating the need for users to frequently say the wake-up word.

[0216] Scene 6: Answering a call.

[0217] In the related art, this scenario includes the following steps 1 to 5:

[0218] Step 1: When the mobile phone receives a call request from the caller, the operating system of the mobile phone pushes a call notification.

[0219] Step 2: The user says "Hello, Xiaohua".

[0220] Step 3: The terminal collects the voice command and determines that the voice command contains "Hello, Xiaohua", then the mobile phone is awakened and the voice "I'm here" is played.

[0221] Step 4: The user says “Please answer the phone for me”.

[0222] Step 5: The mobile phone collects the voice command and confirms that the voice command includes "answer the call for me", then the mobile phone answers the call.

[0223] By applying the method provided in this embodiment, the scenario may include the following steps 1 to 3:

[0224] Step 1: When the mobile phone receives a call request from the caller, the operating system of the mobile phone pushes a call notification. The mobile phone detects the call notification, queries the mapping relationship based on the call notification, and obtains that the target service type is communication, thereby predicting that the user has the intention to perform voice interaction for the communication service type.

[0225] Step 2: The user says “Please answer the phone for me”.

[0226] Step 3: The mobile phone collects the voice command, and according to the semantic information "answer the call" corresponding to "answer the call for me", the service type corresponding to the semantic information "answer the call" is the communication type. If the predicted target service type is the same as the service type expressed by the user through voice, the mobile phone answers the call. In addition, if the content said by the user in step 3 is irrelevant to communication, the vehicle terminal may not respond to the content said by the user.

[0227] From the above description, it can be seen that if the mobile phone receives an incoming call, the wake-up word-free voice interaction function of the mobile phone for the communication service type can be activated, thereby eliminating the need for the user to frequently say the wake-up word.

[0228] Scenario 7: The car drives into an area with poor air quality.

[0229] In the related art, this scenario includes the following steps 1 to 4:

[0230] Step 1: The user says "Hello, Xiaohua".

[0231] Step 2: The terminal collects the voice command and determines that the voice command contains "Hello, Xiaohua". Then the mobile phone is awakened and plays the voice "Are you there?" to respond to the user's voice command.

[0232] Step 3: The user says "Turn on the air purifier".

[0233] Step 4: The vehicle-mounted terminal collects the voice command and determines that the voice command includes "turn on the air purifier", then the vehicle-mounted terminal sends a signal to the car's controller, and the controller controls the air purifier to turn on.

[0234] By applying the method provided in this embodiment, the scenario may include the following steps 1 to 3:

[0235] Step 1: The air purifier's sensor detects the dust concentration and sends the detected dust concentration to the vehicle terminal. If the vehicle terminal determines that the dust concentration exceeds the threshold, it queries the mapping relationship based on the dust concentration and obtains the target service type of air purifier, thereby predicting that the user has the intention to conduct voice interaction for the air purifier type of service.

[0236] Step 2: The user says "Turn on the air purifier".

[0237] Step 3: The vehicle terminal collects the voice command, obtains the service type corresponding to "turn on the air purifier" as air purifier, and determines that the predicted target service type is the same as the service type expressed by the user through voice. Then the vehicle terminal sends a signal to the car controller, and the controller controls the air purifier to turn on. In addition, if the content said by the user in step 2 is not related to the air purifier, the vehicle terminal may not respond to the content said by the user.

[0238] From the above description, it can be seen that if the dust concentration exceeds the threshold, the wake-up word-free voice interaction function of the vehicle terminal for the air purifier service type can be activated, thereby eliminating the need for users to frequently say the wake-up word.

[0239] Scenario 8: While driving, the car's sunshade is halfway opened, and the user does not want the sunshade to open any further.

[0240] In the related art, this scenario includes the following steps 1 to 4:

[0241] Step 1: The user says "Hello, Xiaohua".

[0242] Step 2: The vehicle terminal collects the voice command and determines that the voice command contains "Hello, Xiaohua", then the vehicle terminal is awakened and plays the voice "Are you there" to respond to the user's voice command.

[0243] Step 3: The user says "Stop lowering the sunshade".

[0244] Step 4: The vehicle terminal collects voice commands and determines that the voice command includes "lower the sunshade". The vehicle terminal sends a stop signal to the car's controller. The stop signal is used to instruct to stop opening the sunshade. After receiving the stop signal, the controller controls the drive circuit of the sunshade to stop opening the sunshade.

[0245] By applying the method provided in this embodiment, the scenario may include the following steps 1 to 3:

[0246] Step 1. The sunshade sends its current status to the vehicle terminal. The vehicle terminal determines whether the opening degree of the sunshade meets the conditions based on the status of the sunshade. Then, based on the sunshade, the mapping relationship is queried to obtain the target service type "sunshade", thereby predicting that the user has the intention to perform voice interaction for the "sunshade" type of service.

[0247] Step 2: The user says "Stop lowering the sunshade".

[0248] Step 3: The vehicle terminal collects the voice command, and according to the semantic information "stop lowering the sunshade" corresponding to the voice command, the service type corresponding to the semantic information "stop lowering the sunshade" is the sunshade. Then the vehicle terminal sends a stop signal to the car controller, and the stop signal is used to instruct to stop opening the sunshade. After receiving the stop signal, the controller controls the driving circuit of the sunshade to stop opening the sunshade. In addition, if the content of the user's speech in step 2 is irrelevant to the sunshade, the vehicle terminal may not respond to the content of the user.

[0249] From the above description, it can be seen that if the opening degree of the sunshade meets the conditions, the wake-up word-free voice interaction function of the vehicle terminal for the sunshade service type can be activated, thereby eliminating the need for the user to frequently say the wake-up word.

[0250] The following is an exemplary introduction to the method flow of the present application.

[0251] Figure 4 is a flow chart of a voice interaction method provided in an embodiment of the present application. This embodiment is described by taking the execution subject as a terminal as an example. Figure 4 , the method comprising:

[0252] 401. The terminal determines that a target event occurs.

[0253] The target event is an event that can trigger voice interaction. If the target event occurs, the user will have a certain probability of having the intention of voice interaction and have the need to wake up the terminal for voice interaction. In view of this, the terminal will detect the target event so that when it is determined that the target event occurs, it will combine the target event and the voice command to determine whether to activate the voice interaction function.

[0254] Optionally, the target event may have one or more modalities, where modality refers to the form or dimension of the target event. As an illustration, the modality of the target event may be the user's operation, the modality of the target event may also be a notification message, the modality of the target event may also be that the environmental parameters satisfy the first condition, the modality of the target event may also be that the progress of the current business satisfies the second condition, and of course, the modality of the target event may also be replaced with other modalities according to business requirements, and this embodiment does not limit the modality of the target event.

[0255] The user's operation may be, but is not limited to, one or more of the operation of physical buttons, the operation of the interface, voice commands, and browsing behaviors. Among them, the physical button may be a button on the terminal body, or a button on other devices that have established a communication connection with the terminal. For example, if the terminal is a vehicle-mounted terminal, the physical button may be a button on any device carried on the car, for example, the physical button may be the air conditioning switch, radio switch, etc. of the car. If the user operates the physical button, the physical button may send a signal to the terminal, and the terminal determines that the operation of the physical button is detected. The interface may be the system interface of the terminal or the interface of the application. If the user operates the interface, the display screen of the interface may send a signal to the terminal, and the terminal determines that the operation of the interface is detected. The voice command may be collected by a microphone. The browsing behavior may be the behavior of the user browsing the interface of the terminal. If the user performs browsing behavior, the terminal may capture the user's line of sight through a camera, thereby determining that the browsing behavior is detected. The operation may be a pressing operation, a clicking operation, or a sliding operation, etc., and this embodiment does not limit the specific type of operation.

[0256] Notification messages can be messages pushed by the operating system or application. For example, notification messages can be one or more of incoming call notifications, text messages, instant messaging messages, alarm messages, and resource recommendation messages. Among them, alarm messages can indicate that a terminal has failed, such as the battery level is less than 10% of the total power, insufficient memory, or a network attack; alarm messages can also indicate that other devices that have established a communication connection with the terminal have failed. For example, an alarm message from a car terminal can indicate that the car's motor has failed; a resource recommendation message can indicate resources recommended to users, such as news recommended by a news application, goods or services recommended by an e-commerce application, virtual items recommended by a game application, etc.

[0257] The environmental parameter may be, but is not limited to, one or more of noise, temperature, humidity, brightness, dust concentration, and oil volume. The first condition may be, but is not limited to, that the environmental parameter exceeds a parameter threshold or that a change in the environmental parameter exceeds a change threshold.

[0258] Regarding the specific process of detecting that the environmental parameters meet the first condition, the sensor can collect environmental parameters in real time or periodically, and send the collected environmental parameters to the terminal. For example, the temperature sensor can collect temperature and send temperature to the terminal; the humidity sensor can collect humidity and send humidity to the terminal; the dust sensor can collect dust concentration and send dust concentration to the terminal; the brightness sensor can collect brightness and send brightness to the terminal; the microphone can collect noise intensity and send noise intensity to the terminal, and the fuel sensor can collect the current remaining fuel of the vehicle and send fuel to the terminal. The terminal can receive environmental parameters from the sensor; the terminal can determine whether the environmental parameters exceed the parameter threshold, and if the environmental parameters exceed the parameter threshold, it is determined that the environmental parameters are detected to meet the first condition. Alternatively, the terminal can obtain the change amount of the environmental parameters based on the currently obtained environmental parameters and the historically obtained environmental parameters, and determine whether the change amount of the environmental parameters exceeds the change amount threshold. If the change amount of the environmental parameters exceeds the change amount threshold, it indicates that the environment has changed, and then it is determined that the environmental parameters are detected to meet the first condition.

[0259] The current service may be a service currently executed by the terminal, or a service currently executed by other devices that have established a communication connection with the terminal. For example, if the terminal is a vehicle-mounted terminal, the vehicle-mounted terminal can establish a communication connection with the air conditioner, sunshade, and windshield wiper installed in the vehicle through the vehicle controller. The current service may be a navigation service or a music service currently executed by the vehicle-mounted terminal, or a temperature adjustment service executed by the air conditioner, a rotation service executed by the windshield wiper, or a lowering service executed by the sunshade.

[0260] The second condition may be, but is not limited to, a change in the progress of the service, such as a change in the progress of the service exceeding a threshold or a change in the progress of the service exceeding a change threshold. In schematic form, the second condition may be that the service is about to end, the service is halfway executed, etc. For example, for an air conditioner, the progress of the current service meeting the second condition may be that the air conditioner starts to execute a temperature adjustment service, such as the air conditioner starts to set the temperature, the air conditioner starts to increase the air volume, etc.; the progress of the current service meeting the second condition may also be that the progress of the temperature adjustment service executed by the air conditioner changes, such as the temperature sensor of the air conditioner detects a temperature change.

[0261] Regarding the specific process of detecting whether the progress of the current business meets the second condition, the terminal can obtain the progress of the current business, determine whether the progress of the current business exceeds the threshold, and if the progress of the current business exceeds the threshold, determine that the progress of the current business meets the second condition; or, the terminal can obtain the change in the progress of the current business based on the current progress of the current business and the historical progress of the current business, and determine whether the change exceeds the change threshold. If the change in the progress of the current business exceeds the change threshold, indicating that the current business has changed, it is determined that the progress of the current business meets the second condition. Among them, the change and the change threshold can be expressed by the percentage of change, or by the time of change, and of course, can also be expressed by data of other dimensions, which is not limited in this embodiment.

[0262] It should be noted that the user's operation, notification message, environmental parameters satisfying the first condition, and the progress of the current business satisfying the second condition are only examples of target events, and the target events are not limited to these. It should be understood that the modality of the target event can be expanded accordingly according to the actual business of the terminal, and any event that can trigger voice interaction can be provided as a target event. This embodiment does not limit the target event.

[0263] 402. The terminal queries a mapping relationship according to the target event to obtain a service type set.

[0264] The service type set includes one or more target service types, each of which is a service type corresponding to the voice interaction intention. The terminal can predict the user's voice interaction intention based on the target event to obtain a service type set. Among them, the target service type can be the type of service performed by the terminal, or the type of service performed by other devices that have established a communication connection with the terminal. For example, if the terminal is the controller of the vehicle, the target service type can be the type of service performed by the air conditioner, lights and other equipment carried by the vehicle. For example, the target service type can be one or more of navigation, schedule consultation, air conditioning, radio, music, vehicle control, mileage inquiry, question and answer consultation, games, system settings, vehicle control, charging, maintenance, and communication.

[0265] The mapping relationship may include one or more events and one or more business types. The mapping relationship may indicate the corresponding relationship between the event and the business type. Each event in the mapping relationship may correspond to one or more business types. The event in the mapping relationship may be a first table item, the business type corresponding to the event may be a second table item, and the positions of the first table item and the second table item correspond, for example, the first table item and the second table item may be located in the same row. As an example, the mapping relationship may be shown in Table 1 below.

[0266] Table 1

[0267]

[0268]

[0269] The terminal can use the target event as an index and query the mapping relationship to obtain a service type set. For example, if the detected target event is the operation of the main switch, query Table 1 to obtain a service type set of (navigation, music, schedule consultation).

[0270] Optionally, the mapping relationship may specifically include one or more of a first mapping relationship between a user's operation and a business type corresponding to the operation, a second mapping relationship between a notification message and message viewing or message processing, a third mapping relationship between environmental parameters and adjustment environmental parameters, and a fourth mapping relationship between a current business and a business type of the current business.

[0271] The first mapping relationship may include one or more operations and business types corresponding to the one or more operations, and any operation may correspond to one or more business types.

[0272] Table 2

[0273]

[0274] In some possible embodiments, the first mapping relationship can be constructed based on the operation continuity rule. The operation continuity rule means: if operation A and operation B are continuous operations, then if the user performs operation A, it can be predicted that the user has the intention to perform operation B. Continuous operation means: after the user performs operation A, operation B is immediately performed. For the convenience of expression, operation A (the user's current operation) is referred to as the first operation, and operation B (the continuous operation associated with the first operation) is referred to as the target second operation. The target second operation is an operation that has a certain probability of being performed after the first operation is performed. The target second operation is a predicted operation. The target second operation may or may not be performed. The target second operation may be the first operation after the first operation. The first mapping relationship may include a mapping relationship between the first operation and one or more business types. Each business type in the first mapping relationship is a business type corresponding to the target second operation, and one or more target second operations are continuous operations associated with the first operation. For example, referring to Table 2, turning on the main switch (first operation) and navigating to the destination (target second operation) are continuous operations, and turning on the main switch (first operation) and clicking the song play button (target second operation) are also continuous operations, and turning on the main switch (first operation) and checking today's schedule (target second operation) are also continuous operations. The business type of the operation of navigating to the destination is navigation, the business type corresponding to the operation of clicking the song play button is music, and the business type of the operation of checking today's schedule is schedule consultation. Therefore, when constructing the first mapping relationship, the operation of the main switch can be taken as the first operation, and navigation, music and schedule consultation can be taken as the business types corresponding to the target second operation, and the operation of the main switch, navigation, music and schedule consultation will be written into the first mapping relationship. For another example, opening the navigation interface (first operation) and entering the navigation destination on the navigation interface (target second operation) are continuous operations, and the business type corresponding to the operation of entering the navigation destination is navigation. Therefore, when constructing the first mapping relationship, the operation of the main switch can be taken as the first operation, and navigation can be taken as the business type corresponding to the target second operation. The mapping relationship between opening the navigation interface and navigation will be stored in the first mapping relationship. Then, when the user opens the navigation interface, the terminal can predict navigation as the target business type.

[0275] The second mapping relationship includes one or more notification messages and message viewing or message processing. Message viewing can be viewing text messages, viewing instant messaging messages of instant messaging applications, and viewing push messages of resource recommendation applications. Message processing can include answering calls, message replies, fault handling, and information query. Schematically, the second mapping relationship can be shown in Table 3 below.

[0276] Table 3

[0277]

[0278] In some possible embodiments, the second mapping relationship can be constructed based on the user's need to view the notification message or the need to process the notification message. Specifically, if a notification message is received, it can be predicted that the user has the intention to view the notification message or process the notification message. Therefore, the second mapping relationship can be a mapping relationship between the notification message and the message viewing, or the second mapping relationship can be a mapping relationship between the notification message and the message processing. For example, referring to Table 3 above, if an instant messaging application pushes an instant messaging message, it can be predicted that the user has the intention to view the instant messaging message, and the business type corresponding to viewing the instant messaging message can be an instant messaging application. Therefore, when constructing the second mapping relationship, the identifier of the instant messaging message and the instant messaging application can be written into the second mapping relationship.

[0279] The third mapping relationship includes one or more environmental parameters and adjustment environmental parameters.

[0280] Table 4

[0281]

[0282] In some possible embodiments, the third mapping relationship can be constructed based on the user's need to cope with environmental changes. Specifically, considering that when environmental parameters meet the conditions, it will affect the user's perception, it can be predicted that the user has the intention to adjust the environmental parameters. Therefore, the third mapping relationship may include a mapping relationship between environmental parameters and the business type corresponding to adjusting the environmental parameters. For example, referring to Table 4 above, if the temperature change meets the threshold, it can be predicted that the user has the need to adjust the temperature, and the business type corresponding to adjusting the temperature is air conditioning. Therefore, when constructing the third mapping relationship, the mapping relationship between the temperature change meeting the threshold and the air conditioning can be stored in the third mapping relationship.

[0283] The fourth mapping relationship may include one or more current services and service types of the current services.

[0284] Table 5

[0285]

[0286] In some possible embodiments, the fourth mapping relationship can be constructed based on the user's response needs. Specifically, considering that when the progress of the current business meets the conditions, it will affect the user's perception, so it can be predicted that the user has the intention to restart the current business, stop the current business, or adjust the current business. Therefore, the fourth mapping relationship can be a mapping relationship between the current business and the business type of the current business. For example, referring to Table 5 above, if the currently playing music is about to end, it can be predicted that the user has the need to play the next song, stop playing the song, or replay the song, and the business types corresponding to playing the next song, stopping playing the song, or replaying the song are all music. Therefore, when constructing the fourth mapping relationship, the mapping relationship between the music that is about to end and the music can be stored in the fourth mapping relationship.

[0287] In some possible embodiments, the process of establishing the mapping relationship may include the following implementation modes 1 and 2:

[0288] Implementation method 1: The terminal obtains the historical business executed in association with the historical target event according to the historical records, and writes the business type of the historical business and the historical target event into the mapping relationship.

[0289] In this implementation, the mapping relationship can be constructed based on the historical voice interaction process. Specifically, if the terminal detects a historical target event at a historical time point, and the user interacts with the terminal by voice, so that the terminal responds to the user's voice command and executes a certain historical service, the terminal can establish a mapping relationship between the historical target event and the service type of the historical service. Then, after the terminal detects the target event, it can be awakened and respond to the voice command to execute the service if the service type corresponding to the voice command is the same as the service type of the historical service.

[0290] The historical records include historical target events and historical services associated with the historical target events. The historical services associated with the historical target events are services executed after the historical target events are detected, and may be services executed by the first voice interaction after the historical target events are detected. For example, if an operation of triggering a main switch was detected yesterday, and the service executed by the first voice interaction thereafter was navigation to cell A, then the historical records may include the operation of triggering the main switch and navigation to cell A, and the operation of triggering the main switch and navigation to cell A may be written into a mapping relationship. Then, if the terminal currently detects an operation of triggering a main switch, the mapping relationship may be queried based on the operation of the main switch, and the target service type in the service type set may include navigation.

[0291] Optionally, the terminal can write the business type of the historical business executed recently into the mapping relationship. Specifically, the terminal can obtain the historical time period according to the current time point and the preset duration, and obtain the historical business executed in association with the historical target event within the historical time period according to the historical records, and write the business type of the historical business and the historical target event into the mapping relationship. Among them, the historical time period can be the most recent day, the most recent week, or the most recent month. Specifically, the end point of the historical time period can be the current time point, and the preset duration can be one day, one week, etc. In this way, the timeliness of the mapping relationship can be guaranteed, so that the mapping relationship can better reflect the user's recent behavioral habits.

[0292] Optionally, the terminal can write the historical business executed with high frequency into the mapping relationship. Specifically, the terminal can obtain the execution times of each historical business executed in association with the target event based on the historical records; the terminal can select the historical business with the most execution times from multiple historical businesses, and write the business type and historical target event of the historical business with the most execution times into the mapping relationship. Alternatively, the terminal can select the historical business with the execution times exceeding the threshold, and write the business type and historical target event corresponding to the historical business with the execution times exceeding the threshold into the mapping relationship.

[0293] Through implementation method one, since the personal behavior pattern of the same user is regular, under normal circumstances, the voice interaction intention generated by the user after the current target event will be the same or similar to the voice interaction intention generated after the historical target event in the past. Then, the business to be executed after the current target event will also be the same or similar to the business associated with the historical target event. Therefore, by combining historical records to predict the target business type targeted by the current voice interaction intention, the accuracy of the target business type can be improved.

[0294] Implementation method two: the terminal calls the machine learning model, inputs the sample target event into the machine learning model, outputs the business type, and writes the output business type and the sample target event into the mapping relationship.

[0295] The machine learning model is used to predict the business type based on the event. For example, the business type of the business executed in association with the event can be predicted based on the current event. In one possible implementation, multiple sample events and multiple sample business types can be used for model training to obtain the machine learning model, wherein the sample event can be an event executed by the terminal of the sample user or an event recorded in the terminal's historical records. The sample business type is the business type of the business executed in association with the sample event. The machine learning model can be, but is not limited to, a neural network model.

[0296] Through the second implementation method, the machine learning model can learn the mapping relationship from events to business types through a large number of samples in advance. Then, through the machine learning model, the target business type targeted by the voice interaction intention can be accurately predicted based on the current target event, thereby improving the accuracy of the target business type.

[0297] It should be noted that the mapping relationship can be constructed by the terminal or by other devices other than the terminal, and then the other devices send the constructed mapping relationship to the terminal, and the terminal can receive the mapping relationship to obtain the mapping relationship. Figure 1 The voice interaction platform 200 is shown. Of course, the terminal can also obtain the mapping relationship through other methods. For example, the mapping relationship can be published through a link address, and the terminal can access the link address to download the mapping relationship from the Internet. This embodiment does not specifically limit how the terminal obtains the mapping relationship.

[0298] In combination with the specific mode of the event, step 402 may be, but is not limited to, one or more of the following situations (1) to (4):

[0299] Case (1) The terminal queries the mapping relationship according to the first operation to obtain the service type set, wherein the target service type included in the service type set is a service type corresponding to one or more target second operations, and the one or more target second operations are continuity operations associated with the first operation.

[0300] If the user performs the first operation, it can be predicted that the user's voice interaction intention is to perform the target second operation. Therefore, the terminal can use the service type corresponding to the target second operation as the target service type. Among them, one first operation can correspond to one or more target second operations. For example, if the user triggers a confirmation operation on the air-conditioning button, generally speaking, the user will adjust the temperature immediately after triggering the confirmation operation on the air-conditioning button. In this case, the confirmation operation on the air-conditioning button is the first operation, and adjusting the temperature is the target second operation. The service type corresponding to adjusting the temperature is air-conditioning, so the terminal can obtain air-conditioning as the target service type; for another example, if the user clicks on the music search option, generally speaking, the user will fill in the song name immediately after clicking the music search option. In this case, clicking on the music search option is the first operation, and filling in the song name is the target second operation. The service type corresponding to filling in the song name is music, so the terminal can obtain music as the target service type; for another example, if the user clicks on the fault display option, generally speaking, the user will search for a fault solution or solve the fault based on the viewed fault information immediately after clicking the fault display option. In this case, clicking on the fault display option is the first operation, and searching for a fault solution or solving the fault is the target second operation. The service type corresponding to searching for a fault solution or solving the fault is fault, vehicle control or search, so the terminal can obtain fault, vehicle control or search as the target service type.

[0301] Generally speaking, if a user performs any operation, he or she will continuously perform the next operation, and therefore will have the intention to perform voice interaction for the business corresponding to the next operation. In this way, the continuity of operations is fully utilized, and the operation currently performed by the user will be mapped to the business type corresponding to the next operation with a certain probability of being performed. In this way, when the user performs an operation, it can be accurately predicted which business type the user wants to perform voice interaction for, thereby ensuring the accuracy of the target business type.

[0302] Regarding the specific process of obtaining the target service type in situation (1), in a possible implementation, the terminal may query the first mapping relationship according to the first operation to obtain the service type set.

[0303] Case (2) The terminal queries the mapping relationship and obtains the service type set, where the target service type included in the service type set is message viewing or message processing corresponding to the notification message.

[0304] Exemplarily, message viewing may be viewing the notification message through the application that pushes the notification message. For instance, if the notification message is an instant messaging message, the message viewing may be viewing the instant messaging message through the instant messaging application. If the notification message is a news recommendation message, the message viewing may be viewing the news message through the news application. Alternatively, message viewing may be playing the notification message, displaying the notification message on the screen, or projecting the notification message, etc. Message processing may be searching for the notification message, replying to the notification message, or processing a fault corresponding to the notification message.

[0305] Generally speaking, if the terminal receives a notification message, the user will have the need to view or process the message, and therefore will have a voice interaction intention to view or process the message. In this way, the user's need to view or process the message is fully taken into consideration, and the target event of receiving the notification message is mapped to the two target business types of message viewing or message processing. In this way, when a notification message is received, it can accurately predict which business type the user wants to perform voice interaction for, thereby ensuring the accuracy of the target business type.

[0306] Specifically, situation (2) may include one or more of the following situations (2.1) to (2.3).

[0307] Case (2.1): According to the incoming call notification, the mapping relationship is queried to obtain the service type set, and the target service type included in the service type set is answering the incoming call.

[0308] Case (2.2): According to the short message or instant messaging message, the mapping relationship is queried to obtain the service type set, and the target service type included in the service type set is message viewing or message replying.

[0309] For example, message viewing can be converting text or instant messaging messages into voice, and playing the text or instant messaging messages in voice form. For example, message replying can be obtaining reply information corresponding to the text or instant messaging message based on the text or instant messaging message, and sending the reply information to the sender of the text or instant messaging message; or, receiving reply information input by the user, and sending the reply information to the sender of the text or instant messaging message.

[0310] Case (2.3): According to the alarm message, the mapping relationship is queried to obtain the service type set, and the target service type included in the service type set is fault handling or information query.

[0311] Fault handling can be output fault handling solutions, repair and maintenance, etc. For example, if the alarm message is a low battery message, the fault handling can be mileage query, charging station. If the alarm message is a motor fault message, the fault handling can be Q&A consultation, motor repair, etc. Information query can be querying fault handling solutions, querying fault sources, etc.

[0312] Through the above cases (2.1) to (2.3), the corresponding target business type is predicted for each notification message, thereby supporting various application scenarios of receiving notification messages and expanding the scope of application.

[0313] Regarding the specific process of obtaining the target service type in situation (2), in a possible implementation, the terminal can query the second mapping relationship according to the notification message to obtain one or more target service types.

[0314] Case (3): According to the environment parameter, the mapping relationship is queried to obtain the service type set, and the target service type included in the service type set is the adjustment environment parameter.

[0315] Adjusting environmental parameters can be reducing dust concentration through an air purifier, increasing humidity through a humidifier, adjusting temperature through air conditioning or car windows, adjusting light intensity through sunshades or car windows, adjusting rainfall through wipers, etc.

[0316] Generally speaking, the environment will affect the user's perception, and the user will have the need to cope with the environment. For example, if a certain environmental parameter changes, the user will have the need to adjust the environmental parameter, and thus will have a voice interaction intention to adjust the environmental parameter. In this way, the user's need to cope with the environment is fully taken into consideration, and the target event of the environmental parameter meeting the first condition is mapped to the target business type of adjusting the environmental parameter. In this way, when the environmental parameter meets the first condition, it can accurately predict which business type the user wants to perform voice interaction for, thereby ensuring the accuracy of the target business type.

[0317] Regarding the specific process of obtaining the target service type in situation (3), in a possible implementation, the terminal may query the third mapping relationship according to the environmental parameters to obtain one or more target service types.

[0318] Case (4): According to the current service, the mapping relationship is queried to obtain the service type set, and the target service type included in the service type set is the service type of the current service.

[0319] The service type of the current service may be, but is not limited to, re-executing the current service, stopping the current service, or adjusting the current service. For example, if the current service is playing music, the service type of the current service may be playing other music, re-playing music, stopping playing music, etc.

[0320] Generally speaking, the progress of the current business will affect the user's perception, and the user will have the need to cope with the current business. For example, if the current business is about to end, the user will usually want to re-execute the current business, stop executing the current business, or adjust the current business. In this way, the user's need to cope with business changes is fully taken into consideration, and the target event that the progress of the current business meets the second condition is mapped to the business type of the current business. In this way, when the progress of the current business meets the second condition, it can be accurately predicted for which business type the user wants to perform voice interaction, thereby ensuring the accuracy of the target business type.

[0321] Optionally, step 402 can be replaced by: the terminal obtains the historical business associated with the historical target event according to the historical records, and obtains the business type corresponding to the historical business as the target business type. That is, the target business type can be obtained by querying the historical records without establishing a mapping relationship according to the historical records.

[0322] Optionally, step 402 can be replaced by: the terminal calls a machine learning model, inputs the target event into the machine learning model, outputs the one or more target business types, and the machine learning model is used to predict the target business type according to the target event. That is, the target business type can be obtained through the machine learning model without establishing a mapping relationship according to the machine learning model.

[0323] Optionally, the mapping relationship may further include the probability of each target service type, and step 402 may be replaced by: the terminal queries the mapping relationship according to the target event to obtain the service type set and the probability corresponding to each target service type in the service type set.

[0324] The probability indicates the possibility of executing the business of the corresponding target business type. The larger the probability, the greater the possibility that the business of the corresponding target business type will be executed after the corresponding event is detected. For example, if there is a mapping relationship between the target event i, the target business type j, the probability 1 corresponding to the target business type j, the target business type k, and the probability 2 corresponding to the target business type k, then the mapping relationship can indicate that if it is determined that the target event i is detected, it is predicted that the business of the target business type j or the target business type k will be executed, and the probability of executing the business of the target business type j is probability 1, and the probability of executing the business of the target business type k is probability 2. Among them, i is the identifier of the target event, and j and k are the identifiers of the target business types.

[0325] Schematically, the mapping relationship can be shown in Table 6 below, where " / " in Table 6 represents nothing. Based on Table 6, if the operation of the main switch is detected, the service type set can be obtained as (navigation, music, schedule query), where the probability corresponding to navigation is 0.6, the probability corresponding to music is 0.7, and the probability corresponding to schedule query is 0.4.

[0326] Table 6

[0327]

[0328]

[0329] One thing that needs to be explained is that the mapping relationship shown in Table 6 can be provided as a multimodal wake-up model, the input parameters of the multimodal wake-up model include the target event, the output parameters of the multimodal wake-up model include a business type set and a probability, and the multimodal wake-up model can be used to predict the business type set based on the detected target event. The user can run the multimodal wake-up model on the terminal or sell the multimodal wake-up model to a third party for use.

[0330] 403. The terminal filters out target service types whose probabilities do not meet a probability threshold from the service type set.

[0331] For each target business type in the business type set, the terminal can compare the probability corresponding to the target business type with the probability threshold. If the probability corresponding to the target business type meets the probability threshold, the terminal will filter out the target business type. If the business type of the first business corresponding to the subsequently collected voice command is the target business type, the first business will be executed; if the probability corresponding to the target business type does not meet the probability threshold, the terminal will filter out the target business type, and the business of the target business type will not be executed subsequently.

[0332] In one possible implementation, the same probability threshold may be pre-set for each target service type, and the probability threshold may be pre-stored in the terminal, so that the terminal may compare each target service type with the same probability threshold. In another possible implementation, a corresponding probability threshold may be set for each target service type, and the probability threshold corresponding to each target service type may be written into the mapping relationship, so that the terminal may query the mapping relationship, obtain the probability threshold corresponding to each target service type, and compare each target service type with the corresponding probability threshold. Among them, the probability thresholds corresponding to different service types may be the same or different, and this embodiment does not limit this.

[0333] Exemplarily, the mapping relationship may include the following Table 7. If the service type set is (navigation, music, schedule consultation), according to the above Table 6, the probability corresponding to navigation is 0.6, the probability corresponding to music is 0.7, and the probability corresponding to schedule consultation is 0.4. According to Table 7, it can be obtained that the probability thresholds corresponding to navigation are 0.5, the probability thresholds corresponding to music are 0.5, and the probability thresholds corresponding to schedule consultation are 0.5. For navigation, its probability 0.6 is greater than the probability threshold 0.5, for music, its probability 0.7 is greater than the probability threshold 0.5, and for schedule consultation, its probability 0.4 is less than the probability threshold 0.5, so navigation and music will be screened out, while schedule consultation will be filtered out.

[0334] Table 7

[0335] Target business type Probability Threshold navigation 0.5 music 0.5 Radio 0.5 communication 0.7 Short message 0.7 WeChat 0.7 Schedule consultation 0.5 news 0.4 Questions and Answers 0.4 air conditioner 0.4 Car Control 0.8 system 0.7 Maintenance 0.5

[0336] If the probability of the target business type does not meet the probability threshold, it indicates that the target business type is less likely to be the business type targeted by the user's voice interaction intention. If the target business type is subsequently processed, the probability of false awakening will be higher. If a false awakening occurs, it will interfere with the user and cause excessive load on the terminal. Therefore, by filtering out the target business type, the device will not respond to voice commands whose semantic information is that of the target business type, thereby reducing the probability of false awakening and avoiding interference to the user and load on the terminal due to false awakening.

[0337] It should be noted that step 403 is an optional step, not a mandatory step. In some possible embodiments, after one or more target service types are acquired, the following steps are performed according to all acquired target service types.

[0338] 404. The terminal collects voice commands.

[0339] In some possible embodiments, the terminal may start monitoring when it is determined that a target event has been detected, and end monitoring when the monitoring duration has elapsed. During the monitoring period, if the user utters a voice, the terminal may collect and obtain the voice command through the microphone. The time period from starting monitoring to ending monitoring may be referred to as a receiving window, and the monitoring duration may be set based on experiments, experience, or needs, and may be pre-stored in the terminal. Optionally, the same monitoring duration may be set for each service type, or different monitoring durations may be set for different service types, and the mapping relationship between the service type and the monitoring duration may be stored in the terminal, which is not limited in this embodiment.

[0340] 405. The terminal obtains a first service corresponding to the semantic information according to the semantic information corresponding to the voice instruction.

[0341] In some possible embodiments, the terminal may perform automatic speech recognition (ASR) on the voice command to obtain text information; perform semantic recognition on the text information to obtain semantic information; and query and obtain the first service based on the semantic information. For example, if the semantic information is "Xiaohua, please help me plan a route to Building YY in XX Community", the first service is "navigate to Building YY in XX Community", and if the semantic information is "Xiaohua, please help me play a ZZ song", the first service is "play a ZZ song".

[0342] 406. If the service type of the first service is any target service type in the service type set, the terminal executes the first service according to the voice instruction.

[0343] The terminal can compare the service type of the first service with each target service type in the service type set. If the service type of the first service is the same as any target service type, it indicates that the service type expressed by the user's voice is within the range of the predicted service type, the terminal correctly predicts the service type, and the user does have the intention of voice interaction, then the wake-up is successful, and the terminal will activate the voice interaction function, respond to the voice command, and execute the first service according to the voice command, that is, execute the service expressed by the voice command. Exemplarily, if the service type set is (navigation, music), and the first service is "navigation to Building YY, XX Community", then the service type of the first service is navigation, and the service type of the first service is the same as the target service type navigation in the service type set, then the terminal will navigate to Building YY, XX Community.

[0344] 407. If the service type of the first service is different from each target service type in the service type set, the terminal discards the voice instruction.

[0345] If the service type set does not include the service type of the first service, it indicates that the service type expressed by the user's voice is outside the range of the predicted service type, the terminal incorrectly predicts the service type, and the user has no intention of voice interaction. The terminal may not respond to the voice command, but discard the voice command, thereby avoiding false wake-up caused by service processing based on the voice command and saving the cache space occupied by the voice command. For example, if the service type set is (navigation, music), and the first service is "turn on the air conditioner", the service type of the first service is air conditioning, and the service type of the first service is different from each target service type in the service type set, then the terminal will not respond to the voice command, that is, the air conditioner will not be turned on.

[0346] In addition, the terminal can start timing when detecting the occurrence of a target event. If the recorded duration reaches a preset duration and the terminal does not receive a voice command, the terminal will exit monitoring.

[0347] It should be noted that step 407 is an optional step, not a mandatory step.

[0348] 408. The terminal updates the probability in the mapping relationship according to the semantic information corresponding to the voice command.

[0349] Through this optional method, whenever the target event occurs, the probability can be dynamically adjusted according to the semantic information expressed by the user this time, so as to continuously correct the probability through iteration by evaluating the correctness of the predicted target business type in a self-learning manner, so that the mapping relationship can be continuously optimized with the occurrence of the target event and the semantics expressed by the user, gradually approaching the user's personal behavioral habits, and ensuring that the mapping relationship is more accurate.

[0350] Specifically, the updating method may include one or more of the following methods (1) to (3).

[0351] Method (1) If the service type of the first service is any target service type in the service type set, increase the probability of the service type of the first service in the mapping relationship.

[0352] Exemplarily, if it is determined that after the operation of the main switch is detected, the service type set is (navigation, music), and the service type of the first service is navigation, the terminal will increase the probability of corresponding to navigation.

[0353] If the user always requests to execute the target business type Y service through voice every time the target event X is detected, it indicates that the target business type Y happens to be the business type that the user has the voice interaction intention after the target event X occurs. Then, through this optional method, the probability of the target business type Y will continue to increase. Therefore, when the target event is detected again subsequently, the probability of the target business type Y will meet the probability threshold, so that the target business type Y will be screened out. Then, if the first business corresponding to the semantic information corresponding to the voice command is the target business type Y service, the terminal will respond to the voice command and execute the target business type Y service.

[0354] It should be noted that if the business type of the first business is different from each target business type in the business type set, the terminal can write the business type of the first business into the mapping relationship. Specifically, if the mapping relationship does not contain the target event, or the mapping relationship does not contain the business type of the first business, or the target event in the mapping relationship does not correspond to the business type of the first business, then in any of these three cases, the terminal can write the target event and the business type of the first business into the mapping relationship, so that the target event and the business type of the first business are added to the mapping relationship.

[0355] Through this optional method, the next time the target event is detected, the mapping relationship can be queried to obtain the business type of the first business, and the business type of the first business can be obtained as the target business type. In this way, if during the historical operation of the terminal, whenever event X is detected, the user expresses the intention of voice interaction with business type Y through voice, by writing event X and business type Y into the mapping relationship, the mapping relationship will add event X and business type Y. Then, on the one hand, as the voice interaction process is executed, the correlation between events and business types can be excavated, and the mapping relationship can be supplemented and improved. On the other hand, as the business expands, the terminal can add new events and new business types to the mapping relationship to improve the scalability and timeliness of the mapping relationship.

[0356] In addition, optionally, a probability corresponding to the business type of the first business can be generated, and the generated probability can be written to the mapping relationship. Specifically, the probability can be determined based on the number of times the target event is detected and the number of times the semantic information of the voice command represents the business type of the first business. For example, if A target events are detected and the semantic information represents the business type of the first business B times, the ratio between A and B can be obtained as the probability. For example, if after detecting 10 occurrences of event X, the user has expressed the intention of voice interaction with business type Y 7 times through voice, the probability corresponding to business type Y can be obtained = 7 / 10 = 0.7, and then the business type Y and the probability 0.7 are written to the mapping relationship. Of course, the default probability can also be used as the probability corresponding to the business type of the first business, and the business type of the first business and the default probability are written to the mapping relationship. The default probability is subsequently adjusted by executing the process shown in step 408.

[0357] Mode (2): If the service type of the first service is different from each target service type in the service type set, the probability of corresponding each target service type in the service type set in the mapping relationship is reduced.

[0358] Exemplarily, if it is determined that after the operation of the main switch is detected, the service type set is (navigation, music), and the service type of the first service is air conditioning, the terminal will reduce the corresponding probabilities of navigation and music.

[0359] If the target event X corresponds to the target service type Y, and each time the target event X is detected, the user does not request to execute the service of the target service type Y through voice, indicating that the target service type Y is not the service type for which the user has the intention of voice interaction after the target event X occurs, then through this optional method, the probability of the target service type Y will continue to decrease. Therefore, when the target event X is detected again subsequently, the probability of the target service type Y will not meet the probability threshold, so that the target service type Y will be filtered out, and the terminal will not process the service of the target service type Y, thereby avoiding false wake-up.

[0360] Method (3) If the semantic information includes a wake-up word, increase the probability corresponding to the business type of the first business in the mapping relationship.

[0361] For example, if it is determined that after the operation of the main switch is detected, the semantic information is "Hello Xiaohua, navigate to the destination of a certain building in a certain community", the terminal will increase the probability of corresponding navigation.

[0362] If the target event X corresponds to the target business type Y, and the user says the wake-up word each time the target event X is detected, indicating that the target business type Y happens to be the business type for which the user has the intention of voice interaction after the target event X occurs, then through this optional method, the probability of the target business type Y will continue to increase, so when the target event is detected again subsequently, the probability of the target business type Y will meet the probability threshold, so that the target business type Y will be screened out, then if the voice command expresses the semantic information of the target business type Y, the terminal will respond to the voice command to process the business.

[0363] It should be noted that step 408 is an optional step, not a mandatory step.

[0364] This embodiment provides a method for triggering voice interaction without a wake-up word. According to the target event that can trigger voice interaction, a set of business types that the user has the intention of voice interaction is predicted. If the business type of the first business expressed by the voice command is the predicted target business type, the first business is executed. By eliminating the tedious operation of the user needing to frequently say the wake-up word during the voice interaction process, the problem of low voice interaction efficiency caused by frequently saying the wake-up word is solved, making the operation more convenient, and making the voice interaction process more natural and humanized, thereby improving the user experience of voice interaction. In addition, the target event covers multiple modes, and the target event of any mode can trigger the voice interaction function of the corresponding business type, so that the wake-up function without the wake-up word can be supported in a variety of application scenarios, expanding the scope of application.

[0365] In combination with the above method embodiment, a specific software architecture of the present application is exemplarily introduced below.

[0366] Figure 5 This is a software architecture diagram of a voice interaction system provided in an embodiment of the present application. The system includes the following functional modules, each of which can be a software module.

[0367] Voice activity detection (VAD) module or front-end speech module: used to collect audio signals, perform noise reduction and enhancement on the collected audio signals, and detect whether the audio signals are voice commands or non-voice commands. If the audio signal is a voice command, the voice command is input into the ASR module, where the non-voice command can be a noise signal, a music signal, etc.

[0368] Automatic speech recognition (ASR) module: used to receive voice commands from the VAD module or the front-end speech module, convert the voice commands into text information, and input the text information into the dialogue understanding module or the dialogue management module.

[0369] Multimodal detection module: used to detect target events. If a target event is detected, the target event is input into the user intention prediction module. For example, the multimodal detection module can receive notification messages pushed by the operating system or application, such as text messages, incoming calls, application recommendation messages, and alarm messages; or, it can be used to detect one or more modal operations, such as physical buttons or interface operations, voice commands; or, it can be used to detect environmental changes or business changes, such as temperature drops, air pollution exceeds the standard, and music is about to end.

[0370] User intention prediction module: used to receive target events from the multimodal detection module, consider the user operation continuity rules, user viewing, processing or response needs, the impact of the environment or business on user perception, predict the user's voice interaction intention, and output a business type set and the probability corresponding to each target business type in the business type set.

[0371] Spoken language understanding (SLU) module or dialog management (DM) module: used to identify user intentions based on the text information of the speech recognition module, convert user intentions into system operations, obtain the reply content of the dialogue based on the context state of the dialogue, and output the reply content to the answer generation module or speech synthesis module.

[0372] Historical data learning: Based on the user’s historical operation data, the data in the user intention prediction module is iteratively updated.

[0373] Response generator (RG) module / text to speech (TTS) module: used to convert the response content received from the dialogue management module into text information, output text information, or convert the response content into voice commands and play voice.

[0374] Combination Figure 5 The system architecture shown in the figure takes the vehicle-mounted terminal as an example. The vehicle-mounted terminal can be implemented by combining hardware and software. The structure of the vehicle-mounted terminal can be as follows: Figure 6 As shown, including:

[0375] Processor (CPU): used to access various functional modules on the memory or other storage, run various functional modules, and can also access the memory and audio manager through the data bus (D-BUS). In addition, the processor can access various cloud services and cloud service management modules through the network interface. The processor can also access the controller area network (CAN) bus through the gateway to read the data of the vehicle and various devices on the vehicle, and control the vehicle and various devices on the vehicle.

[0376] Storage includes memory and disk storage, and the stored content includes Figure 5 The various functional modules shown.

[0377] The audio manager is used to manage vehicle speakers, microphone arrays, or other audio devices.

[0378] The above introduces the voice interaction method of the embodiment of the present application. The following introduces the voice interaction device provided by the embodiment of the present application. It should be understood that the voice interaction device has any function of the terminal in the above-mentioned voice interaction method.

[0379] Figure 7 is a structural diagram of a voice interaction device provided in an embodiment of the present application, such as Figure 7 As shown, the device comprises:

[0380] The determination module 701 is used to execute step 401; the query module 702 is used to execute step 402; the collection module 703 is used to execute step 404; the acquisition module 704 is also used to execute step 405; and the processing module 705 is used to execute step 406.

[0381] Optionally, the determination module 701 is used to determine whether a first operation of the user is detected; and the query module 702 is used to execute situation (1) in step 402.

[0382] Optionally, the determination module 701 is used to receive a notification message from an operating system or an application; the query module 702 is used to execute situation (2) in step 402.

[0383] Optionally, the determination module 701 is used to determine whether the current environmental parameters meet the first condition; and the query module 702 is used to execute situation (3) in step 402.

[0384] Optionally, the determination module 701 is used to determine whether the progress of the current business meets the second condition; the query module 702 is used to execute situation (4) in step 402.

[0385] Optionally, the device further includes: a writing module, configured to write the service type of the first service into the mapping relationship if the service type of the first service is different from each target service type in the service type set.

[0386] Optionally, the query module 702 is further configured to query the mapping relationship according to the target event to obtain the service type set and the probability corresponding to each target service type in the service type set;

[0387] The device also includes: a filtering module, configured to execute step 408 .

[0388] Optionally, the device further includes: an updating module, configured to execute step 408 .

[0389] Optionally, the update module is specifically used to execute one or more of the methods (1) to (3) in step 408.

[0390] Optionally, the device further includes: a discarding module, configured to execute step 407.

[0391] It should be noted that: the voice interaction device provided in the above embodiment only uses the division of the above functional modules as an example for explanation during voice interaction. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the terminal is divided into different functional modules to complete all or part of the functions described above. In addition, the voice interaction device provided in the above embodiment and the voice interaction method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0392] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0393] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions, and the instructions can be executed by a processor of a terminal to complete the voice interaction method in each of the above embodiments. The computer-readable storage medium can be non-transitory. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device.

[0394] In an exemplary embodiment, a computer program product is further provided. The computer program product includes: a computer program code. When the computer program code is executed by a terminal, the terminal executes the voice interaction method in each of the above embodiments.

[0395] In an exemplary embodiment, a chip is provided, including a processor for calling and executing instructions stored in a memory, so that a device equipped with the chip executes the voice interaction method in each of the above-mentioned embodiments.

[0396] In an exemplary embodiment, another chip is provided, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the voice interaction method in the above-mentioned embodiments.

[0397] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer program instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program instructions can be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (digitalvideo disc, DVD)), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0398] The term "and / or" in this application is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this application generally indicates that the associated objects before and after are in an "or" relationship.

[0399] In the present application, the term "plurality" means two or more than two. For example, a plurality of data packets means two or more than two data packets.

[0400] In the present application, the terms "first", "second", etc. are used to distinguish identical items or similar items with substantially the same effects and functions. Those skilled in the art can understand that the terms "first", "second", etc. do not limit the quantity and execution order.

[0401] A person skilled in the art will understand that all or part of the steps of implementing the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0402] The above are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A voice interaction method, characterized in that: The method comprises: Determining that a target event occurs, wherein the target event is an event that can trigger voice interaction and the target event indicates voice interaction based on an application or a device; collecting a voice instruction, where the voice instruction indicates a first service; If the voice instruction is related to the target event, an application or device is controlled to execute the first service according to the voice instruction.

2. The method according to claim 1, characterized in that The method further comprises: If the voice command is irrelevant to the target event, the voice command is not responded to.

3. The method according to claim 1, characterized in that The target event is a first operation of the user; the first operation corresponds to one or more target second operations, and the one or more target second operations are continuous operations associated with the first operation.

4. The method according to claim 3, characterized in that The first operation includes one or more of an operation on a physical button, an operation on an interface, a voice command, and a browsing behavior.

5. The method according to claim 1, characterized in that: The target event is receiving a notification message from an operating system or an application; the notification message corresponds to message viewing or message processing for the notification message.

6. The method according to claim 5, characterized in that The notification message includes at least one of an incoming call notification, a text message, an instant messaging message, and an alarm message. The incoming call notification corresponds to answering an incoming call; the text message or instant messaging message corresponds to message viewing or message replying; the alarm message corresponds to fault handling or information query.

7. The method according to claim 1, characterized in that The target event is that the current environmental parameters satisfy the first condition; the environmental parameters correspond to the adjustment environmental parameters.

8. The method according to claim 7, characterized in that The environmental parameters include one or more of noise, temperature, humidity, brightness, dust concentration, and oil volume.

9. The method according to claim 1, characterized in that: The target event is that the progress of the current service satisfies the second condition; the current service corresponds to the service being executed by the application or device.

10. The method according to claim 9, characterized in that The current service includes at least one of the following: a navigation service, a music service, a temperature adjustment service performed by an air conditioner, a rotation service performed by a wiper, and a lowering service performed by a sunshade.

11. The method according to any one of claims 1 to 10, characterized in that The method further comprises: According to the target event, query the mapping relationship to obtain a business type set, where the business type set includes one or more target business types; If the service type of the first service is any target service type in the service type set, the application or device is controlled to execute the first service according to the voice instruction.

12. The method according to claim 11, characterized in that The process of establishing the mapping relationship includes: According to the historical records, a historical business executed in association with the historical target event is obtained, and the business type of the historical business and the historical target event are written into the mapping relationship; Call the machine learning model, input the sample target event into the machine learning model, output the business type, write the output business type and the sample target event into the mapping relationship, and the machine learning model is used to predict the business type based on the event.

13. The method according to claim 11, characterized in that The method further comprises: If the service type of the first service is different from each target service type in the service type set, the service type of the first service is written into the mapping relationship.

14. The method according to claim 11, characterized in that The querying the mapping relationship according to the target event to obtain the service type set includes: querying the mapping relationship according to the target event to obtain the service type set and the probability corresponding to each target service type in the service type set, wherein the probability indicates the possibility of executing the service of the corresponding target service type; If the service type of the first service is any target service type in the service type set, before executing the first service according to the voice instruction, the method further includes: filtering out target service types whose probabilities do not meet a probability threshold from the service type set.

15. The method according to claim 14, characterized in that The method further comprises: The probability in the mapping relationship is updated according to the semantic information corresponding to the voice instruction.

16. The method according to claim 15, characterized in that The updating of the probability in the mapping relationship according to the semantic information corresponding to the voice instruction includes any of the following: If the service type of the first service is any target service type in the service type set, increase the probability corresponding to the service type of the first service in the mapping relationship; If the service type of the first service is different from each target service type in the service type set, reducing the probability corresponding to each target service type in the service type set in the mapping relationship; If the semantic information includes a wake-up word, a probability corresponding to the service type of the first service in the mapping relationship is increased.

17. The method according to claim 11, characterized in that The method further comprises: If the service type of the first service is different from each target service type in the service type set, the voice instruction is discarded.

18. A voice interaction device, characterized in that: The device comprises: A determination module, configured to determine that a target event is detected, wherein the target event is an event that can trigger voice interaction and the target event indicates voice interaction based on an application or a device; A collection module, used for collecting voice instructions, where the voice instructions indicate a first service; A service execution module is used to control an application or device to execute the first service according to the voice instruction if the voice instruction is related to the target event.

19. The device according to claim 18, characterized in that The device is also used for: If the voice command is irrelevant to the target event, the voice command is not responded to.

20. The device according to claim 18, characterized in that The target event is a first operation of the user; the first operation corresponds to one or more target second operations, and the one or more target second operations are continuous operations associated with the first operation.

21. The device according to claim 20, characterized in that The first operation includes one or more of an operation on a physical button, an operation on an interface, a voice command, and a browsing behavior.

22. The device according to claim 18, characterized in that The target event is receiving a notification message from an operating system or an application; the notification message corresponds to message viewing or message processing for the notification message.

23. The device according to claim 22, characterized in that The notification message includes at least one of an incoming call notification, a text message, an instant messaging message, and an alarm message. The incoming call notification corresponds to answering an incoming call; the text message or instant messaging message corresponds to message viewing or message replying; the alarm message corresponds to fault handling or information query.

24. The device according to claim 18, characterized in that The target event is that the current environmental parameters satisfy the first condition; the environmental parameters correspond to the adjustment environmental parameters.

25. The device according to claim 24, characterized in that The environmental parameters include one or more of noise, temperature, humidity, brightness, dust concentration, and oil volume.

26. The device according to claim 18, characterized in that The target event is that the progress of the current service satisfies the second condition; the current service corresponds to the service being executed by the application or device.

27. The device according to claim 26, characterized in that The current service includes at least one of the following: a navigation service, a music service, a temperature adjustment service performed by an air conditioner, a rotation service performed by a wiper, and a lowering service performed by a sunshade.

28. The device according to any one of claims 18 to 27, characterized in that The device also includes: A query module, configured to query a mapping relationship according to the target event to obtain a service type set, wherein the service type set includes one or more target service types; The service execution module is configured to control an application or device to execute the first service according to the voice instruction if the service type of the first service is any target service type in the service type set.

29. The device according to claim 28, characterized in that The process of establishing the mapping relationship includes: According to the historical records, a historical business executed in association with the historical target event is obtained, and the business type of the historical business and the historical target event are written into the mapping relationship; Call the machine learning model, input the sample target event into the machine learning model, output the business type, write the output business type and the sample target event into the mapping relationship, and the machine learning model is used to predict the business type based on the event.

30. The device according to claim 29, characterized in that The device also includes: A writing module is used to write the business type of the first business into the mapping relationship if the business type of the first business is different from each target business type in the business type set.

31. The device according to claim 28, characterized in that The query module is further used to query the mapping relationship according to the target event to obtain the service type set and the probability corresponding to each target service type in the service type set, wherein the probability indicates the possibility of executing the service of the corresponding target service type; The device further includes: a filtering module, configured to filter out target service types whose probabilities do not meet a probability threshold from the one or more target service types.

32. The device according to claim 31, characterized in that The device also includes: An updating module is used to update the probability in the mapping relationship according to the semantic information corresponding to the voice instruction.

33. The device according to claim 32, characterized in that The update module is used to perform any of the following: If the service type of the first service is any target service type in the service type set, increase the probability corresponding to the service type of the first service in the mapping relationship; If the service type of the first service is different from each target service type in the service type set, reducing the probability corresponding to each target service type in the service type set in the mapping relationship; If the semantic information includes a wake-up word, a probability corresponding to the service type of the first service in the mapping relationship is increased.

34. The device according to claim 28, characterized in that The device also includes: The discarding module is used to discard the voice instruction if the service type of the first service is different from each target service type in the service type set.

35. A terminal, characterized in that: The terminal includes one or more processors and one or more memories, wherein at least one instruction is stored in the one or more memories, and the instruction is loaded and executed by the one or more processors to implement the voice interaction method as described in any one of claims 1 to claim 17.

36. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, which is loaded and executed by the processor to implement the voice interaction method as described in any one of claims 1 to 17.

Images