Information processing method and device based on artificial intelligence and storage medium

Abstract

The invention provides an information processing method and device based on artificial intelligence and a storage medium, relates to the technical field of artificial intelligence, and can analyze and process demand information of a user to obtain one or more instructions executable by a computer. The method comprises the following steps: acquiring demand information of a user through an AI client; wherein the AI client is arranged at a user terminal; processing the demand information based on a preset artificial intelligence model to obtain a first control instruction matched with the demand information; and executing an operation matched with the first control instruction, and outputting a result corresponding to the operation through the user terminal.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to an information processing method, device and storage medium based on artificial intelligence. Background Art

[0002] With the development of artificial intelligence, artificial intelligence (AI) assistants play an important role in improving work efficiency, providing personalized services, enhancing decision-making capabilities, improving quality of life, promoting innovation and development, achieving barrier-free communication, and coping with future challenges. With the continuous advancement of technology and the continuous expansion of application scenarios, the importance of AI assistants will become increasingly prominent.

[0003] At present, although existing AI assistants have made significant technological progress in the process of processing user commands, they are unable to fully summarize all the contents of user commands in the process of converting user commands into machine executable commands, and do not have a complete execution process in the execution of machine commands. As a result, AI assistants lack functions in the actual work process, increase resource consumption, and reduce user experience. Summary of the invention

[0004] The present application provides an information processing method, device and storage medium based on artificial intelligence, which can convert user instructions into an instruction sequence composed of multiple types of instructions and execute them according to a complete instruction execution process.

[0005] In order to achieve the above purpose, this application adopts the following technical solutions:

[0006] In a first aspect, the present application provides an information processing method based on artificial intelligence, which is applied to an AI assistant, the method comprising: obtaining user demand information through an AI client; wherein the AI ​​client is set in a user terminal; based on a preset artificial intelligence model, processing the demand information to obtain a first control instruction that matches the demand information; and executing an operation that matches the first control instruction.

[0007] According to the above technical means, the present application can convert the user's demand information into different types of instructions, execute the operation corresponding to the instruction, and output the result of the operation to the user. Converting the user's demand information into different types of instructions improves the user experience.

[0008] In a possible implementation, the first control instruction includes one or more of an operation instruction, an interaction instruction, and a query instruction.

[0009] According to the above technical means, the instructions generated according to the user's demand information are classified, and different types of instructions can be used to explain the user's demand information, so that the content of the first control instruction can include all the user's needs.

[0010] In a possible implementation, an operation matching the operation instruction is performed on the virtual terminal, wherein the virtual terminal is set on a cloud server.

[0011] According to the above technical means, the AI ​​assistant can perform operations on the virtual terminal that match the operation instructions, so as to complete the part of the demand information that needs to be executed by the virtual terminal. This reduces the user's manual operations and improves the user experience.

[0012] In a possible implementation, the information to be confirmed is fed back to the user through the AI ​​client, and the user's feedback result on the information to be confirmed is obtained through the AI ​​client.

[0013] According to the above technical means, the AI ​​assistant can interact with the user according to the content of the interactive instruction. The content output by the virtual terminal can be fed back to the user through the AI ​​client, meeting the user's needs. The information to be confirmed can also be fed back to the user, increasing the accuracy of the information processing process.

[0014] In a possible implementation, the interaction instruction is used to instruct the virtual terminal to interact with the user, and control the virtual terminal to display a virtual terminal interface through the AI ​​client, where the virtual terminal interface is an interface displayed by the virtual terminal.

[0015] According to the above technical means, when the user needs to enter personal information, he or she can interact with the virtual terminal through the AI ​​client on the terminal device, thereby improving the security of user information and the reliability of data.

[0016] In one possible implementation, an interface switching event notification is received from a virtual terminal. The interface switching event notification is used to indicate that a window switch has occurred in the virtual terminal interface. The switched virtual terminal interface information is input into the AI ​​model to obtain a third control instruction that matches the switched virtual terminal interface information. When the third control instruction is used to indicate that no interaction with the user is required, a termination instruction is sent to the virtual terminal, and the termination instruction is used to instruct to stop displaying the virtual terminal interface.

[0017] According to the above technical means, after the user completes the required operation, the direct communication between the virtual terminal and the user can be disconnected, and the AI ​​assistant continues to control the virtual terminal. The execution process of the interactive instructions is improved to avoid unnecessary operations on the virtual terminal after the user completes the required operation.

[0018] In a possible implementation, corresponding query results are obtained according to the query instruction.

[0019] According to the above technical means, the AI ​​assistant can query unknown information based on the user's historical information, the context of the preset artificial intelligence model and other data, thus avoiding the user's frequent participation in the information processing process and improving the user experience.

[0020] In a possible implementation, an artificial simulation operation is performed according to the operation instruction to generate an artificial simulation signal, and the operation corresponding to the artificial simulation signal is executed on the virtual terminal.

[0021] According to the above technical means, the AI ​​assistant can operate the virtual terminal by simulating manual operations, which can be applied to a wider range of applications, enriching the operation methods of the AI ​​assistant on the virtual terminal.

[0022] In a possible implementation, the output information and / or query result of the virtual terminal is obtained, the output information corresponds to the operation instruction, and the query result corresponds to the query instruction. The output information and / or the query result are processed by a preset artificial intelligence model to obtain a second control instruction. An operation matching the second control instruction is executed, and the second control instruction includes one or more of an operation instruction, an interaction instruction, and a query instruction.

[0023] According to the above technical means, by processing the output results and / or query results with a preset artificial intelligence model, it can be determined whether the existing output results and / or query results can meet the user's demand information. This avoids the user's need to repeatedly send out demand information when the output results cannot meet the user's needs. This improves the user experience.

[0024] In a possible implementation, instruction information is sent to a target application program among multiple applications for executing the operation instruction, and an execution result is received from the target application program.

[0025] Based on the above technical means, the AI ​​assistant can be operated through the API interface of the application, which improves the operating efficiency of the virtual terminal and simplifies the workflow.

[0026] In the second aspect, the present application provides an information processing method based on artificial intelligence, which is applied to an AI client, and the method includes: obtaining user demand information through a human-machine interface and / or a sensor, and sending the demand information to an AI assistant, so that the AI ​​assistant processes the demand information based on a preset artificial intelligence model, obtains a first control instruction matching the demand information, and executes an operation matching the first control instruction to obtain a corresponding operation result. Receive the operation result from the AI ​​assistant, and output the operation result to the user.

[0027] According to the above technical means, the present application can obtain the user's demand information through the human-machine interface and / or sensor, and output the operation results to the user through the AI ​​client, so that the acquisition of user demand information is more comprehensive. Outputting the operation results to the user enhances the efficiency of interaction with the user.

[0028] In a possible implementation, the first control instruction includes one or more of an operation instruction, an interaction instruction, and a query instruction.

[0029] According to the above technical means, the present application can classify the instructions generated according to the user's demand information, and use different types of instructions to explain the user's demand information, so that the content of the first control instruction can include all the user's needs.

[0030] In a possible implementation, the information to be confirmed is received from the AI ​​assistant, and the information to be confirmed is outputted. The user's feedback result on the information to be confirmed is obtained, and the feedback result of the information to be confirmed is sent to the AI ​​assistant.

[0031] According to the above technical means, the present application can improve the processing flow of user demand information and enhance user experience by sending information to be confirmed to the user and obtaining feedback results of the information to be confirmed.

[0032] In a possible implementation, a virtual terminal interface is received from a virtual terminal and displayed. The virtual terminal interface is output information generated by the virtual terminal, and the virtual terminal is set in a cloud server.

[0033] According to the above technical means, the present application can realize direct interaction between the virtual terminal and the user. When the user is required to input private data, direct interaction between the user and the virtual terminal can improve the security of the user data and the reliability of the data.

[0034] In a possible implementation, a termination instruction is received from the virtual terminal, and in response to the termination instruction, the display of the virtual terminal interface is stopped.

[0035] According to the above technical means, the present application can end the direct interaction between the user and the virtual terminal when the user completes the direct interaction with the virtual terminal, thereby avoiding the user from performing more operations.

[0036] In a possible implementation, the user's demand information or feedback result of the information to be confirmed is obtained through a human-machine interface and / or a sensor, and the operation result or the information to be confirmed is output to the user through the human-machine interface and / or the sensor.

[0037] According to the above technical means, the present application can fully obtain the user's demand information and the feedback results of the information to be confirmed, and can also output the operation results or the information to be confirmed to the user, making the ways of obtaining the user's demand information more abundant.

[0038] In a third aspect, the present application provides an information processing device, which includes: a communication unit and a processing unit.

[0039] The communication unit is used to obtain the user's demand information through the AI ​​client, wherein the AI ​​client is set in the user terminal.

[0040] The processing unit is used to process the demand information based on a preset artificial intelligence model to obtain a first control instruction matching the demand information.

[0041] The processing unit is further used to execute an operation matching the first control instruction and output a result corresponding to the operation through the user terminal.

[0042] In a possible implementation, the first control instruction includes one or more of an operation instruction, an interaction instruction, and a query instruction.

[0043] In a possible implementation, the processing unit is further configured to execute an operation matching the operation instruction on the virtual terminal, wherein the virtual terminal is arranged on a cloud server, and the configuration information of the virtual terminal is the same as the configuration information of the user terminal.

[0044] In a possible implementation, the processing unit is further configured to feed back the information to be confirmed to the user through the AI ​​client, and obtain the user's feedback result on the information to be confirmed through the AI ​​client.

[0045] In a possible implementation, the processing unit is further used to control the virtual terminal to output the information to be confirmed through the AI ​​client.

[0046] In a possible implementation, the processing unit is further used to obtain corresponding query results according to the query instruction, and output the query results through the AI ​​client.

[0047] In a possible implementation, the processing unit is further configured to perform artificial simulation operations according to the operation instructions to generate simulation signals. The processing unit is further configured to execute operations corresponding to the simulation signals on the virtual terminal.

[0048] In a possible implementation, the processing unit is further used to obtain output information of the virtual terminal, the output information corresponds to the operation instruction, and the processing unit is further used to process the output information through a preset artificial intelligence model to obtain a second control instruction, and control the virtual terminal to perform an operation corresponding to the second control instruction.

[0049] In a possible implementation, the processing unit is further configured to send indication information to a target application program among the multiple applications for executing the operation instruction. The processing unit is further configured to receive an execution result from the target application program.

[0050] In a fourth aspect, the present application provides an information processing device based on artificial intelligence, which is applied to an AI client. The device includes: a communication unit and a processing unit:

[0051] The communication unit is used to obtain the user's demand information through the human-machine interface and / or the sensor, and send the demand information to the AI ​​assistant, so that the AI ​​assistant processes the demand information based on the preset artificial intelligence model, obtains the first control instruction matching the demand information, and executes the operation matching the first control instruction to obtain the corresponding operation result. The processing unit is used to receive the operation result from the AI ​​assistant and output the operation result to the user.

[0052] In a possible implementation, the first control instruction includes one or more of an operation instruction, an interaction instruction, and a query instruction.

[0053] In a possible implementation, the processing unit is further configured to receive information to be confirmed from the AI ​​assistant and input the information to be confirmed, obtain feedback from the user on the information to be confirmed, and send the feedback to the AI ​​assistant.

[0054] In a possible implementation, the communication unit is further configured to receive a virtual terminal interface from the virtual terminal and display the virtual terminal interface. The virtual terminal interface is output information generated by the virtual terminal, and the virtual terminal is set on a cloud server.

[0055] In a possible implementation, the processing unit is further configured to receive a termination instruction from the virtual terminal, and in response to the termination instruction, stop displaying the virtual terminal interface.

[0056] In a possible implementation, the communication unit is further configured to obtain the user's demand information or feedback results of the information to be confirmed through the human-machine interface and / or the sensor, and output the operation results or the information to be confirmed to the user through the human-machine interface and / or the sensor.

[0057] In a fifth aspect, the present application provides an information processing system, the system comprising an AI assistant and an AI client, wherein the AI ​​assistant is used to execute the method of the first aspect, and the AI ​​client is used to execute the method of the second aspect.

[0058] In a sixth aspect, the present application provides an electronic device, comprising: a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run a computer program or instruction to implement the information processing method based on artificial intelligence as described in the first aspect and any possible implementation of the first aspect.

[0059] In the seventh aspect, the present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a terminal, the terminal executes the artificial intelligence-based information processing method described in the first aspect and any possible implementation of the first aspect.

[0060] In an eighth aspect, the present application provides a computer program product comprising instructions, which, when the computer program product is run on an electronic device, enables the electronic device to execute the artificial intelligence-based information processing method as described in the first aspect and any possible implementation of the first aspect.

[0061] In a ninth aspect, the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run a computer program or instructions to implement the artificial intelligence-based information processing method as described in the first aspect and any possible implementation method of the first aspect.

[0062] Specifically, the chip provided in the present application also includes a memory for storing computer programs or instructions. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A schematic diagram of the structure of a cloud terminal technology provided in an embodiment of the present application;

[0064] Figure 2 A schematic diagram of the structure of an information processing system based on artificial intelligence provided in an embodiment of the present application;

[0065] Figure 3 A schematic diagram of a deployment method of an information processing system based on artificial intelligence provided in an embodiment of the present application;

[0066] Figure 4 A schematic diagram of another deployment method of an information processing system based on artificial intelligence provided in an embodiment of the present application;

[0067] Figure 5 The process of an information processing method based on artificial intelligence provided in the embodiment of the present application Figure 1 ;

[0068] Figure 6 The process of an information processing method based on artificial intelligence provided in the embodiment of the present application Figure 2 ;

[0069] Figure 7 The process of an information processing method based on artificial intelligence provided in the embodiment of the present application Figure 3 ;

[0070] Figure 8 The process of an information processing method based on artificial intelligence provided in the embodiment of the present application Figure 4 ;

[0071] Fig. 9 A schematic diagram of the structure of an information processing device based on artificial intelligence provided in an embodiment of the present application Figure 1 ;

[0072] Fig.10 A schematic diagram of the structure of an information processing device based on artificial intelligence provided in an embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0073] The following is a detailed description of the artificial intelligence-based information processing method, device and storage medium provided in the embodiments of the present application in conjunction with the accompanying drawings.

[0074] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0075] The terms "first" and "second" and the like in the specification and drawings of this application are used to distinguish different objects, or to distinguish different processing of the same object, rather than to describe a specific order of objects.

[0076] In addition, the terms "including" and "having" and any variations thereof mentioned in the description of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0077] It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0078] In the description of the present application, unless otherwise specified, “plurality” means two or more.

[0079] The following explains the terms involved in the embodiments of the present application to facilitate the reader's understanding.

[0080] (1) Cloud terminal technology: It is a terminal device solution based on cloud computing. It stores the desktop environment, applications and data of traditional PCs, mobile phones and other terminals on cloud servers, which are called virtual terminals. Users remotely access and operate virtual terminals through lightweight terminal devices or applications on traditional terminal devices. The images and sounds output by various software on the virtual terminal are transmitted to the terminal device for display through audio and video streaming. Figure 1 As shown, this is its basic structure.

[0081] (2) Virtual terminal: One or a group of computing units allocated on a cloud server, such as a virtual machine, container, etc. The same operating system as a normal terminal can be deployed on the virtual terminal, such as Android, Windows, Linux, etc. Figure 1 As shown in the figure, like ordinary terminals, various applications can be installed and run on virtual terminals. The cloud terminal client is a software on the terminal hardware device that provides an input and output interface for the virtual terminal. The virtual terminal and the cloud terminal client are connected through the network. The user's interaction with the cloud terminal is basically the same as that of an ordinary terminal.

[0082] (3) Artificial Intelligence Assistant: Also known as AI Agent or intelligent voice assistant. An intelligent assistant in the field of artificial intelligence that interacts with users through natural language processing technology. It can realize functions such as smart home control, information query, and entertainment playback. It provides users with convenient services and experience. The AI ​​assistant in this application only refers to software that implements the AI ​​assistant functions defined in the technical solution section, and should not be completely corresponding to the AI ​​assistant generally defined in the industry.

[0083] The above is a brief introduction to some concepts involved in the embodiments of this application.

[0084] At present, AI assistants can convert user demand information into computer executable instructions when processing user instructions. However, they cannot fully summarize all the contents of user instructions, and do not have a perfect execution process in the execution of machine instructions, which leads to an increase in the error rate and resource consumption of AI assistants in the actual work process, which reduces the user experience.

[0085] In view of this, an embodiment of the present application provides an information processing method based on artificial intelligence, which can convert user instructions into an instruction sequence consisting of one or more different types of instructions, and execute them according to a complete instruction execution process.

[0086] In one example, Figure 2 As shown, a communication system provided by the present application. The communication system may include a cloud server and a terminal device. The cloud server is connected to the terminal device in communication. For example, the connection may be via a wireless network (such as a 5G network).

[0087] Among them, the cloud server can be used to process the user's demand information based on the preset artificial intelligence model to obtain multiple computer executable instructions. It can also perform operations matching the instructions. The terminal device can be used to call the human-machine interface and sensor through the AI ​​client to obtain the user's demand information. The processing results of the user's demand information can also be displayed to the user through the human-machine interface and / or sensor.

[0088] In one possible implementation, combining Figure 2 ,like Figure 3 As shown, the AI ​​assistant can be deployed on a cloud server. The AI ​​client communicates with the cloud server. The cloud server and the terminal device can include the following programs or devices:

[0089] The human-machine interface can be used to obtain the user's demand information and can also be used to output the results of the operation performed according to the demand information. For example, the human-machine interface can include a display, a touch interface, a voice interface and other devices.

[0090] The sensor may be used to obtain image information, vibration information, etc. of the user, and may also be used to output image information, vibration information, etc. For example, the sensor may include a gyroscope, a camera, and other devices.

[0091] AI client can be used to provide an interactive interface for users. It can also be used to obtain user demand information through human-machine interfaces and sensors. It can also be used to display interactive information to users through human-machine interfaces and sensors.

[0092] The AI ​​assistant can be used to obtain user demand information through the AI ​​client. It can also be used to select a suitable preset artificial intelligence model for processing based on the user's demand information. It can also be used to operate the virtual terminal. It can also control the connection between the virtual terminal and the AI ​​client. It can also obtain the output content of the virtual terminal and send it to the AI ​​client. It can also be used to perform query operations based on user data to obtain the information required by the user's needs.

[0093] The preset artificial intelligence model can be used to analyze and process user demand information, and can also be used to process the content output by the virtual terminal, and generate instructions based on the user demand information and / or the content output by the virtual terminal.

[0094] A virtual terminal can install and run various application software to meet user needs.

[0095] In another possible implementation, combining Figure 2 ,like Figure 4 , the AI ​​assistant can be deployed in the terminal device. The AI ​​assistant is connected to the cloud server in communication. When the AI ​​assistant is deployed in the terminal device, the user data will also be stored in the terminal device, which can improve the security of the data. The deployment of the AI ​​assistant in the terminal device does not affect the solution and technical effects of the present invention.

[0096] In another possible implementation, combining Figure 2 , AI assistants can also be deployed in virtual terminals. Deploying AI assistants in virtual terminals does not affect the solutions and technical effects of this application.

[0097] In another possible implementation, combining Figure 2 , all or part of the preset artificial intelligence model can be deployed in the virtual terminal. The deployment of all or part of the preset artificial intelligence model in the virtual terminal does not affect the solution and technical effects of the present application.

[0098] For example, Figure 2 FIG. 2 shows a schematic diagram of the structure of an information processing system 20 provided in an embodiment of the present application. Figure 2 As shown, the communication system may include: a terminal device 201 and a server 202. The server 202 may establish a connection with the terminal device 201 via a wireless network.

[0099] The server 202 can be used to deploy a preset artificial intelligence model and a virtual terminal to process user demand information. The server 202 can also send a virtual terminal and an output result of a preset artificial intelligence model to the terminal device 201 in response to a request from the terminal device 201.

[0100] In some embodiments, the server 202 may be a single server, or a server cluster composed of multiple servers. In some implementations, the server cluster may also be a distributed cluster. The present disclosure does not limit the specific implementation of the server 202.

[0101] Among them, when server 202 is a single server, the server can provide the mobile device with a virtual terminal that can be operated by the AI ​​assistant selected by the user, and can also provide the mobile device with a preset artificial intelligence model. When server 202 is a server cluster composed of multiple servers, some of the multiple servers can be used to provide the mobile device with a virtual terminal that can be operated by the AI ​​assistant selected by the user, and another part can be used to provide the mobile device with a virtual terminal that can be operated by other AI assistants, so that the mobile device can change the AI ​​assistant according to the user's needs.

[0102] In a possible implementation, the server 202 may also have a parameter control interface. Developers can use the parameter control interface to configure, debug, and update the preset artificial intelligence model, and can also configure, debug, and update the virtual terminal. In response to the user's demand information, the server can analyze and process the user's demand information.

[0103] The terminal device 201 can be used to obtain user demand information. The terminal device 201 can have an AI client. When the terminal device 201 starts the AI ​​client, the terminal device 201 can call the human-machine interface and the sensor through the AI ​​client to obtain the user demand information.

[0104] In some embodiments, the mobile device may be a mobile phone, tablet computer, desktop, laptop, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, cellular phone, personal digital assistant (PDA), augmented reality (AR)\virtual reality (VR) device, etc., which can install and use AI client, and the present disclosure does not impose any special restrictions on the specific form of the mobile device. It can interact with the user through one or more methods such as keyboard, touchpad, touch screen, remote control, voice interaction or handwriting device.

[0105] Optionally, the above Figure 2 In the communication system shown, the server 202 can be connected to at least one terminal device 201. The present disclosure does not limit the number and type of the terminal devices 201.

[0106] In addition, the actions, terms, etc. involved in the various embodiments of the present application can refer to each other without limitation. The message name or parameter name in the message exchanged between the various devices in the embodiments of the present application is only an example, and other names can also be used in the specific implementation without limitation.

[0107] The following describes the information processing method based on artificial intelligence provided by the embodiment of the present application in conjunction with the accompanying drawings. Among them, the actions, terms, etc. involved in the various embodiments of the present application can refer to each other without limitation. The message name or parameter name in the message exchanged between the various devices in the embodiment of the present application is only an example, and other names can also be used in the specific implementation without limitation. The actions involved in the various embodiments of the present application are only an example, and other names can also be used in the specific implementation, such as: "included in" in the embodiment of the present application can also be replaced by "carried on" or "carried in", etc.

[0108] In one embodiment, Figure 4 As shown, a schematic diagram of the deployment structure of an information processing system proposed in an embodiment of the present application includes a cloud and a terminal device. The terminal device is deployed with an AI client. Human-machine interface, sensor and other devices. The cloud is deployed with a preset artificial intelligence model, a virtual terminal, an AI assistant, and saves user data.

[0109] Among them, the AI ​​client obtains the user's demand information and displays the output results of the cloud to the user by calling the human-machine interface and sensor. The human-machine interface can be a display screen, a touch interface, a voice interface and other devices. The sensor can be a gyroscope, a camera and other devices. The preset artificial intelligence model is one or more artificial intelligence models set and installed in advance. User data includes historical data of users using the information processing system, conversations of the preset artificial intelligence model, context and other user-related data.

[0110] In another embodiment, Figure 5 As shown, an embodiment of the present application proposes an information processing method based on artificial intelligence, which includes: S501 to S503.

[0111] S501. The AI ​​client obtains the user's demand information and sends the demand information to the AI ​​assistant.

[0112] Among them, the AI ​​client is set in the user terminal. The AI ​​client can obtain the user's demand information through the human-machine interface and sensor of the user terminal. The AI ​​client can be an AI assistant client. The AI ​​assistant client can be installed on the user's terminal device.

[0113] In one application scenario, the AI ​​assistant client can respond to the user's trigger operation and provide the user with an open natural language interaction interface. The user can use the natural language interaction interface to express the required information in natural language such as voice or text.

[0114] S502: The AI ​​assistant receives user demand information.

[0115] In one example, the AI ​​assistant can obtain the user's demand information by interacting with the user. For example, in response to a trigger operation, the AI ​​assistant client can output a voice to inquire about the user's needs. The user's voice is collected through the voice interface of the terminal device, and the user's natural language is converted into text information and recognized through natural language processing technology (NLP) to obtain the user's demand information. In this way, the AI ​​assistant client can send the obtained user's demand information to the AI ​​assistant.

[0116] In an application scenario, such as Figure 3As shown, the AI ​​assistant is deployed on a cloud server. The AI ​​assistant can obtain the user's demand information in the AI ​​assistant client through a wireless network. For details, please refer to the description of the following embodiment 1, which will not be repeated here.

[0117] In another application scenario, Figure 4 As shown, the AI ​​assistant is deployed on the terminal device. The AI ​​assistant can directly obtain the user's demand information in the AI ​​assistant client.

[0118] S503: The AI ​​assistant processes the demand information based on a preset artificial intelligence model to obtain a first control instruction that matches the demand information.

[0119] The preset artificial intelligence model can be used to analyze and process the input information to obtain the first control instruction. The preset artificial intelligence model can be one or more. The virtual terminal is set on the cloud server.

[0120] In one example, the first control instruction may include one or more of an operation instruction, an interaction instruction, and a query instruction. The three different types of instructions are explained as follows.

[0121] 1. Operation instructions.

[0122] Among them, the operation instruction is used to instruct the AI ​​assistant to perform an operation on the virtual terminal that matches the operation instruction.

[0123] For example, the user demand is: set an alarm for 8 o'clock tomorrow morning. The operation instruction corresponding to the user demand is used to instruct to set an alarm for 8 o'clock the next day. In response to the operation instruction, the AI ​​assistant can control the virtual terminal to open the clock application and set the alarm for 8 o'clock the next day.

[0124] 2. Interactive instructions.

[0125] Among them, the interaction instructions are used to instruct the AI ​​assistant to interact with the user through the AI ​​assistant client.

[0126] For example, the user demand is: play a piece of music. The interaction instruction corresponding to the user demand is used to instruct the user to feedback the music to be played. The AI ​​assistant can respond to the interaction instruction, obtain the music to be played, and output the query information of whether the music is played.

[0127] Furthermore, if the user confirms to play, the AI ​​assistant can call the speaker of the terminal device through the AI ​​assistant client to play the music.

[0128] 3. Query instructions.

[0129] Among them, the query instruction is used to instruct the AI ​​assistant to retrieve information that meets the user's needs and output the search results to the user.

[0130] For example, the user's requirement is: What is the phone number of classmate A? The query instruction corresponding to the user's requirement is used to instruct to query the phone number of classmate A. The AI ​​assistant can respond to the query instruction and retrieve the user's personal information to obtain the phone number of classmate A.

[0131] Among them, a preset artificial intelligence model can generate one or more types of instructions. Different types of instructions can be distinguished by the type parameters contained in the instructions.

[0132] In one possible implementation, a preset artificial intelligence model can generate multiple types of instructions. The generated instruction content may include an instruction type parameter, and the AI ​​assistant can determine the instruction type based on the instruction type parameter.

[0133] In another possible implementation, a preset artificial intelligence model can generate one type of instruction. Since the preset artificial intelligence model only generates one type of instruction, the instruction type parameter may not be included in the instruction, and the AI ​​assistant can determine the instruction type based on the type of the preset artificial intelligence model.

[0134] In one example, the user's demand information is input into a preset artificial intelligence model to output one or more tasks, a task may include one or more instruction sequences, and an instruction sequence may include one or more instructions. Multiple instructions in each instruction sequence may be executed in serial or parallel.

[0135] In an application scenario, such as Figure 3 As shown, the AI ​​assistant is deployed on a cloud server. The AI ​​assistant can send the user's demand information to the corresponding preset artificial intelligence model. For details, please refer to the description of the following embodiment 1, which will not be repeated here.

[0136] In another application scenario, Figure 4 As shown, the AI ​​assistant is deployed on the terminal device. The AI ​​assistant can send the user's demand information to the corresponding preset artificial intelligence model through the wireless network.

[0137] S504: The AI ​​assistant executes an operation matching the first control instruction and obtains a result corresponding to the operation.

[0138] The operation matched with the first control instruction indicates the operation corresponding to the operation information contained in the first control instruction. The result corresponding to the operation indicates the content output by the virtual terminal or AI assistant after the first control instruction is executed. The result corresponding to the operation may be an image, sound, data, etc.

[0139] In one example, when the first control instruction is an operation instruction, an operation matching the operation instruction is performed on the virtual terminal. When the first control instruction is an interaction instruction, information to be confirmed is fed back to the user through the AI ​​assistant client, and the user's feedback result on the information to be confirmed is obtained through the AI ​​assistant client. When the first control instruction is a query instruction, the corresponding query result is obtained according to the query instruction.

[0140] Among them, the AI ​​assistant can control the virtual terminal to output the information to be confirmed through the AI ​​assistant client.

[0141] In another example, the AI ​​assistant can obtain the result corresponding to the operation, process the result corresponding to the operation through a preset artificial intelligence model, and obtain a second control instruction.

[0142] Wherein, when the first control instruction is an operation instruction, the result corresponding to the operation is the result output by the virtual terminal when the AI ​​assistant performs the operation matching the operation instruction on the virtual terminal. When the first control instruction is an interaction instruction, the result corresponding to the operation is the feedback result of the user on the confirmation information. When the first control instruction is a query instruction, the result corresponding to the operation is the query result. The second control instruction can be an operation instruction, an interaction instruction, or a query instruction.

[0143] In an application scenario, such as Figure 3 As shown, the AI ​​assistant is deployed on a cloud server. The AI ​​assistant can perform operations corresponding to instructions on the virtual terminal and user data. The AI ​​assistant can input the results of the operation output into the AI ​​assistant client via a wireless network. For details, please refer to the description of the following embodiment 1, which will not be repeated here.

[0144] In another application scenario, Figure 4 As shown, the AI ​​assistant is deployed on the terminal device. The AI ​​assistant can perform operations corresponding to instructions on the virtual terminal and user data through the wireless network. The AI ​​assistant can also obtain user demand information directly through the AI ​​assistant client.

[0145] S505. The AI ​​assistant client receives the operation results from the AI ​​assistant and outputs the operation results to the user.

[0146] Among them, the AI ​​assistant client can output operation results to the user through the human-computer interface and / or sensors.

[0147] In one example, the user's request is to play a song. The corresponding operation result is the audio information of the song. The AI ​​assistant client plays the audio information for the user through the speaker.

[0148] Based on the above Figure 5The information processing method based on artificial intelligence in this application can split the user's demand information into one or more tasks, and convert the user's natural language into one or more computer-executable instructions, thus achieving full coverage of the user's demand information, improving the instruction execution process and enhancing the user experience.

[0149] In another embodiment (embodiment 1), as Figure 6 As shown, it is a schematic diagram of another user instruction processing flow provided by the present application, which may include the following steps S601 to S606.

[0150] S601. The AI ​​assistant client responds to the user's startup operation, displays an interactive interface to the user, and obtains the demand information input by the user.

[0151] Among them, after the AI ​​assistant client obtains the demand information input by the user, it sends the user's demand information to the AI ​​assistant.

[0152] In one example, a user uses voice input to input the demand information such as "book an airport shuttle for my flight tomorrow afternoon, and leave from the company gate two hours in advance." The AI ​​assistant client obtains the voice data through the human-computer interface and sends the voice data to the AI ​​assistant.

[0153] S602, the AI ​​assistant receives user demand information and sends the user demand information to a preset artificial intelligence model.

[0154] The preset artificial intelligence model may be a generative artificial intelligence model, for example, a pre-trained large language model (LLM), a large multimodal model (LMM), etc. The AI ​​assistant may perform a preliminary analysis of the user's demand information and select a corresponding artificial intelligence model for further processing according to the demand.

[0155] In one example, the AI ​​assistant client sends the demand information to the AI ​​assistant. After preliminary analysis, the AI ​​assistant needs to input the voice data into the voice recognition model for processing to generate the demand information in text form. The AI ​​assistant obtains the generated text-based demand information and sends the text-based demand information to the preset artificial intelligence model that processes natural language instructions.

[0156] One possible implementation is that when the AI ​​assistant calls the preset AI model, if a session exists, it continues to use the previously existing session and its context. If a session does not exist, the AI ​​assistant can create a session and its context. The AI ​​assistant can add the user's demand information to the session context, send the session context data to the preset AI model, and call the preset AI model.

[0157] Among them, the conversation is the interaction process when the AI ​​assistant calls the preset artificial intelligence model. The context is the input and output information generated during the interaction between the AI ​​and the preset artificial intelligence model. The AI ​​assistant can create and maintain conversations and contexts. The context can be changed, and the context will be different for different rounds of model calls. For example, each round of output of the preset artificial intelligence model will be added to the context. At the same time, the AI ​​assistant can also add other data to the context as needed, or delete certain data from the context.

[0158] S603, presetting the artificial intelligence model to process the user's demand information.

[0159] In one example, a preset artificial intelligence model analyzes and processes the user's demand information to obtain an execution plan. The execution plan includes multiple tasks. For example, after analyzing and processing "book a shuttle for my flight tomorrow afternoon, leaving from the company gate two hours in advance", the execution plan obtained includes three tasks:

[0160] Task 1: Query the user data to determine the specific address of "Company".

[0161] Task 2: Check the detailed information of tomorrow afternoon's flight through the schedule APP in the virtual terminal.

[0162] Task 3: Book a ride-hailing vehicle through the online car-hailing app in the virtual terminal.

[0163] Each task includes one or more instruction sequences, and an instruction sequence includes one or more instructions. The instructions are not related to each other and can be executed serially or in parallel. The preset artificial intelligence model outputs the execution plan to the AI ​​assistant. The AI ​​assistant adds the execution plan to the conversation context.

[0164] S604: The AI ​​assistant executes the interaction instructions generated by the preset artificial intelligence model and interacts with the user through the AI ​​assistant client.

[0165] The interactive instruction includes interactive information, which may be text information, and / or image information, and / or audio information.

[0166] In one example, after the preset artificial intelligence model recognizes the user's voice data, another preset artificial intelligence model analyzes and determines that the output text content requires user confirmation, then an interaction instruction is generated. The AI ​​assistant executes the interaction instruction, displays the text content to the user through the AI ​​assistant client, and requests the user to confirm.

[0167] In an application scenario, a user needs to interact with a virtual terminal. For details, reference may be made to the description of the following second embodiment, which will not be described in detail here.

[0168] S605: The AI ​​assistant executes the query instruction generated by the preset artificial intelligence model.

[0169] The query instruction includes the information to be obtained, search conditions, etc.

[0170] In one example, the above-mentioned Task 1 requires querying the specific address of the company, and the preset artificial intelligence model generates a query instruction to instruct to retrieve the specific address of the user's company. The AI ​​assistant can use vector database technology to store and retrieve user data. After the query instruction is executed, the AI ​​assistant obtains the retrieval results, adds the retrieval results to the session context, and re-requests the preset artificial intelligence model to process the user's demand information. For example, in Task 1, the specific address of the "company" is required to be determined, the first query result is added to the current session context, and the preset artificial intelligence model is re-requested to process the user's demand information.

[0171] Among them, the conversation context is the context information directly related to the current dialogue during the conversation. When the preset artificial intelligence model needs to query more information, a new query instruction will be generated, and this step will be repeated multiple times until the information obtained by the preset artificial intelligence model can meet the needs of the user. For example, after the first query result of Task 1 is added to the conversation context, the updated context is input into the preset artificial intelligence model. The preset artificial intelligence model determines that the current query result is not specific enough, and outputs the query instruction again, requiring a more detailed address to be queried. When the information obtained by the preset artificial intelligence model can meet the needs of the current user, the next instruction is generated. The next instruction can be an interactive instruction or an operation instruction.

[0172] S606. The AI ​​assistant executes the operation instructions generated by the preset artificial intelligence model, generates operation signals, and operates the virtual terminal.

[0173] Wherein, when the instruction type is an operation instruction, the AI ​​assistant operates the virtual terminal. The operation mode of the AI ​​assistant on the virtual terminal can be a direct communication mode or a mode of simulating user behavior.

[0174] In one example, after the AI ​​assistant operates the virtual terminal, it obtains the output information of the virtual terminal, adds the output information to the conversation context, and generates new instructions through the preset artificial intelligence model. If the new instruction is still an operation instruction, the AI ​​assistant repeats this step until the preset artificial intelligence model generates an interaction instruction.

[0175] In one possible implementation, the AI ​​assistant operates the virtual terminal in a direct communication manner, and the application on the virtual terminal provides an access API, so the AI ​​assistant can operate the virtual terminal in this way. For example, for the above-mentioned task 2, the preset artificial intelligence model generates an operation instruction. The operation instruction is used to instruct the AI ​​assistant to operate the schedule application in the virtual terminal in a direct communication manner to obtain all the schedule information for the specified date and the specified time period, that is, tomorrow afternoon.

[0176] The operation instruction also includes various parameters required for calling the API of the calendar application.

[0177] In another possible implementation, the AI ​​assistant operates the virtual terminal in a manner that simulates user behavior. Applicable to applications with user interfaces. For example, for the above-mentioned Task 3, the online car-hailing application is operated in a manner that simulates user behavior. This method requires multiple rounds of interaction between the AI ​​assistant, the preset artificial intelligence model, and the virtual terminal. The preset artificial intelligence model generates an instruction each time, and the AI ​​assistant generates one or more operation signals, i.e., simulation signals, based on the instruction. The AI ​​assistant sends these input signals to the virtual terminal and obtains the output information of the virtual terminal. The AI ​​assistant adds the output information to the session context and requests the preset artificial intelligence model to process it again.

[0178] The analog signal may be a keyboard key signal, a mouse click signal, a mouse drag signal, a touch screen click signal, a touch screen slide signal, etc. The output information may be image output information, sound output information, or text output information.

[0179] In another embodiment (Example 2), in some scenarios, the user needs to interact directly with the virtual terminal. The AI ​​assistant can control the AI ​​assistant client to display the user interface of the virtual terminal, and the user can interact with the virtual terminal. Figure 7 As shown, Figure 7 The schematic diagram of the method for user interaction with a virtual terminal provided in the present application may include the following steps S701 to S704.

[0180] S701. The preset artificial intelligence model outputs virtual terminal interaction instructions.

[0181] The preset artificial intelligence model determines whether a user needs to interact directly with the virtual terminal during processing of the virtual terminal output information, and generates a virtual terminal interaction instruction if necessary.

[0182] In one example, the AI ​​assistant operates a virtual terminal to start an application, which requires the user to enter an account and password to log in. The AI ​​assistant can display the login interface to the user through the AI ​​assistant client. In this case, the user needs to interact with the application.

[0183] In another example, when the AI ​​assistant operates the audio and video application on the virtual terminal to play audio and video content for the user, the user needs to see and / or hear the audio and video, and the user needs to interact directly with the audio and video application.

[0184] S702. The AI ​​assistant receives the virtual terminal interaction instruction and displays the page information of the virtual terminal to the user through the AI ​​assistant client.

[0185] In one example, when the interaction instruction is used to instruct the user to input information, the virtual terminal is controlled to display the virtual terminal interface through the AI ​​client, and the virtual terminal interface is the interface displayed by the virtual terminal. The AI ​​assistant client responds to the interaction instruction, the AI ​​assistant instructs the AI ​​assistant client to interact with the virtual terminal, and the AI ​​assistant client establishes a connection with the virtual terminal. During the interaction process, the AI ​​assistant client displays the virtual terminal interface of the virtual terminal to the user, and sends the user input and the sensor input directly to the virtual terminal. During the interaction process, if a window switch occurs, the virtual terminal sends a virtual terminal window switch event notification to the AI ​​assistant.

[0186] In one application scenario, the user command execution process includes two processes: an AI assistant operates a virtual terminal and a user interacts with the virtual terminal. For details, please refer to the description of the following embodiment 3, which will not be repeated here.

[0187] S703. The AI ​​assistant obtains a notification of a virtual terminal window switching event.

[0188] Before the AI ​​assistant instructs the AI ​​assistant client to interact with the virtual terminal, it will also request the virtual terminal to monitor the window switching event. The window switching event means that when the operating system of the virtual terminal detects that the current graphical interface system has a window switch, an event notification will be generated and sent to the AI ​​assistant.

[0189] In one example, the AI ​​assistant receives an interface switching event notification from a virtual terminal. The AI ​​assistant inputs the switched virtual terminal interface information into the AI ​​model to obtain a third control instruction matching the switched virtual terminal interface information.

[0190] The interface switching event notification is used to indicate that a window switching has occurred in the virtual terminal interface. The third control instruction is used to indicate that a termination instruction is sent to the virtual terminal when no interaction with the user is required, and the termination instruction is used to indicate to stop displaying the virtual terminal interface.

[0191] In one application scenario, after receiving the window switching event notification, the AI ​​assistant obtains the output information of the virtual terminal and sends it to the preset artificial intelligence model for processing. If the preset artificial intelligence model determines that it is necessary to end the direct interaction between the user and the virtual terminal, it generates an instruction to end the virtual terminal interaction. For example, in the above application login example, when the user completes the account and password input on the application login interface, logs in successfully, and switches to the interface after login, it will be necessary to end the direct interaction between the user and the virtual terminal.

[0192] Here, "window" refers to the basic organizational unit of a graphical operating system when performing graphic display, and has different names for different operating systems. It is called "window" in Windows system and "activity" in Android system. Different names do not affect the implementation effect of the present invention.

[0193] S704. The AI ​​assistant receives an instruction to end the virtual terminal interaction, and controls the AI ​​assistant client to end the connection with the virtual terminal.

[0194] In one example, the AI ​​assistant receives a termination instruction and sends it to the AI ​​assistant client. The AI ​​assistant client receives the termination instruction and stops displaying the virtual terminal interface in response to the termination instruction.

[0195] In one application scenario, the AI ​​assistant sends a termination instruction to the AI ​​assistant client, notifying the AI ​​assistant client to terminate direct interaction with the virtual terminal. The AI ​​assistant client disconnects from the virtual terminal and switches the user interface to the client's own interface.

[0196] According to the above technical means, the AI ​​assistant can control the AI ​​assistant client to display the user interface of the virtual terminal, and the user can interact with the virtual terminal. During the interaction process, the user's private information can be directly input into the virtual terminal, reducing the intermediate processing process and improving the security of the data.

[0197] In another embodiment (Example 3), in the process of executing user needs, "AI assistant operates virtual terminal" and "user interacts with virtual terminal" are two main processes. In the process of realizing a user's needs, these two processes can be included at the same time. Both processes can be one or more. Figure 8 As shown, Figure 8Another information processing method based on artificial intelligence provided by the present application includes S801 to S807.

[0198] S801. The AI ​​assistant client obtains user demand information.

[0199] In one example, the AI ​​assistant client obtains user demand information through the human-computer interface and sensors, and sends it to the AI ​​assistant.

[0200] S802. The AI ​​assistant obtains user demand information.

[0201] In one example, the AI ​​assistant can input the acquired user demand information into a preset artificial intelligence model to generate instructions that match the user's demand information.

[0202] S803. The preset artificial intelligence model generates instructions according to the user's demand information.

[0203] In one example, when the instructions output by the preset artificial intelligence model are interactive instructions, the AI ​​assistant executes the interactive instructions through the AI ​​assistant client and ends the instruction execution process.

[0204] In another example, when the instructions output by the preset artificial intelligence model are operation instructions, the AI ​​assistant is required to perform operations on the virtual terminal that match the operation instructions.

[0205] S804: The AI ​​assistant performs operations on the virtual terminal that match the operation instructions.

[0206] In one example, during the process of the AI ​​assistant executing operation instructions, the user is required to interact with the virtual terminal. The AI ​​assistant realizes the interaction between the user and the virtual terminal through the AI ​​assistant client, and inputs the interaction results into a preset artificial intelligence model.

[0207] S805. The AI ​​assistant obtains the interaction content between the AI ​​assistant client and the virtual terminal, and inputs a preset artificial intelligence model. The preset artificial intelligence model can output new instructions based on the interaction content.

[0208] In one example, after the AI ​​assistant inputs the interaction content between the AI ​​assistant client and the virtual terminal into a preset artificial intelligence model, the instructions output by the preset artificial intelligence model are interaction instructions between the user and the virtual terminal, indicating that the user interacts with the virtual terminal.

[0209] In another example, the instructions output by the preset artificial intelligence model are other interactive instructions, and the AI ​​assistant outputs content matching the interactive instructions through the AI ​​assistant client and ends the information processing flow.

[0210] S806: The user interacts with the virtual terminal.

[0211] In one example, after the AI ​​assistant inputs the screen information of the virtual terminal into the preset artificial intelligence model, the preset artificial intelligence model outputs an interaction instruction, instructing the user to interact directly with the virtual terminal. For example, the user is required to enter private information such as an account password.

[0212] S807. The AI ​​assistant obtains the result of the user's interaction with the virtual terminal and inputs a preset artificial intelligence model. The preset artificial intelligence model can output new instructions based on the interaction result.

[0213] In one example, the instructions output by the preset artificial intelligence model are operation instructions, and the AI ​​assistant performs an operation matching the operation instruction on the virtual terminal, and returns to step S804, and the AI ​​assistant performs an operation matching the operation instruction on the virtual terminal.

[0214] In another example, the instructions output by the preset artificial intelligence model are interactive instructions, and the AI ​​assistant outputs content matching the interactive instructions through the AI ​​assistant client and ends the information processing flow.

[0215] According to the above technical means, the present application can simultaneously include "AI assistant operating virtual terminal" and "user interacting with virtual terminal" as two main processes in one information processing process.

[0216] The embodiment of the present application also provides an information processing system based on artificial intelligence, which includes an AI assistant and an AI assistant client for executing Figure 2-Figure 8 The information processing method based on artificial intelligence is shown.

[0217] It is understandable that the above-mentioned information processing method based on artificial intelligence can be implemented by an information processing device. In order to realize the above-mentioned functions, the information processing device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the modules and algorithm steps of each example described in the embodiments disclosed herein, the embodiments disclosed in this application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments disclosed in this application.

[0218] The embodiments disclosed in the present application can divide the functional modules of the information processing device generated by the above method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiments disclosed in the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0219] Fig. 9 The present invention provides a schematic diagram of an information processing device based on artificial intelligence, which is applied to an AI assistant. Fig. 9 As shown, the information processing device 90 can be used to execute Figure 2-Figure 8 The information processing method based on artificial intelligence is shown. The information processing device 90 includes: a communication unit 901 and a processing unit 902.

[0220] The communication unit 901 is used to obtain the user's demand information through the AI ​​client. The AI ​​client is set in the user terminal. The processing unit 902 is used to process the demand information based on the preset artificial intelligence model to obtain a first control instruction matching the demand information. The operation matching the first control instruction is executed, and the result corresponding to the operation is output through the user terminal.

[0221] In a possible implementation, the first control instruction includes one or more of an operation instruction, an interaction instruction, and a query instruction.

[0222] In a possible implementation, the processing unit 902 is further configured to execute an operation matching the operation instruction on the virtual terminal, wherein the virtual terminal is set on a cloud server.

[0223] In a possible implementation, the communication unit 901 is further configured to feed back the information to be confirmed to the user through the AI ​​client, and obtain the user's feedback result on the information to be confirmed through the AI ​​client.

[0224] In a possible implementation, the processing unit 902 is further configured to control the virtual terminal to display a virtual terminal interface through the AI ​​client when the interaction instruction is used to instruct the virtual terminal to interact with the user, where the virtual terminal interface is an interface displayed by the virtual terminal.

[0225] In a possible implementation, the processing unit 902 is further configured to receive an interface switching event notification from the virtual terminal. The interface switching event notification is used to indicate that a window switch has occurred in the virtual terminal interface. The switched virtual terminal interface information is input into the AI ​​model to obtain a third control instruction that matches the switched virtual terminal interface information. When the third control instruction is used to indicate that no interaction with the user is required, a termination instruction is sent to the AI ​​client, and the termination instruction is used to instruct to stop displaying the virtual terminal interface.

[0226] In a possible implementation, the processing unit 902 is further configured to obtain corresponding query results according to the query instruction.

[0227] In a possible implementation, the processing unit 902 is further configured to perform an artificial simulation operation according to the operation instruction, generate an artificial simulation signal, and execute the operation corresponding to the artificial simulation signal on the virtual terminal.

[0228] In a possible implementation, the processing unit 902 is further configured to obtain a result corresponding to the operation. The result corresponding to the operation is processed by a preset artificial intelligence model to obtain a second control instruction. An operation matching the second control instruction is executed, and the second control instruction includes one or more of an operation instruction, an interaction instruction, and a query instruction.

[0229] In a possible implementation, the processing unit 902 is further configured to send instruction information to a target application program among the multiple applications for executing the operation instruction, and receive an execution result from the target application program.

[0230] Fig.10 A schematic diagram of the structure of another information processing device based on artificial intelligence provided in the embodiment of the present application is applied to an AI client. Fig.10 As shown, the information processing device 100 can be used to execute Figure 2-Figure 8 The information processing method based on artificial intelligence is shown. The information processing device 100 includes: a communication unit 1001 and a processing unit 1002.

[0231] Communication unit 1001 is used to obtain user demand information through a human-machine interface and / or a sensor, and send the demand information to the AI ​​assistant, so that the AI ​​assistant processes the demand information based on a preset artificial intelligence model, obtains a first control instruction that matches the demand information, and executes an operation that matches the first control instruction to obtain a corresponding operation result.

[0232] Processing unit 1002 is used to receive operation results from the AI ​​assistant and output the operation results to the user.

[0233] In a possible implementation, the first control instruction includes one or more of an operation instruction, an interaction instruction, and a query instruction.

[0234] In a possible implementation, the processing unit 1002 is further configured to receive information to be confirmed from the AI ​​assistant and input the information to be confirmed, obtain feedback from the user on the information to be confirmed, and send the feedback to the AI ​​assistant.

[0235] In a possible implementation, the communication unit 1001 is further configured to receive a virtual terminal interface from a virtual terminal and display the virtual terminal interface. The virtual terminal interface is output information generated by the virtual terminal, and the virtual terminal is set in a cloud server.

[0236] In a possible implementation, the processing unit 1002 is further configured to receive a termination instruction from the virtual terminal, and in response to the termination instruction, stop displaying the virtual terminal interface.

[0237] In a possible implementation, the communication unit 1001 is further configured to obtain the user's demand information or feedback results of the information to be confirmed through a human-machine interface and / or a sensor, and output the operation results or the information to be confirmed to the user through the human-machine interface and / or a sensor.

[0238] Through the description of the above implementation methods, technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0239] An embodiment of the present application also provides a computer-readable storage medium having instructions stored thereon. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the artificial intelligence-based information processing method provided in the above-mentioned embodiment of the present application.

[0240] An embodiment of the present application also provides a computer program product containing instructions, which, when executed on an electronic device, enables the electronic device to execute the artificial intelligence-based information processing method provided by the above-mentioned embodiment of the present disclosure.

[0241] Among them, the computer readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a register, a hard disk, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above, or any other form of computer readable storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application-specific integrated circuit (ASIC). In the embodiments of the present application, a computer-readable storage medium may be any tangible medium that contains or stores a program, which may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0242] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An information processing method based on artificial intelligence, characterized in that: Applied to an AI assistant, the method includes: Obtaining user demand information through an AI client; wherein the AI ​​client is arranged in a user terminal; Based on a preset artificial intelligence model, the demand information is processed to obtain a first control instruction matching the demand information; An operation matching the first control instruction is executed to obtain a result corresponding to the operation.

2. The method according to claim 1, characterized in that The first control instruction includes one or more of an operation instruction, an interaction instruction, and a query instruction.

3. The method according to claim 2, characterized in that The first control instruction includes an operation instruction, including: An operation matching the operation instruction is executed on the virtual terminal; wherein the virtual terminal is set on a cloud server.

4. The method according to claim 2, characterized in that: The first control instruction includes an interaction instruction, including: The information to be confirmed is fed back to the user through the AI ​​client, and the feedback result of the user on the information to be confirmed is obtained through the AI ​​client.

5. The method according to claim 2, characterized in that: The first control instruction includes an interaction instruction, including: In the case where the interaction instruction is used to instruct the virtual terminal to interact with the user, the virtual terminal is controlled to display a virtual terminal interface through the AI ​​client, where the virtual terminal interface is an interface displayed by the virtual terminal.

6. The method according to claim 5, characterized in that The method further comprises: Receiving an interface switching event notification from the virtual terminal; the interface switching event notification is used to indicate that a window switching has occurred on the virtual terminal interface; Inputting the switched virtual terminal interface information into the AI ​​model to obtain a third control instruction matching the switched virtual terminal interface information; In a case where the third control instruction is used to indicate that interaction with the user is not required, a termination instruction is sent to the AI ​​client, where the termination instruction is used to instruct to stop displaying the virtual terminal interface.

7. The method according to claim 2, characterized in that The first control instruction includes a query instruction, including: According to the query instruction, the corresponding query result is obtained.

8. The method according to claim 3, characterized in that The performing of the operation matching the operation instruction on the virtual terminal includes: According to the operation instruction, an artificial simulation operation is performed to generate an artificial simulation signal; The operation corresponding to the artificial simulation signal is performed on the virtual terminal.

9. The method according to any one of claims 1 to 8, characterized in that The method further includes: Obtaining the result corresponding to the operation; Processing the result corresponding to the operation by the preset artificial intelligence model to obtain a second control instruction; An operation matching the second control instruction is executed, where the second control instruction includes one or more of an operation instruction, an interaction instruction, and a query instruction.

10. The method according to claim 3, 8 or 9, characterized in that The virtual terminal is configured with a plurality of application programs, and the performing of an operation matching the operation instruction on the virtual terminal includes: Sending instruction information to a target application program among the multiple applications for executing the operation instruction; An execution result from the target application is received.

11. An information processing method based on artificial intelligence, characterized in that: Applied to an AI client, the method includes: Obtaining user demand information and sending the demand information to the AI ​​assistant, so that the AI ​​assistant processes the demand information based on a preset artificial intelligence model, obtains a first control instruction matching the demand information, and executes an operation matching the first control instruction to obtain a corresponding operation result; Receive the operation result from the AI ​​assistant and output the operation result to the user.

12. The method according to claim 11, characterized in that The first control instruction includes one or more of an operation instruction, an interaction instruction, and a query instruction.

13. The method according to claim 12, characterized in that The first control instruction includes an interaction instruction, and the method further includes: Receiving information to be confirmed from the AI ​​assistant and outputting the information to be confirmed; Obtain the user's feedback result on the information to be confirmed, and send the feedback result of the information to be confirmed to the AI ​​assistant.

14. The method according to claim 12, characterized in that The first control instruction includes an interaction instruction, and the interaction instruction is used to instruct the user to input information. The method further includes: A virtual terminal interface is received from a virtual terminal, and the virtual terminal interface is displayed; the virtual terminal interface is output information generated by the virtual terminal, and the virtual terminal is set in a cloud server.

15. The method according to claim 14, characterized in that The method further comprises: A termination instruction is received from the virtual terminal, and in response to the termination instruction, the display of the virtual terminal interface is stopped.

16. The method according to any one of claims 11 to 15, characterized in that The method further comprises: Obtaining the user's demand information or feedback results of the information to be confirmed through a human-machine interface and / or a sensor; The operation result or the information to be confirmed is output to the user through the human-machine interface and / or the sensor.

17. An information processing system, characterized in that: The system includes an AI assistant and an AI client; wherein the AI ​​assistant is used to execute the method described in any one of claims 1-10, and the AI ​​client is used to execute the method described in any one of claims 11-16.

18. An information processing device based on artificial intelligence, characterized in that: Applied to an AI assistant, the device comprises: a communication unit and a processing unit; The communication unit is used to obtain the user's demand information through the AI ​​client; wherein the AI ​​client is set in the user terminal; The processing unit is used to process the demand information based on the preset artificial intelligence model to obtain a first control instruction matching the demand information; The processing unit is further configured to execute an operation matching the first control instruction.

19. An information processing device based on artificial intelligence, characterized in that: Applied to an AI client, the device comprises: a communication unit and a processing unit: The communication unit is used to obtain the user's demand information through the human-machine interface and / or the sensor, and send the demand information to the AI ​​assistant, so that the AI ​​assistant processes the demand information based on a preset artificial intelligence model, obtains a first control instruction matching the demand information, and executes an operation matching the first control instruction to obtain a corresponding operation result; The processing unit is used to receive the operation result from the AI ​​assistant and output the operation result to the user.

20. An electronic device, characterized in that: include: A memory and a processor; the memory and the processor are coupled; The memory is used to store instructions executable by the processor; When the processor executes the instructions, it performs the artificial intelligence-based information processing method as described in any one of claims 1-10 and / or any one of claims 11-16.

21. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer executes the artificial intelligence-based information processing method as described in any one of claims 1 to 10 and / or any one of claims 11 to 16.

22. A computer program product, characterized in that The computer program product comprises computer program instructions, which, when executed by a processor, implement the information processing method based on artificial intelligence as described in any one of claims 1 to 10 and / or any one of claims 11 to 16.

Images