Model interaction method and apparatus, and electronic device and interaction system

Abstract

Disclosed are a model interaction method and apparatus, and an electronic device and an interaction system, relating to the technical field of artificial intelligence. An AI model in an electronic device anticipates user requirements in advance, and proactively outputs answers to questions, thereby reducing the number of interactions between a user and the AI model, and improving the model interaction efficiency. Specifically, the electronic device proactively displays, on the basis of displayed first content and related information, an expected solution to the first content, and thus the electronic device does not need to wait for the user to describe their questions in detail, thereby simplifying an interaction process between the electronic device and the user and thus reducing the time consumed for the electronic device to wait for user interaction, such that the electronic device can predict information required by the user more quickly and intuitively, thereby helping to improve the convenience and overall satisfaction of the user when using a large model.

Description

Model interaction method, device, electronic device and interactive system

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on March 20, 2024, with application number 202410328701.1 and application name “A model interaction method, device, electronic device and interactive system”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of artificial intelligence technology, and in particular to a model interaction method, device, electronic device, and interaction system. Background Art

[0003] With the development of the Internet and big data technology, artificial intelligence (AI) models have made progress in information processing and knowledge extraction. The AI ​​model may include but is not limited to a large language model (LLM) or a model that can be used to implement interactive functions. In the process of users using AI models to solve their own needs, users continuously ask a series of questions to the AI ​​model through voice or uploading text, and the AI ​​model continuously outputs the answers to the series of questions. However, in order for the AI ​​model to meet the needs of users, users need to use one or more methods (such as voice interaction, pasting text or uploading files, etc.) to input questions to the AI ​​model multiple times, and wait for the AI ​​model to process these questions before the AI ​​model outputs the processing results. The user's multiple interactions with the AI ​​model take a long time. After each interaction, the device also waits for a long time for the AI ​​model to process, and the efficiency of the model interaction is low. Summary of the Invention

[0004] The present application provides a model interaction method, device, electronic device and interaction system, which solves the problem that users need to interact with large models multiple times and takes a long time, and is conducive to improving the efficiency of model interaction.

[0005] This application adopts the following technical solution.

[0006] In the first aspect, the present application provides a model interaction method. The model interaction method is applied to an electronic device or an interactive system including an electronic device, and the model interaction method includes: the electronic device outputs a first interface of an AI model; in response to a first operation on the first interface, the electronic device displays a first content on the first interface, and the first content is obtained by the AI ​​model based on the first operation. In addition, the electronic device collects first information of the first interface and second information associated with the first interface, the first information includes the first content, and the second information includes at least one of the context information of the first interface and the user's identification. Finally, the electronic device outputs a second interface, the second interface includes the first content and a preview window, the preview window includes: the second content obtained by the AI ​​model based on the first information and the second information, and the second content indicates the expected solution of the first content.

[0007] In the present application, the electronic device can actively collect the information of the first interface that has been output and other information associated with the first interface without the need for user voice input or pasting text, etc., and the number of interactions between the electronic device and the user is reduced. Before the user makes a request, the electronic device can generate an expected solution for the first content based on the first content that has been displayed, based on the actively collected information of the first interface and other information associated with the first interface, and output the expected solution for the first content using a preview window. Therefore, compared with the conventional technology, the ability of the AI ​​model has changed from passive feedback after receiving the user's input to actively outputting the expected solution to the first content based on the output first content and related information in the present application. The electronic device no longer needs to wait for the user to describe the user's problem in detail, which simplifies the interaction process between the electronic device and the user, thereby reducing the time the electronic device waits for user interaction, and enables the electronic device to predict the information required by the user more quickly and intuitively, which is conducive to improving the convenience and overall satisfaction of users using the AI ​​model.

[0008] In combination with the model interaction method provided in the first aspect, in an optional implementation, the above-mentioned second interface also includes an operation window, which is used to display the first content, and the preview window is on the upper layer of the operation window. In the present application, the preview window is on the upper layer of the operation window containing the first content obtained by the AI ​​model, so that the second content displayed in the preview window (that is, the expected solution to the first content predicted by the electronic device) does not affect the display of the first content. Among all the content displayed by the electronic device, the user can easily distinguish between the first content and the second content, and the user can obtain the required information more quickly and intuitively, thereby greatly improving the convenience of use and overall satisfaction of the user.

[0009] In combination with the model interaction method provided in the first aspect, in an optional implementation, the above-mentioned preview window includes a first control. After outputting the second interface, the model interaction method provided in this application also includes: in response to a second operation on the first control in the preview window, the electronic device outputs a third interface; wherein the aforementioned third interface includes first content and a second control, and when the second control is triggered, the electronic device outputs a preview window.

[0010] In combination with the model interaction method provided in the first aspect, in an optional implementation method, the electronic device outputs a second interface, including: the electronic device outputs a third interface, the third interface includes the first content and the second control; and if the aforementioned second control is triggered, the electronic device outputs a preview window.

[0011] In this application, if the user does not want to see the second content on the electronic device, the user can operate the first control in the preview window to fold the preview window in the second interface. This can reduce the occlusion of the preview window on the first content when the user needs to operate the first content, thereby improving the effect of the user's interaction with the AI ​​model. When the user needs to view the expected solution to the first content, the second control in the third interface can be operated, and the electronic device will then display the second interface including the first content and the second content. Thus, the user can choose whether to display the second content output by the AI ​​model based on the interaction with the AI ​​model, avoiding the time of waiting for the electronic device to generate the second content based on the AI ​​model, saving the user's waiting time when seeking a solution, and improving the accuracy of problem matching, thereby ensuring the professional consistency and applicability of the solution.

[0012] In combination with the model interaction method provided in the first aspect, in an optional implementation, before the electronic device outputs the second interface, the model interaction method provided in this application also includes: the electronic device generates a target question template based on the first information and the second information; and the electronic device uses an AI model to process the target question template to obtain the second content corresponding to the first content. Since the user may still be browsing the first content displayed on the electronic device while the electronic device is obtaining the second content, the user and the electronic device do not need to have other interactions, and the electronic device can prepare the answer (second content) before the user asks the question, which can significantly reduce the user's waiting time and improve the efficiency of model interaction.

[0013] In combination with the model interaction method provided in the first aspect, in an optional implementation, the electronic device generates a target question template based on the first information and the second information, including: the electronic device uses a deep neural network to process the first information and the second information to obtain a target vector; and the electronic device determines the target question template associated with the target vector from the question library of the AI ​​model. The question library of the AI ​​model includes a plurality of set question templates and answers corresponding to the plurality of question templates. In the present application, the electronic device may use a deep neural network to vectorize different information, and match the obtained vector with the question library of the AI ​​model to obtain the question template corresponding to the vector, so that the AI ​​model processes and analyzes the question template to obtain the expected solution corresponding to the aforementioned information, such as the expected solution refers to a preferred solution or a better solution for the aforementioned information.

[0014] In combination with the model interaction method provided in the first aspect, in an optional implementation, the electronic device uses a deep neural network to process the first information and the second information to obtain a target vector, including: the electronic device deduplicates the first information and the second information to obtain deduplicated data; the electronic device uses a deep neural network to process the deduplicated data to obtain a target vector. For example, in the process of the electronic device obtaining a vector, deduplicating the data stored in the information cache area of ​​the electronic device can reduce redundant repeated data in the target vector. In the process of the electronic device matching the problem template according to the target vector, it is beneficial to improve the accuracy of the target problem template matched by the electronic device, thereby speeding up the speed at which the large model AI model outputs the second content according to the matched problem template, reducing the processing time of the large model AI model, and further improving the efficiency of model interaction.

[0015] In combination with the model interaction method provided in the first aspect, in an optional implementation, the electronic device uses an AI model to process the target question template and predicts the second content corresponding to the first content, including: the electronic device inputs the target question template into the AI ​​model and predicts the second content corresponding to the first content, where the second content is the answer corresponding to the target question template among the answers corresponding to multiple question templates. In this application, the electronic device can use a deep neural network to vectorize different information, and match the obtained vector with the question library of the AI ​​model to obtain the question template corresponding to the vector, so that the AI ​​model processes and analyzes the question template to obtain the expected solution corresponding to the aforementioned information, such as the expected solution refers to a preferred solution or a better solution for the aforementioned information.

[0016] In combination with the model interaction method provided in the first aspect, in an optional implementation, the first content includes: an alarm list of the data management and operation engine, the alarm list including one or more alarms. The second content includes: detailed information of the first alarm, the first alarm being one of one or more alarms, and the detailed information includes: one or more of the device information associated with the first alarm, the error number, the alarm name, the occurrence time, and the repair suggestions. The context information of the aforementioned first interface includes: a full list, the identifiers of multiple alarms, multiple source devices, multiple device information, multiple error numbers, and one or more of multiple repair suggestions.

[0017] In conjunction with the model interaction method provided in the first aspect, in an optional implementation, the first content includes: one or more of: name, clinic number, main complaint, medical history, and physical examination information. The second content includes: treatment suggestions for the combination of the main complaint and name.

[0018] In conjunction with the model interaction method provided in the first aspect, in an optional implementation, the electronic device outputs the first interface of the AI ​​model, including: the electronic device displays the first interface of the AI ​​model. The electronic device outputs the second interface, including: the electronic device displays the second interface.

[0019] In combination with the model interaction method provided in the first aspect, in an optional implementation method, a content acquisition tool is provided in the electronic device. The aforementioned electronic device collects the first information of the first interface and the second information associated with the first interface, including: the content acquisition tool collects the first information of the first interface, and obtains the context information of the first interface and the user's identification based on the first information. In the present application, the background of the electronic device collects the first information and the second information, and actively analyzes and predicts so that the user can obtain the expected solution without any input. Thus, the electronic device makes intelligent predictions based on the user's historical operations and current interface activities, which greatly improves the response speed of the solution and the user experience, and significantly improves the user's work efficiency.

[0020] In conjunction with the model interaction method provided in the first aspect, in an optional implementation, the AI ​​model provided in this application is deployed on a first node of an electronic device, the first node communicates with a second node, and the second node is used to collect information displayed on the electronic device. The electronic device collects first information on a first interface and second information associated with the first interface, including: the first node of the electronic device receives the first information and the second information collected by the second node.

[0021] Exemplarily, the second node refers to a hardware device in the electronic device that is specifically used to collect information in the interface.

[0022] As another example, the second node refers to a software program installed in the electronic device, such as the software program can be installed on the electronic device in the form of a plug-in.

[0023] Also illustratively, the second node refers to another device or software program that communicates with the electronic device, and the other device or software program can be used to collect information displayed in the electronic device.

[0024] In combination with the model interaction method provided in the first aspect, in an optional implementation, before the electronic device collects the first information of the first interface and the second information associated with the first interface, the model interaction method provided in this application also includes: the electronic device obtains the user's authorization information, and the authorization information indicates that the electronic device supports collecting the first information of the first interface and the second information associated with the first interface. In this application, with the user's authorization, the electronic device can collect the information of the first interface and its associated information, so that when the user has no input, the AI ​​model can recommend the second content based on the information of the first interface and its associated information, so that the user can obtain the expected content more conveniently and quickly, reduce the waiting time for the AI ​​model to process, and help improve the efficiency of model interaction.

[0025] In combination with the model interaction method provided in the first aspect, in an optional implementation method, the model interaction method provided in this application also includes: the electronic device updates the question library of the AI ​​model based on the first content, the second content and the first operation; the question library of the AI ​​model includes a plurality of set question templates and answers corresponding to the plurality of question templates. In this application, the electronic device continuously optimizes and iterates the question library of the AI ​​model by collecting and analyzing user data (first content, second content and operation) in real time, thereby improving the AI ​​model's ability to predict user needs. Moreover, this continuous data-driven progress not only improves the overall performance and response speed of the AI ​​model, but also ensures the consistency and professional level of the content output by the AI ​​model. Over time, this method enables the AI ​​model to more accurately understand and adapt to the specific needs of users, thereby providing more accurate and personalized solutions.

[0026] In a second aspect, the present application provides a model interaction device, which is applied to an electronic device or an interactive system including an electronic device, and includes a module or software unit for executing the first aspect or any optional implementation of the first aspect.

[0027] In a third aspect, the present application provides an electronic device. The electronic device includes: one or more memories and a processor; the one or more memories are coupled to the processor. The memory stores computer program code, which includes computer instructions. When the computer instructions are executed by the processor, the electronic device performs the method of the first aspect or any optional implementation of the first aspect.

[0028] In a fourth aspect, the present application provides an interactive system. The interactive system comprises: a display unit and the electronic device provided in the third aspect; the display unit is configured to display a first interface of the large model; and the electronic device, in response to a first operation on the first interface, executes the method of the first aspect or any optional implementation of the first aspect.

[0029] In a fifth aspect, the present application provides a computer-readable storage medium comprising computer instructions, which, when executed on an electronic device, causes the electronic device to execute the method of the first aspect or any optional implementation of the first aspect.

[0030] In a sixth aspect, the present application provides a computer program product. The computer program product includes a computer program or instructions, and when the computer program or instructions are executed on an electronic device, the electronic device executes the method of the first aspect or any optional implementation of the first aspect.

[0031] Regarding the beneficial effects of the technical solutions provided in aspects 2 to 6, reference may be made to the description of aspect 1 or any optional implementation of aspect 1, and no further description is given here. Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] FIG1 is a schematic diagram of the structure of the AI ​​system provided by this application;

[0033] FIG2 is a schematic diagram of the structure of an electronic device provided by this application;

[0034] FIG3 is a flow chart of a model interaction method provided by the present application;

[0035] FIG4 is a schematic diagram of the first interface and the first content provided by this application;

[0036] FIG5 is a schematic diagram of a model interaction provided by this application;

[0037] FIG6A is a second flow chart of a model interaction method provided by the present application;

[0038] FIG6B is a third flow chart of a model interaction method provided by the present application;

[0039] FIG7 is a schematic diagram of determining a question template provided by this application;

[0040] FIG8 is a fourth flow chart of a model interaction method provided by the present application;

[0041] FIG9 is a flowchart diagram 5 of a model interaction method provided by this application;

[0042] FIG10 is a sixth flow chart of a model interaction method provided by the present application;

[0043] FIG11 is a schematic diagram comparing different model interaction methods provided in this application;

[0044] FIG12 is a schematic structural diagram of a model interaction device provided in this application. DETAILED DESCRIPTION

[0045] The present application provides a model interaction method, in which the AI ​​model in the electronic device predicts user needs in advance and actively outputs answers to questions, thereby reducing the number of interactions between the user and the AI ​​model and improving the efficiency of model interaction. Specifically, compared with the conventional technology in which the AI ​​model receives multiple user inputs and provides multiple passive feedbacks, in the present application, the electronic device actively outputs and displays the expected solution to the first content based on the displayed first content and related information. The electronic device no longer needs to wait for the user to describe the problem in detail multiple times, which simplifies the interaction process between the electronic device and the user, thereby reducing the time the electronic device waits for user interaction, enabling the electronic device to predict the information required by the user more quickly and intuitively, which is conducive to improving the convenience and overall satisfaction of users in using large models.

[0046] This application can be applied not only to existing artificial intelligence (AI) technology and model interaction scenarios, but also to future AI technology and model interaction scenarios. The terms used in the implementation methods of this application are only used to explain the specific embodiments of this application and are not intended to limit this application. The following is a brief introduction to some concepts that may be involved in this application.

[0047] Large models: A machine learning model with a large number of parameters and computing resources. The main feature of large models is the large number of parameters, which can improve the model's generalization ability and performance through large amounts of data. Large models are more complex, with deeper and more complex network structures, and can capture richer features and relationships, thereby improving the expressive power of large models. Large models can be divided into sparse large models and dense large models. Among them, sparse large models have a large number of sparse parameters and are commonly used for tasks such as search, recommendation, and advertising. Sparse large models are characterized by massive samples and large-scale sparse parameters, and are suitable for training using CPU / graphics processing unit (GPU) parameter server mode. Dense large models have most parameters with non-zero values ​​and no obvious sparsity characteristics. They are commonly used for computer vision (CV) and natural language processing (NLP) tasks. Dense large models are characterized by regular sample data and large-scale dense parameters, and are suitable for training using pure GPU collective communication mode.

[0048] Large Language Models (LLMs): Leveraging their powerful computing power and sophisticated algorithms, they can effectively process these massive amounts of data, providing users with efficient and accurate information processing and analysis services. Large language models not only excel in understanding and generating human language, but also demonstrate tremendous potential in solving complex problems and tasks. For example, large language models are widely used in automated question-answering systems, text summarization, machine translation, and language generation, significantly improving efficiency and accuracy. Especially when processing large datasets, large language models can uncover deep patterns and connections to support decision-making. Furthermore, the self-learning capabilities of large language models enable them to continuously evolve, continuously improving their performance and intelligence through continuous learning from new data.

[0049] Commonly used large language models include the Bidirectional Encoder Representation from Transformer (BERT) model, which can pre-train deep bidirectional representations using unlabeled text by jointly adjusting the left and right contexts in all layers.

[0050] It is worth noting that in some optional situations, the large language model can also be referred to as a large model. The large model provided in the embodiment of the present application can refer not only to a large language model, but also to a model whose model parameters reach a certain level. For example, according to the changes in the field in which the model is applied, the large model can also refer to a model with various functions such as image processing functions, human-computer interaction functions, semantic search and dialogue functions. This application does not limit the fields in which the large model can be applied and the specific name. In this article, for the sake of simplicity of description, it is named as a large model, but this should not be understood as a limitation of this application and will not be repeated later.

[0051] A deep neural network (DNN), also known as a multi-layer neural network, can be understood as a neural network with many hidden layers. The "many" here does not have a specific metric. Based on the position of different layers in a DNN, the neural network inside the DNN can be divided into three categories: input layer, hidden layer, and output layer. Generally speaking, the first layer is the input layer, the last layer is the output layer, and the layers in between are all hidden layers. The layers are fully connected, that is, any neuron in the i-th layer must be connected to any neuron in the i+1-th layer. Although DNN looks complicated, the work of each layer is actually not complicated. Simply put, it is the following linear relationship expression: in, is the input vector, is the output vector, is the offset vector, W is the weight matrix (also called coefficient), and α() is the activation function. Each layer is just an input vector After such a simple operation, the output vector Since there are many DNN layers, the coefficient W and the offset vector The definition of these parameters in DNN is as follows: Take the coefficient W as an example: Assume that in a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as The superscript 3 represents the layer number of the coefficient W, while the subscript corresponds to the output of the third layer index 2 and the input of the second layer index 4. In summary, the coefficient from the kth neuron in the L-1th layer to the jth neuron in the Lth layer is defined as It's important to note that the input layer has no W parameter. In deep neural networks, more hidden layers allow the network to better capture complex real-world situations. Theoretically, a model with more parameters has higher complexity and greater "capacity," meaning it can handle more complex learning tasks. Training a deep neural network is essentially the process of learning the weight matrix, with the ultimate goal of obtaining the weight matrices for all layers of a trained deep neural network (a weight matrix formed by the vectors W across many layers).

[0052] The following is an illustrative description of the scenarios in which the embodiments of the present application can be applied, with reference to the accompanying drawings.

[0053] As shown in Figure 1, Figure 1 is a schematic diagram of the structure of the AI ​​system provided in this application. As shown in Figure 1, the AI ​​system includes a data center and multiple electronic devices (electronic devices 111 to 113 as shown in Figure 1). The data center can communicate with the electronic devices through a network, which can be the Internet or other networks. The network can include one or more network devices, such as a router or a switch.

[0054] The data center includes: an electronic device with data processing capabilities or a virtual device with data processing capabilities. For example, when the data center includes an electronic device with data processing capabilities, the data center includes one or more servers, such as the server 120 shown in Figure 1, such as an application server that supports application services, which can provide video services, image services, other AI processing services based on video or images, etc. In an optional scenario, the server 120 refers to a server cluster in which multiple servers are deployed. The server cluster can have a rack, and the rack can establish communication for the multiple servers through a wired connection, such as a universal serial bus (USB), a peripheral component interconnect express (PCIe) high-speed bus, or a unified multimedia interconnect line. For another example, when the data center includes a virtual device with data processing capabilities, the virtual device can be a virtual machine, a microservice or a cloud service, etc., which can be provided to users through a cloud service subscription model, and users can choose different subscription levels according to their needs.

[0055] Server 120 can also obtain data from electronic devices, perform AI processing on the data, and send the results of the AI ​​processing to the corresponding electronic devices. This AI processing can include using AI models to perform tasks such as object recognition, target detection, image classification, automated question answering, text summarization, machine translation, and language generation on the data. It can also include obtaining an AI model that meets the requirements based on samples collected by the electronic devices.

[0056] In addition, the data center shown in Figure 1 may also include other physical devices with AI processing capabilities, such as mobile phones, tablets, or other devices.

[0057] An electronic device may also be referred to as a terminal, terminal device, user equipment (UE), mobile station (MS), mobile electronic device (MT), etc. For example, an electronic device may include but is not limited to: a server, a personal computer, a tablet computer or other electronic device. For another example, when the electronic device is a wireless terminal, the electronic device may be a mobile phone (electronic device 111 shown in FIG1 ), a face payment device with a mobile payment function (electronic device 112 shown in FIG1 ), a camera device with a data (such as an image or video) acquisition and processing function (electronic device 113 shown in FIG1 ), etc. The electronic device may also be a tablet computer (Pad), a computer with a wireless transceiver function, a virtual reality (VR) electronic device, an augmented reality (AR) electronic device, a wireless electronic device in a smart city, an electronic device in a smart home, etc. The embodiments of the present application do not limit the specific technology and specific device form adopted by the electronic device.

[0058] It is worth noting that the electronic device can obtain the AI ​​model stored in the server 120, and then use the AI ​​model to generate various content that meets user needs.

[0059] For example, if the user inputs "show payment code" through voice, the AI ​​model responds to the voice information and controls the electronic device to display the payment code interface.

[0060] For example, if a user inputs "perform image classification" via voice, the AI ​​model responds to the voice message and controls the electronic device to classify the image based on the objects in the image and display different labels for the image, such as "Class 3" and "Class 2" in Figure 1.

[0061] Figure 1 is only a schematic diagram and should not be construed as limiting the present application. The embodiments of the present application do not limit the application scenarios of electronic devices and servers.

[0062] The AI ​​model provided in the embodiments of the present application may refer to a large language model (large model or LLM), or may refer to other neural network models or algorithm models that provide interactive functions, without limitation.

[0063] With respect to the hardware implementation of the electronic device shown in FIG1 , FIG2 below provides a possible implementation method, as shown in FIG2 , which is a schematic diagram of the structure of the electronic device provided in this application. The electronic device includes: a processor 210, an external memory interface 220, and an internal memory 221. Optionally, the electronic device may further include a universal serial bus (USB) interface 230, a unified multimedia interconnection interface 231, an audio unit 270, a speaker 270A, a receiver 270B, a microphone 270C, a sensor module 280, a button 290, an indicator 292, a camera 293, and a display screen 294, etc.

[0064] The processor 210 may include one or more processing units. For example, the processor 210 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0065] The controller can be the nerve center and command center of the electronic device. The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0066] Processor 210 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 210 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 210. If processor 210 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 210 latency, and thus improves system efficiency.

[0067] In some embodiments, the processor 210 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a USB interface, a unified multimedia interconnect interface, etc.

[0068] It is understood that the interface connection relationship between the modules illustrated in this embodiment is only for illustrative purposes and does not constitute a structural limitation on the electronic device. In other embodiments, the electronic device may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0069] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, a modem processor, and a baseband processor. In some embodiments, antenna 1 and mobile communication module 250 of the electronic device are coupled, and antenna 2 and wireless communication module 260 are coupled, so that the electronic device can communicate with the network and other devices through wireless communication technology.

[0070] The wired communication function of the electronic device can be implemented through the USB interface 230 or the unified multimedia interconnection interface 231. For example, the electronic device receives or sends video streams, audio streams and video metadata messages through the bus connected to the unified multimedia interconnection interface 231.

[0071] The electronic device implements display functionality through a GPU, display screen 264, and an application processor. A GPU is a microprocessor for image processing that connects display screen 264 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 210 may include one or more GPUs that execute program instructions to generate or modify display information.

[0072] The display screen 264 is used to display images, videos, etc. For example, the display screen 264 includes a display panel.

[0073] The electronic device can implement a camera function using an ISP, camera 263, a video codec, a GPU, a display 264, and an application processor. The ISP is responsible for processing data fed back by camera 263. Camera 263 is responsible for capturing still images or videos. In some embodiments, the electronic device may include one or N cameras 263, where N is a positive integer greater than one.

[0074] In this embodiment, the above display screen 264, video codec, GPU, display screen 264 and application processor can also be collectively referred to as a display unit of the electronic device, which is used to process and display the received video stream.

[0075] The external memory interface 220 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 210 via the external memory interface 220 to implement data storage functions. For example, music, videos, or other files can be stored on the external memory card.

[0076] The internal memory 221 can be used to store computer executable program codes, which include instructions. The processor 210 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 221. For example, in an embodiment of the present application, the processor 210 can execute instructions stored in the internal memory 221, and the internal memory 221 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device (such as audio data, a phone book, etc.), etc. In addition, the internal memory 221 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0077] The electronic device can implement audio functions (audio input and audio output) such as music playback and recording through the audio unit 270 , the speaker 270A, the receiver 270B, the microphone 270C, and the application processor.

[0078] Buttons 260 include a power button, a volume button, and the like. Buttons 260 may be mechanical buttons or touch buttons. Indicator 262 may be an indicator light that can be used to indicate charging status, battery level changes, messages, missed calls, notifications, and the like.

[0079] It should be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than shown, or may combine or separate certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0080] The model interaction method provided in this application is described in detail below in conjunction with the contents shown in Figures 1 and 2.

[0081] Figure 3 is a flow chart of a model interaction method provided in the present application. The model interaction method can be executed by an electronic device or an interactive system including an electronic device. For example, the electronic device can refer to any terminal among the electronic devices 111 to 113 shown in Figure 1, or the electronic device in Figure 2. In some optional examples, the model interaction method can also be executed by other devices.

[0082] This embodiment is described by taking an electronic device including a display unit as an example. The interface output by the electronic device may refer to the display unit in the electronic device displaying the interface. It is worth noting that in some optional situations, if the electronic device does not have a display unit, the interface output by the electronic device may be sent to another device (such as a display device) in the form of data, and the display device displays the interface that can be represented by the data. This application is not limited to this and will not be repeated later.

[0083] Here, an electronic device executing the model interaction method provided by this embodiment is taken as an example for description. As shown in FIG3 , the model interaction method provided by this embodiment includes the following steps S310 to S340 .

[0084] S310: The electronic device displays a first interface of the AI ​​model.

[0085] The AI ​​model provided in this embodiment may refer to a large model (such as a large language model), or may refer to other neural network models or algorithm models that provide interactive functions, which is not limited in this application.

[0086] In one possible example, when the AI ​​model is applied to a data management platform, the first interface may refer to the homepage displayed by the electronic device to the user, or the first interface may refer to the interface displayed by the electronic device to the user after the user logs in. The data management platform refers to a system that collects and analyzes information to understand user needs or solve user problems. The data management platform may be implemented based on a server or a cloud-based virtual device, and this application does not limit this. For example, the data management platform includes a data operation and maintenance engine, a memory, and other possible hardware devices.

[0087] For example, the data management platform may include but is not limited to: a data center, a cloud-based intelligent operation and maintenance platform, a cloud computing platform, a cloud storage platform, or a cloud desktop, etc., which is not limited in this application.

[0088] In another possible example, when the AI ​​model is applied to a hospital's professional medical record management system, the first interface may refer to the case processing interface displayed by the electronic device to the user (such as a doctor). The medical record management system is a medical-specific software. The hospital uses electronic medical records to electronically record patient information, including the homepage, medical records, examination and test results, medical orders, surgical records, nursing records, etc. The medical record management system manages not only structured information but also unstructured free text and image information.

[0089] The above two examples are only two possible implementation methods of the first interface provided in this embodiment. The AI ​​model provided in this embodiment can also be applied in other scenarios, such as personalized teaching scenarios in the education field, corporate management in the sales field, and other scenarios. This application does not specifically limit the fields in which the AI ​​model can be applied and the first interface.

[0090] S320: In response to a first operation on the first interface, the electronic device displays first content on the first interface.

[0091] Optionally, the first operation may be a sliding operation, a clicking operation, a gesture operation or a voice operation, etc., which is not limited in this application. For example, taking the first operation as a language operation as an example, the user inputs the user's needs to the electronic device through voice.

[0092] In some special cases, the first operation may also be that the user pastes the copied text information into the input box of the first interface (not shown in Figure 3) through input and output devices such as buttons or display screens of the electronic device. This application does not limit the specific implementation method of the first operation.

[0093] In this embodiment, the first content is obtained by the AI ​​model according to the first operation.

[0094] In one possible example, when the AI ​​model is applied to a data management platform, if the first operation is a user selecting the "Data Risk Identification" control, the first content displayed on the first interface may include, but is not limited to, a list of alarms from the data management and operation engine and other possible information, such as one or more alarms. If the first operation is a user selecting "Text Translation," the first content may include, for example, a text translated from the text on the first interface into the target language.

[0095] As shown in Figure 4, Figure 4 is a schematic diagram of the first interface and the first content provided by this application: the first content includes a navigation bar and a first content window. For example, the navigation bar includes in sequence: data management platform, home page, equipment (storage, hyper-convergence, computing), analysis (hard disk, data protection, capacity, performance, virtual machine, application), planning (business load, capacity), message (alarm), support (work order, maintenance contract). Among them, the user selected "Message-Warning" through the first operation, and the first content displayed by the electronic device includes: from left to right, there is a ring distribution graph with a total of 82 alarms and a bar graph with a total of 82 alarms, and an alarm list located below [involving (82 alarms)]. Among them, the ring distribution graph and the bar graph divide the alarms into four different alarm types: emergency, important, warning and minor, and the number of each alarm type is: 79, 2, 1, 0.

[0096] In another possible example, when the AI ​​model is applied to a medical record management system, the first content displayed in the first interface may include, but is not limited to, name, clinic number, chief complaint, medical history, physical examination information, or other information. For example, the name may be "Zhang San" or "Li Si", the clinic number may be "001", the chief complaint may be "suspected diabetes", the medical history may be "none", and the physical examination information may include relevant information about the physical examination of the named person stored in the medical record management system.

[0097] The above two possible examples are merely possible implementations of the first content provided in this embodiment and should not be understood as limitations on this application.

[0098] S330: The electronic device collects first information of the first interface and second information associated with the first interface.

[0099] The first information includes the first content, and the second information includes the context information of the first interface and the user's identifier.

[0100] In one possible example, when the AI ​​model is applied to a data management platform, the context information of the first interface includes one or more of the following: a full list, identifications of multiple alarms, the source device of each of the multiple alarms, multiple device information, multiple error numbers, and multiple repair suggestions.

[0101] The full list may include: all data associated with the first information within a certain time period; for example, a full list, also known as a full table, refers to a table that stores all data in the same column, which is characterized by large data volume and high query efficiency. In this embodiment, the full list may include all data associated with the user's identifier in the data management platform, such as user information, user operation records in the AI ​​model (such as operation logs), etc.

[0102] The source device of the alarm may refer to the serial number of a device that is communicating with the electronic device and is at risk, etc. For example, the source devices of the alarm include: device 1 corresponding to alarm 1, device 2 corresponding to alarm 2, etc.

[0103] Multiple device information includes: hardware information of the device that communicates with the electronic device, etc.; for example, the hardware information of the device may include but is not limited to: device name, full name of the device, processing model, machine band, device ID, product ID, system type in the device (such as 64-bit operating system) and other possible information, etc.

[0104] The error number can also be called the error code or sequence number. The repair suggestion indicates the strategy for correcting the device error. For example, if error 1 is multiple duplicates of data 1 stored in the device, the corresponding repair suggestion for error 1 is: delete the redundant data 1 in the device and retain at least one data 1.

[0105] In another possible example, when the AI ​​model is applied to a medical record management system, the contextual information of the first interface includes: historical cases associated with the name, other cases related to the main complaint in the first content, or one or more of other information.

[0106] In this embodiment, two optional implementations are provided below for the process of the electronic device collecting the first information and the second information.

[0107] In a first optional implementation, a content acquisition tool is provided in an electronic device. The process of the electronic device acquiring first information of a first interface and second information associated with the first interface includes: the content acquisition tool acquiring the first information of the first interface, and acquiring context information of the first interface and a user identifier based on the first information.

[0108] Exemplarily, for electronic devices deployed with AI models, an embedded software content acquisition tool is also deployed inside the electronic device. For example, the software content acquisition tool can be tightly integrated with the user interface (such as the first interface) of the electronic device through an information collection interface or an information collection interface to ensure that the data collected by the plug-in is both comprehensive and accurate. The software content acquisition tool regularly collects the first content included in the first interface of the electronic device and other information in the first interface to obtain the first information of the first content, and obtains the context information associated with the first interface maintained by the electronic device based on the information in the first interface (such as the user's identification).

[0109] In a second optional implementation, the AI ​​model is deployed on a first node of an electronic device, the first node communicating with a second node, and the second node is configured to collect information displayed on the electronic device. The process of the electronic device collecting first information on a first interface and second information associated with the first interface includes: the first node receiving the first information and the second information collected by the second node.

[0110] In one possible example, when the first node and the second node are implemented by hardware, the first node may refer to a chip included in the electronic device, and the second node may refer to other chips included in the electronic device that are different from the first node; or, the first node may refer to a chip included in the electronic device, and the second node may refer to other devices or chips that communicate with the electronic device, etc.

[0111] In another possible example, when the first node and the second node are implemented by software, the second node may refer to a plug-in that can intercept data or information displayed in an electronic device. The plug-in may be fixedly installed on the electronic device, such as the plug-in may be tightly integrated with the user interface of the electronic device through an information collection interface or an information acquisition interface to ensure that the data collected by the plug-in is both comprehensive and accurate. Alternatively, the plug-in may be installed in a non-embedded manner on an application in the electronic device, such as the application being a browser on the electronic device, and the plug-in being a page screenshot plug-in installed on the browser.

[0112] The second node is a tool implemented based on optical character recognition (OCR) technology. The OCR tool extracts the first information and the second information from the captured image or existing text.

[0113] In an optional embodiment, in order to protect the user's privacy information and improve the security of data in the electronic device, before the electronic device collects the first information of the first interface and the second information associated with the first interface, the model interaction method provided in the embodiment of the present application may also include: the electronic device obtains the user's authorization information, and the authorization information indicates: the electronic device supports the collection of the first information of the first interface and the second information associated with the first interface.

[0114] Exemplarily, the user's authorization information may be provided by the electronic device in a pop-up window, such as "Do you agree to authorize xxx to collect device information" or "Do you agree to authorize xxx to access the device's images and videos", etc., and after the user selects "Yes" or "Agree", the above-mentioned content acquisition tool or the second node collects the context information and other possible information of the first interface in the electronic device.

[0115] In an embodiment of the present application, with the user's authorization, the electronic device can collect information on the first interface and its associated information, so that without user input, the AI ​​model can recommend second content based on the information on the first interface and its associated information, so that the user can obtain the expected content more conveniently and quickly, reducing the waiting time for the AI ​​model to process, which is conducive to improving the efficiency of model interaction.

[0116] Please continue to refer to FIG. 3 . The model interaction method provided in this embodiment further includes the following S340 .

[0117] S340: The electronic device displays a second interface.

[0118] Among them, the second interface includes first content and a preview window, and the preview window includes: second content obtained by the AI ​​model based on the first information and the second information, and the second content indicates the expected solution to the first content.

[0119] In some optional examples, the expected solution (second content) of the first content may refer to a user's preferred solution. For example, the expected solution may be determined by an AI model based on the user's historical operation records (such as logs).

[0120] In other optional examples, the expected solution to the first content (the second content) may be a reference solution determined by the AI ​​model based on the first information and the second information. For example, the reference solution can be viewed and directly used by the user to solve problems that may exist in the first content.

[0121] In some further optional examples, the expected solution to the first content (second content) may refer to a better solution determined by the AI ​​model based on historical data.

[0122] The above three examples are only possible implementations of the second content provided in this embodiment and should not be understood as limiting the present application. In some optional situations, the second content may also include other content.

[0123] For example, when the AI ​​model is applied to a data management platform, the second content includes detailed information about a first alarm. For example, the first alarm is one of one or more alarms included in the first content. The first alarm may be predicted by the AI ​​model based on the first and second information. The detailed information about the first alarm may include, but is not limited to, one or a combination of the following: device information associated with the first alarm, error number, alarm name, occurrence time, and repair suggestions.

[0124] For another example, when the AI ​​model is applied to a medical record management system, the second content includes: at least one of the treatment opinions for the combination of the chief complaint and the name in the first content.

[0125] Regarding the display method of the preview window and the first content in the second interface, the embodiment of the present application provides two optional implementation methods.

[0126] In a first optional implementation, the second interface further includes an operation window, which is used to display the first content. For example, the operation window refers to the blank filled pattern (rectangle) in Figure 3, and the preview window is on the upper layer of the operation window. In Case 2 of Figure 3, it is assumed that the window used to display the first content in the second interface is the operation window, which is used to display the first content; the preview window (black filled circular pattern) is on the upper layer of the operation window (displaying the first content). In some feasible examples, since the preview window and the first content are in the same interface, the second interface can also be called the user's main application interface or main interface, etc.

[0127] For example, electronic devices can proactively communicate with users through a floating interactive window (preview window), providing relevant information and suggestions based on the user's current work environment and context. For example, if a user is viewing information about a specific patient, the system may proactively provide a generated case or treatment suggestions. Of course, the preview window can also proactively communicate with the user to provide the required information, greatly simplifying the user's operation process and making information acquisition more direct and efficient.

[0128] In a second optional implementation, there is no overlapping area between the preview window and the first content, and the second content is displayed in the preview window. In case 1 of Figure 3, the preview window (a black-filled square pattern) is on the right side of the second interface, and there is no overlapping area between the preview window and the first content, and the second content is displayed in the preview window. In some feasible examples, since there is no overlapping area between the preview window and the first content, the preview window can also be called an AI model assistant interface, an AI model auxiliary interface, or other names.

[0129] In Case 1 of Figure 3 , the length of the preview window (along the top and bottom of the preview window) does not fully occupy the second interface. That is, the top edge of the preview window does not overlap with the top edge of the second interface, and the bottom edge of the preview window does not overlap with the bottom edge of the second interface. However, in some optional situations, the top edge of the preview window may overlap with the top edge of the second interface, or the bottom edge of the preview window may overlap with the bottom edge of the second interface. This application does not impose any restrictions on the length of the preview window in the second interface.

[0130] It is worth noting that, in some possible examples, the preview window may also be located on the upper side of the second interface or on the lower side of the second interface, which is not limited in this application.

[0131] In an embodiment of the present application, the electronic device can actively collect information of the displayed first interface and other information associated with the first interface without the need for user voice input or pasting text, etc., and the number of interactions between the electronic device and the user is reduced. Before the user makes a request, the electronic device can generate an expected solution for the first content based on the first content that has been displayed, based on the actively collected information of the first interface and other information associated with the first interface, and use a preview window to display the expected solution for the first content.

[0132] Compared with the conventional technology in which the AI ​​model's ability to provide passive feedback after receiving user input, the electronic device actively displays the expected solution to the first content based on the displayed first content and related information. The electronic device no longer needs to wait for the user to describe the user's problem in detail, which simplifies the interaction process between the electronic device and the user, thereby reducing the time the electronic device waits for user interaction, enabling the electronic device to predict the information required by the user more quickly and intuitively, which is conducive to improving the convenience and overall satisfaction of users in using the AI ​​model.

[0133] In addition, with respect to the display method of the preview window in the second interface, an embodiment of the present application also provides a possible example, as shown in Figure 5, which is a schematic diagram of a model interaction provided by the present application. The preview window includes a first control 510, and the first control 510 includes a minimize icon (-) and an exit icon (×). After the above-mentioned electronic device displays the second interface, the model interaction method provided by the embodiment of the present application may further include: displaying a third interface in response to a second operation on the first control in the preview window. If the second operation refers to clicking the minimize icon, the preview window is minimized to a bubble-type floating window or other type of widget.

[0134] The third interface includes the first content and a second control 520. The second control 520 includes a maximize icon (□) and an exit icon (×).

[0135] When the maximized icon (□) in the second control 520 is triggered, the electronic device displays a preview window, that is, the electronic device displays the aforementioned second interface. For example, after the user pauses on the interface, the electronic device displays a prompt button (second control 520); when the user clicks this button, a bubble-style dialog box (preview window) pops up, providing pre-generated information or solutions (second content). This interactive method is suitable for users who are not in a hurry to obtain information but need detailed guidance.

[0136] It is worth noting that the preview window shown in FIG5 is merely an example provided in this embodiment and should not be construed as limiting the present application. In some alternative embodiments, the various control components (controls) in the preview window may also be other patterns. Of course, controls with more functions may also be defined in the preview window, which will not be detailed here.

[0137] In an embodiment of the present application, if the user does not want to see the second content on the electronic device, the user can operate the first control in the preview window to put the preview window in the second interface into a folded state (such as the third interface in Figure 5). In this way, when the user needs to operate the first content, the preview window can reduce the obstruction of the first content, thereby improving the effect of user interaction with the AI ​​model.

[0138] When the user needs to view the expected solution to the first content, he can operate the second control in the third interface, and the electronic device will then display a second interface including the first content and the second content (such as the second interface in Figure 5). In this way, the user can choose whether to display the second content output by the AI ​​model based on the interaction with the AI ​​model, avoiding the time of waiting for the electronic device to generate the second content based on the AI ​​model, saving the user's waiting time when seeking a solution, and improving the accuracy of problem matching, ensuring the professional consistency and applicability of the solution.

[0139] In order to reduce the time it takes for the AI ​​model to generate the second content and improve the processing efficiency of the AI ​​model, the present application provides a possible implementation method for the generation process of the second content, as shown in Figure 6A, which is a flow chart of a model interaction method provided by the present application. Before the above-mentioned electronic device displays the second interface, the model interaction method provided in the embodiment of the present application includes the following S610 to S620.

[0140] S610: The electronic device pre-processes the first information and the second information to generate a target question template.

[0141] Exemplarily, the preprocessing process in S610 may include, but is not limited to, format matching, text vectorization, data deduplication, or data screening, etc. Format matching refers to processing information so that the format of the processed information matches a preset format. Text vectorization refers to processing text information to obtain a vector representing the text information. Data deduplication refers to deleting duplicate or redundant data from multiple data. Data screening refers to filtering out data that meets certain conditions from multiple data.

[0142] Regarding the specific implementation of S610, the following provides a feasible example using the preprocessing process including text vectorization and data deduplication as an example, as shown in Figure 6B, which is a flow chart of a model interaction method provided by this application. The above-mentioned S610 may include the following S611 and S612.

[0143] S611. The electronic device processes the first information and the second information using a deep neural network to obtain a target vector.

[0144] Optionally, the process of the electronic device obtaining the target vector may include: the electronic device deduplicating the first information and the second information to obtain deduplicated data; and the electronic device using a deep neural network to process the deduplicated data to obtain the target vector.

[0145] In this embodiment, the deep neural network (DNN) may include but is not limited to: convolutional neural network (CNN), recurrent neural network (RNN), and transformer-based neural network, etc.

[0146] Optionally, the deep neural network is deployed in an electronic device. When the electronic device needs to perform vectorized processing on the first information and the second information, the electronic device calls the deep neural network and generates a target vector.

[0147] As an optional example, the process of an electronic device obtaining a target vector specifically includes: the electronic device stores the first information and the second information in an information cache area, and deduplicates the data stored in the information cache area; and the electronic device uses a deep neural network to process the deduplicated data in the information cache area to obtain the target vector.

[0148] Among them, the information cache area can be a storage address space provided by the memory in the electronic device, and the information cache area is used to temporarily store the input data or output data of the AI ​​model. For example, the input data refers to the first information and the second information collected by the electronic device, and the output data refers to the first content or the second content output by the AI ​​model in the electronic device.

[0149] It is worth noting that in some optional methods, the deep neural network can also be deployed in other devices, and the electronic device is configured with a calling interface connected to the deep neural network. When the electronic device inputs the first information and the second information into the deep neural network through the calling interface, it obtains the target vector output by the deep neural network.

[0150] In this way, in the process of the electronic device obtaining the vector, deduplication of the data in the information cache area can reduce redundant repeated data in the target vector. In the process of the electronic device matching the question template according to the target vector, it is beneficial to improve the accuracy of the question template matched by the electronic device, thereby speeding up the speed at which the AI ​​model outputs the second content according to the matched question template, reducing the processing time of the AI ​​model, and further improving the efficiency of model interaction.

[0151] S612. The electronic device determines a target question template associated with the target vector from the question library of the AI ​​model.

[0152] The question library of the AI ​​model includes multiple set question templates and answers corresponding to the multiple question templates.

[0153] In one feasible embodiment, the question library of the AI ​​model is stored in the memory of the electronic device.

[0154] In another feasible approach, the AI ​​model's question library is stored in a data center that communicates with the electronic device. For example, the electronic device is configured with an API for the question library. When the electronic device executes the model interaction method, the electronic device calls the API to query the question library for the target question template.

[0155] Regarding the process of determining the target question template by an electronic device, a possible example is provided below in conjunction with Figure 7, which is a schematic diagram of determining a question template provided by this application. The process of determining the question template by an electronic device includes a vectorized embedding model generation phase and a question template matching phase. The two phases are exemplarily described below.

[0156] 1. The generation phase of the vectorized embedding model.

[0157] The electronic device uses multiple question templates and cached deduplicated software content information as training data, and performs auxiliary training on the vectorized embedding model, so that the trained vectorized embedding model can support vectorized processing of question templates or software content information to obtain vectors that meet the search requirements of the question library. The multiple question templates may include multiple set question templates and other user-defined templates, and the cached deduplicated software content information may include the deduplicated software content information cached by the electronic device and other user-defined information.

[0158] The electronic device deploys the generated vectorized embedding model to assist the interaction process between the AI ​​model and the user.

[0159] Optionally, the vectorized embedding model provided in this embodiment may refer to a deep neural network.

[0160] 2. Question template matching stage.

[0161] The matching process of the question template includes the following steps ① to ③:

[0162] ①. The electronic device vectorizes multiple question templates through a vectorized embedding model (deep neural network) to obtain question templates in the question library of the AI ​​model.

[0163] ②. The electronic device uses a vectorized embedding model (deep neural network) to vectorize the information collected in the user interface (such as the first information and the second information mentioned above, that is, the software content information after cache deduplication) to obtain a target vector.

[0164] ③. The electronic device performs a nearest neighbor search between the target vector and the question template in the question library of the AI ​​model to obtain the target question template.

[0165] For example, during the nearest neighbor search process, the electronic device sets the top-K results, that is, the electronic device outputs K candidate question templates and determines the target question template from these K candidate question templates. For example, the target question template refers to the question template with the highest similarity between the K candidate question templates and the target vector, or the target question template refers to another question template among the K candidate question templates.

[0166] In this embodiment, the electronic device can accurately match the question template (question template) relevant to the user's needs, thereby quickly and accurately generating relevant answers. This method aims to reduce the user's burden of waiting and input, and through a periodic or user behavior-driven prediction mechanism, it can meet the user's immediate needs, while improving the system's matching accuracy to user questions and accelerating response time.

[0167] 6A , the process of the electronic device obtaining the second content further includes the following S620 .

[0168] S620: The electronic device uses the AI ​​model to process the target question template and predicts the second content corresponding to the first content.

[0169] Exemplarily, the electronic device inputs the target question template into the AI ​​model and predicts the second content corresponding to the first content, where the second content is the answer corresponding to the target question template among the answers corresponding to multiple question templates.

[0170] Electronic devices can use deep neural networks to vectorize different information, and match the obtained vectors with the problem library of the AI ​​model to obtain the problem template corresponding to the vector. The AI ​​model then processes and analyzes the problem template to obtain the expected solution corresponding to the aforementioned information, such as the expected solution refers to a preferred solution or a better solution for the aforementioned information.

[0171] Since the user may still be browsing the first content displayed on the electronic device while the electronic device is obtaining the second content, the user does not need to have any other interactions with the electronic device. The electronic device can prepare the answer (the second content) before the user asks a question. This can significantly reduce the user's waiting time and improve the efficiency of model interaction.

[0172] As an optional implementation method, in order to continuously optimize the model interaction process, the embodiment of the present application also provides a possible example: the electronic device updates the question library of the above-mentioned AI model based on the first content, the second content and the first operation.

[0173] In one possible example, the electronic device updates the question library of the AI ​​model, including: the electronic device determines the first information and the second information corresponding to the combination of the first content and the first operation, adds the question template corresponding to the first information and the second information as a reference question template to the question library of the AI ​​model, and adds the second content and the operation that the user may perform on the second content as the answer corresponding to the reference question template to the question library of the AI ​​model.

[0174] In another possible example, the electronic device updates the question library of the AI ​​model, including: the electronic device determines the first information and the second information corresponding to the combination of the first content and the first operation, uses the question template corresponding to the first information and the second information as reference information to modify the above-mentioned target question template in the question library, and uses the second content and the operation that the user may perform on the second content as the modification information corresponding to the target question template, and assists in correcting the answer to the target question template in the question library of the AI ​​model based on the modification information.

[0175] The above two examples are only optional ways to update the question library of the AI ​​model provided in this embodiment and should not be understood as limiting this application. In the embodiment of this application, the electronic device continuously optimizes and iterates the question library of the AI ​​model by collecting and analyzing user data (first content, second content, and operations) in real time, thereby improving the AI ​​model's ability to predict user needs.

[0176] Moreover, this continuous data-driven advancement not only improves the overall performance and responsiveness of AI models, but also ensures the consistency and professionalism of the content output by AI models. Over time, this approach enables AI models to more accurately understand and adapt to users' specific needs, thereby providing more precise and personalized solutions.

[0177] For the above model interaction method, a complete implementation method is provided below in conjunction with the accompanying drawings. Figure 8 is a flow chart of a model interaction method provided by this application. The model interaction method includes two processes: information collection and content prediction. The following are exemplary descriptions of these two processes.

[0178] 1. Information collection.

[0179] The information collection method may include one or both of the two methods: embedded software content acquisition and OCR acquisition (non-embedded).

[0180] During the information collection process, the electronic device may selectively use an embedded method or a non-embedded method to efficiently and accurately collect information from the user interface, such as the first information and the second information mentioned above.

[0181] 2. Content prediction.

[0182] The electronic device automatically deduplicates the collected information through a specific algorithm and stores the deduplicated data in an information cache area.

[0183] Based on the data stored in the information cache area, the electronic device matches the vectors corresponding to these data with the question templates in the question library to determine the target question template. In some examples, because the question templates in the question library are stored in the form of vectors, the question library is also called a vector database. In this way, the electronic device combines the user interface content analysis with the question templates in the vector database to automatically generate question inputs for the AI ​​model.

[0184] That is to say, in an embodiment of the present application, the electronic device can actively search for the second information associated with the first content based on the first content displayed on the current interface, and after predicting the question the user wants to ask based on the first information containing the first content and the aforementioned second information, the electronic device generates in advance and actively outputs the expected solution to the first content, i.e., the second content, which reduces the number of interactions between the user and the electronic device and improves the efficiency of the model interaction. Before the electronic device predicts the question the user wants to ask, the electronic device also vectorizes the information collected in the user interface through a deep neural network, which can accurately match the question template (target question template) related to the user's needs, and processes the target question template, key question prompt words and existing information based on the AI ​​model to obtain the output result, thereby quickly and accurately generating relevant answers. Among them, before the electronic device obtains the target question template, the electronic device can also perform format matching or text vectorization on the first information and the second information obtained by the aforementioned search, so that the first information and the second information can be converted into a target vector by the deep neural network in the electronic device, and the target vector can achieve accurate matching of the target question template with the question library, thereby avoiding the problem that the input of the AI ​​model cannot be effectively recognized due to insufficient professional ability of the user, thereby improving the accuracy of the model interaction.

[0185] 8 for illustration, the electronic device proactively collects information displayed on the electronic device and information outside the interface, reducing the user's burden on waiting and input, and meets the user's immediate needs through a periodic or user behavior-driven prediction mechanism, while improving the system's matching accuracy for user questions and accelerating response speed.

[0186] Furthermore, electronic devices can collect real-time data on user operations and interactions, allowing the AI ​​model's question library to continuously self-learn and iteratively optimize. This data-driven advancement model not only enhances the system's predictive capabilities and accuracy, but also enables the AI ​​model to better adapt to and meet users' future needs, forming a self-reinforcing cycle of continuous improvement. Over time, this approach enables the AI ​​model to more accurately understand and adapt to users' specific needs, thereby providing more precise and personalized solutions. Different manufacturers can leverage the vast amount of demand data accumulated over time to build strong technical barriers in related fields, making personalized and precise matching more advanced.

[0187] Combined with the content shown in Figure 8, the model interaction process of the data management platform and medical record management system scenarios is explained respectively.

[0188] FIG9 is a flowchart diagram of a model interaction method provided by the present application, which is applied to the anomaly detection scenario of the data management platform: in this scenario, the user may have different information in different levels of operation interfaces. When the user interface is limited, one interface cannot accommodate too much information at the same time. For example, after the user asks a question for the first time, the first interface displayed by the electronic device is an alarm list, which displays all current alarm conditions. When the user selects an alarm in the alarm list for the second time, the electronic device displays detailed information of the selected alarm, including the source device, error number, solution suggestions, etc. Since the user needs to perform at least two operations to display the detailed information of the alarm, this may cause inconvenience in user decision-making and information collection.

[0189] To solve the above problem, the model interaction method provided in FIG9 includes the following S910 to S940.

[0190] S910: Respond to the user's first operation and implement system-level information input.

[0191] In the intelligent O&M anomaly detection scenario of a data management platform, system-level information input is a critical first step. This process is tightly integrated with the user interface through the information collection API, ensuring that the collected data is both comprehensive and accurate. The data management platform proactively collects information, accurately identifying and capturing key data both on and off the interface, such as the aforementioned full list, all alarm IDs, all source devices, all error codes, and other device information. Seamless integration with the user interface ensures that data collection does not interfere with normal user operations.

[0192] Regarding the information collection method of S910, reference can be made to the description of S330 above, which will not be repeated here.

[0193] S920: Information caching and deduplication.

[0194] The purpose of information caching and deduplication is to improve the response speed and accuracy of AI models in electronic devices. This process uses intelligent deduplication algorithms to compare existing data, quickly identify and remove duplicate information, and utilize efficient information caching mechanisms to accelerate data processing and improve the overall response speed of AI models.

[0195] S930: Match the question template vector.

[0196] This step uses the vector database (question library) to intelligently match the target question template. The electronic device vectorizes the collected information through a deep neural network, and then matches the target question template that is most relevant to the user's needs in the vector database (question library). This process includes converting the collected information data into vector form to facilitate efficient pattern recognition and semantic matching, as well as using advanced semantic vectorization models and matching algorithms to accurately locate the most appropriate question template (target question template) for the user's current needs. For example, in the current scenario, the electronic device matches the alarm screening template under the list interface (the first content displayed in the first interface as shown in Figure 3), and the electronic device matches the alarm handling solution template under the details page (the second content displayed in the second interface as shown in Figure 3).

[0197] For the specific implementation of S930, please refer to the description of FIG6 above, which will not be repeated here.

[0198] S940: Generate in advance.

[0199] After the electronic device confirms the target problem template, it generates a corresponding processing solution or alarm processing priority for the target problem template in advance and displays the generated content in the corresponding user interface, such as the second content displayed in the second interface shown in Figure 3. The electronic device automatically generates solutions or suggestions based on the matched problem template, including alarm processing steps, priority ranking, etc., and displays them on the interface in a user-friendly manner, such as a bubble pop-up window or a floating dialogue window, to facilitate users to quickly understand and operate.

[0200] For the interactive diagram of the user interface, please refer to the description of Figures 3 to 5 above, which will not be repeated here.

[0201] Figure 10 is a sixth flow diagram of a model interaction method provided by this application. This model interaction method is applied to a medical record management system. For software that is relatively slow to iterate and cannot quickly adapt to embedded APIs, electronic devices will use OCR or plug-in extraction to collect information. As shown in Figure 10, the model interaction method provided by this embodiment is described in detail below.

[0202] S1010, OCR / plugin extraction.

[0203] The difference between the S1010 and S910 is that the electronic device uses optical character recognition (OCR) technology or a specific plug-in to extract key information from the user interface, such as name, gender, and clinic number. This approach is suitable for software systems that cannot directly access embedded APIs and therefore require external tools for data extraction. For example, OCR and plug-ins, as means of information acquisition, focus on identifying and extracting data from existing text or images.

[0204] S1020: Information caching and deduplication.

[0205] The specific implementation method of S1020 can refer to S920 and will not be repeated here.

[0206] S1030: Problem template vector matching.

[0207] The difference between S1030 and S930 is that in the medical record management system, a case report generation template is matched under the entire interface, and a treatment opinion generation template is matched in the treatment opinion box.

[0208] S1040, generate in advance.

[0209] The specific implementation of S1040 can refer to S940 and will not be described in detail here.

[0210] In addition to the specific scenarios shown in Figures 9 and 10 above, the model interaction method provided in this application can also be applied to the field of intelligent assistants, such as providing more personalized services and suggestions through context perception and user behavior prediction. At the same time, this method combining vector databases and deep neural networks can also be extended to the fields of automated market research and consumer analysis, helping companies to more accurately predict market trends and consumer needs. In addition, the medical industry can also benefit from this technology by analyzing patients' historical data and current symptoms to provide doctors with more accurate diagnostic support. In addition, in the field of education, the model interaction method provided in this application can be used for personalized teaching, automatically adjusting the teaching content and difficulty according to the students' learning progress and abilities. Finally, the principles and methods of the model interaction method provided in this application can be applied to smart city management systems to provide more effective urban planning and management strategies by analyzing urban data. These are all potential technical areas within the scope of protection of this application, demonstrating its diverse application prospects and huge market potential.

[0211] With respect to the embodiments provided in the above Figures 3 to 10, the model interaction method provided in the embodiments of the present application is compared with the model interaction process of the conventional technology in combination with Figure 11 below to further illustrate the beneficial effects of the model interaction method provided by the present application. Figure 11 is a schematic diagram of the comparison of different model interaction methods provided by the present application.

[0212] Please refer to Figure 11. In common technology, it takes two minutes (2 minutes) or more for the user to describe the problem, one minute (1 minute) for the user to post information, and one minute for model inference. The total time for model interaction is more than 4 minutes.

[0213] Please refer to Figure 11. In the model interaction method provided in the embodiment of the present application, the early data preparation is actively collected by the electronic device, which usually takes less than 1 second (second, s), the model inference requires 1 minute, and the total time for model interaction is about 1 minute, saving about 3 minutes.

[0214] In the model interaction method provided in the embodiment of the present application, the electronic device can collect the aforementioned first information and second information (such as the context information of the first interface), simplify the process of providing context information, and reduce the user's operation complexity on the basis of ensuring the accuracy and timeliness of the AI ​​model's response (answer or second content). Moreover, the AI ​​model generates the second content in advance, shortening the time the user waits for the model to respond. In addition, because the electronic device uses a deep neural network to vectorize the collected information, when the electronic device matches the obtained target vector with the question template in the question library, the adaptability between these question templates and the AI ​​model is improved, thereby ensuring that in professional fields such as medicine, law or engineering, when different users express the same question, the AI ​​model can provide consistent output results. Solving these problems will greatly improve the application efficiency and user experience of large language models (AI models) in different professional fields, while simplifying the interaction process between users and models, improving the professional performance and consistency of the model, and thus bringing significant value.

[0215] It is understandable that in order to implement the functions in the above embodiments, the electronic device includes hardware structures and / or software modules corresponding to the execution of each function. It should be readily apparent to those skilled in the art that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0216] The above describes in detail the model interaction method provided according to the embodiment of the present application in conjunction with Figures 1 to 11. The following describes the model interaction device provided according to the embodiment of the present application in conjunction with Figure 5.

[0217] Figure 12 is a schematic diagram of the structure of a model interaction device provided by the present application. These model interaction devices can be used to implement the functions of the electronic device in the above-mentioned method embodiments, and thus can also achieve the beneficial effects possessed by the above-mentioned method embodiments. In the embodiments of the present application, the model interaction device can be any electronic device as shown in Figure 1, or the electronic device described in the subsequent embodiments, or can be applied to a model interaction system including an electronic device and a display unit.

[0218] As shown in Figure 12, the model interaction device 1200 includes: a display module 1210 and an acquisition module 1220. The display module 1210 is used to display the first interface of the AI ​​model; and, in response to a first operation on the first interface, output (such as displaying) the first content on the first interface, the first content is obtained by the AI ​​model based on the first operation. The acquisition module 1220 is used to collect the first information of the first interface and the second information associated with the first interface, the first information includes the first content, and the second information includes at least one of the context information of the first interface and the user's identification. In addition, the display module 1210 is also used to output (such as displaying) the second interface, the second interface includes the first content and a preview window, the preview window includes: the second content obtained by the AI ​​model based on the first information and the second information, the second content indicates the expected solution of the first content.

[0219] The model interaction device 1200 of the embodiment of the present application can be implemented by a software module. The model interaction device 1200 according to the embodiment of the present application can be corresponding to the execution of the method described in the embodiment of the present application, and the above-mentioned and other operations and / or functions of each module in the model interaction device 1200 are respectively for implementing the method flow in the aforementioned figures, and for the sake of brevity, they are not further described here.

[0220] It is worth noting that if the model interaction device 1200 is implemented through a software module, for example, the software module can be provided to users through a cloud service subscription model, and users can choose different subscription levels according to their needs; for example, the software module can also provide enterprise-level customization services with professional domain customization, interface personalization and extended functions according to the needs of users or enterprises.

[0221] In addition, the model interaction device with the function of generating the second content in advance provided by this application can also be made into a value-added service and provided to users, which is not limited by this application. When the model interaction device 1200 is through a software module, the model interaction device 1200 can also be embedded in the eDataMate TM Or other large language model (AI model) tool chain systems.

[0222] The model interaction device 1200 of the embodiment of the present application can also be implemented by hardware. For example, the hardware refers to an electronic device. The specific implementation method of the electronic device can be referred to the description of Figure 2 and will not be repeated here.

[0223] In addition, when the model interaction device 1200 is implemented by a model interaction system, the model interaction system may include the electronic device and the display unit provided in FIG2 . The display unit may refer to a display screen or a display panel, or may refer to a projection device such as a projector or a projection unit in a projector, etc., which is not limited in this application.

[0224] The method steps in this embodiment can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in RAM, flash memory, ROM, PROM, EPROM, EEPROM, registers, hard disk, mobile hard disk, CD-ROM or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and storage medium can be located in an ASIC. In addition, the ASIC can be located in a computing device or an electronic device. Of course, the processor and storage medium can also exist as discrete components in a network device or a terminal device.

[0225] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the process or function described in the embodiments of the present application is performed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program or instruction can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired or wireless means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, a hard disk, or a tape; it can also be an optical medium, such as a digital video disc (DVD); it can also be a semiconductor medium, such as a solid state drive (SSD).

[0226] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A model interaction method, characterized in that: Applied to electronic equipment, the method includes: The first interface for outputting AI models; In response to a first operation on the first interface, outputting first content on the first interface, where the first content is obtained by the AI ​​model based on the first operation; collecting first information of the first interface and second information associated with the first interface, where the first information includes the first content, and the second information includes one or both of context information of the first interface and a user identifier; Output a second interface, the second interface includes the first content and a preview window, the preview window includes: the second content obtained by the AI ​​model based on the first information and the second information, the second content indicates the expected solution of the first content.

2. The method according to claim 1, characterized in that The second interface further includes an operation window, where the operation window is used to display the first content, and the preview window is located on an upper layer of the operation window.

3. The method according to claim 1 or 2, characterized in that The preview window includes a first control. After outputting the second interface, the method further includes: In response to a second operation on the first control in the preview window, a third interface is output; wherein the third interface includes the first content and the second control, and when the second control is triggered, the electronic device outputs the preview window.

4. The method according to claim 1 or 2, characterized in that The output second interface includes: outputting a third interface, wherein the third interface includes the first content and the second control; If the second control is triggered, the preview window is output.

5. The method according to any one of claims 1 to 4, characterized in that Before outputting the second interface, the method further includes: generating a target question template based on the first information and the second information; The target question template is processed using the AI ​​model to obtain second content corresponding to the first content.

6. The method according to claim 5, characterized in that The generating a target question template based on the first information and the second information includes: Processing the first information and the second information using a deep neural network to obtain a target vector; A target question template associated with the target vector is determined from a question library of the AI ​​model; the question library of the AI ​​model includes a plurality of set question templates and answers corresponding to the plurality of question templates.

7. The method according to claim 6, characterized in that The using a deep neural network to process the first information and the second information to obtain a target vector includes: Deduplicating the first information and the second information to obtain deduplicated data; A deep neural network is used to process the deduplicated data to obtain the target vector.

8. The method according to claim 6 or 7, characterized in that The using the AI ​​model to process the target question template to obtain second content corresponding to the first content includes: The target question template is input into the AI ​​model to obtain second content corresponding to the first content, where the second content is the answer corresponding to the target question template among the answers corresponding to the multiple question templates.

9. The method according to any one of claims 1 to 8, characterized in that The first content includes: an alarm list of a data management and operation engine, wherein the alarm list includes one or more alarms; The second content includes: detailed information of the first alarm, where the first alarm is one of the one or more alarms, and the detailed information includes one or more of: device information associated with the first alarm, error number, alarm name, occurrence time, and repair suggestions; The context information of the first interface includes: one or more of: a full list, multiple alarm identifiers, multiple source devices, multiple device information, multiple error numbers and multiple repair suggestions.

10. The method according to any one of claims 1 to 9, characterized in that The first content includes: one or more of: name, clinic number, main complaint, medical history, and physical examination information; The second content includes: treatment advice for the combination of the main complaint disease and the name.

11. The method according to any one of claims 1 to 10, characterized in that The method further comprises: According to the first content, the second content and the first operation, the question library of the AI ​​model is updated, and the question library of the AI ​​model includes multiple question templates and answers corresponding to the multiple question templates.

12. The method according to any one of claims 1 to 11, characterized in that The outputting the first interface of the AI ​​model includes: displaying the first interface of the AI ​​model; The outputting the second interface includes: displaying the second interface.

13. A model interaction device, characterized in that: include: A display module, configured to display a first interface of the AI ​​model; In response to a first operation on the first interface, displaying first content on the first interface, where the first content is obtained by the AI ​​model based on the first operation; a collection module, configured to collect first information of the first interface and second information associated with the first interface, wherein the first information includes the first content, and the second information includes at least one of context information of the first interface and a user identifier; The display module is also used to display a second interface, which includes the first content and a preview window. The preview window includes: the second content obtained by the AI ​​model based on the first information and the second information, and the second content indicates the expected solution to the first content.

14. An electronic device, characterized in that: include: One or more memories and a processor; the one or more memories being coupled to the processor; The memory stores computer program code, which includes computer instructions. When the computer instructions are executed by the processor, the electronic device executes the method according to any one of claims 1 to 12.

15. An interactive system, characterized in that: include: A display unit and an electronic device as described in claim 14; the display unit is used to display a first interface of an AI model, and the electronic device responds to a first operation on the first interface and executes the method described in any one of claims 1-12.

16. A computer-readable storage medium, characterized in that The method comprises computer instructions, which, when executed on an electronic device, cause the electronic device to execute the method according to any one of claims 1 to 12.

17. A computer program product, characterized in that The method comprises a computer program or an instruction. When the computer program or the instruction is run on an electronic device, the electronic device executes the method according to any one of claims 1 to 12.

Images