Display control method, system and related device

Abstract

The embodiment of the invention provides a display control method and system and a related device. The method comprises the steps of obtaining a user instruction text corresponding to user voice; wherein the user instruction text is used for expressing a user instruction carried by the user voice; calling a locally deployed first large language model to output response pointing information by the first large language model; wherein the response pointing information is used for expressing an execution large language model for the user instruction text; calling model result data obtained by executing the large language model represented by the response pointing information based on the user instruction text to obtain response result data aiming at the user instruction; and providing response result data to the user. Through the response pointing information output by the first large language model, the execution large language model with the response capability matched with the user instruction text is called to output the corresponding response result data, so that the response result data can flexibly adapt to different requirements of the user, and the user experience is improved.

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of artificial intelligence technology, and more particularly to a display control method, system and related devices. Background Art

[0002] With the rapid development of artificial intelligence technology, especially the gradual maturity of voice recognition technology and natural language processing, the display can be controlled by the user's voice commands, and the interaction between the user and the display is more convenient. Specifically, the voice command issued by the user can be converted into a command text through voice recognition, and then the command text is understood based on natural language processing technology, and finally the display responds.

[0003] However, the existing display has limited responsiveness to the user's voice commands and is difficult to adapt to the various needs of the user, resulting in a poor user experience. Summary of the invention

[0004] In view of this, multiple embodiments of this specification are dedicated to providing a display control method, system and related devices, which can enable response result data to flexibly adapt to different needs of users and improve user experience.

[0005] One embodiment of the present specification provides a method for controlling a display, the method comprising: obtaining a user instruction text corresponding to a user voice; wherein the user instruction text is used to express the user instruction carried by the user voice; calling a locally deployed first large language model so that the first large language model outputs response pointing information; wherein the response pointing information is used to express an execution large language model for the user instruction text; obtaining response result data for the user instruction by obtaining model result data by calling the execution large language model represented by the response pointing information based on the user instruction text; and providing the response result data to the user.

[0006] One embodiment of the present specification provides a control system for a display, the control system comprising: a text acquisition unit, used to acquire a user instruction text corresponding to a user voice; wherein the user instruction text is used to express the user instruction carried by the user voice; a first model calling unit, used to call a locally deployed first large language model so that the first large language model outputs response pointing information; wherein the response pointing information is used to express an execution large language model for the user instruction text; an execution model calling unit, used to obtain response result data for the user instruction by obtaining model result data obtained by calling the execution large language model represented by the response pointing information based on the user instruction text; and a result providing unit, used to provide the response result data to the user.

[0007] One embodiment of the present specification provides a computer device, which includes a memory and a processor, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the display control method described in any of the above embodiments.

[0008] One embodiment of the present specification provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one computer program, and when the at least one computer program is executed by a processor, the display control method described in any of the above embodiments can be implemented.

[0009] In the multiple implementations provided in this specification, the first language model deployed locally is called to obtain the response pointing information output by the first language model for the user instruction text, and then the execution language model represented by the response pointing information is called to obtain the model result data, thereby obtaining the response result data for the user instruction. According to the response pointing information, the execution language model with the response capability matching the user instruction text is selected to output the corresponding response result data, so that the response result data can flexibly adapt to the different needs of the user, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A data interaction diagram of a control system of a display provided for one embodiment of the present specification.

[0011] Figure 2 A schematic diagram of a method for controlling a display provided in accordance with an embodiment of the present disclosure.

[0012] Figure 3 A schematic diagram of a control system of a display provided for one embodiment of the present specification.

[0013] Figure 4 A schematic diagram of a computer device is provided for one embodiment of the present specification. DETAILED DESCRIPTION

[0014] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0015] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0016] Traditional displays usually only display a static interface. When it is necessary to control or operate it, such as changing or switching the displayed content, it is more dependent on hardware devices such as physical buttons, touch screens or remote controls. Through these devices, users can input commands such as clicks and keys to control the display. For users, the convenience is poor.

[0017] In the related art, in order to improve the convenience of user interaction with the display, the control system of the display usually uses voice recognition technology and natural language processing technology to realize the user's voice control of the display. Specifically, for the user voice carrying the user command, the control system of the display can first convert it into text form through voice recognition, and then parse the converted user command text to obtain the command that the display can execute, and finally send the command to the display to obtain a response.

[0018] However, currently, the response that the display can make to the user voice carrying user instructions is usually limited to its internally integrated functional modules, such as interface switching, chart expansion, data display, etc. The response capability is relatively limited, and it is difficult to meet the various different needs of user interaction, and the user experience is poor. For example, for a certain abnormal data in the interface, the user may interact with the display through voice to help analyze the cause of the abnormal data. At this time, even if the user instruction corresponding to the user voice can be recognized and understood, because the functional module for analyzing the cause is not integrated in the control system of the display, the response capability is insufficient, making it difficult for the user to obtain a response that meets his needs.

[0019] Therefore, it is necessary to provide a display control method that can flexibly respond to different needs of users and improve user experience.

[0020] See also Figure 1 This specification provides an example of an application scenario of a display control method. The display can be used as a large screen in an enterprise control center or data monitoring room, and can usually be used to display real-time data and key indicator data. The display control method can be applied to the display to achieve interaction with the user.

[0021] Based on their own needs, users can issue user instructions to the display through voice. The display can obtain user voice carrying user instructions through the voice acquisition module to form audio data, and can convert it into user instruction text by calling the speech-to-text service (STT) and the speech synthesis service (TTS). Specifically, a browser (Web Browser) can be integrated in the display, and the browser can carry the control call logic for the display, and the browser parses and renders the data to present the corresponding interface. Of course, the control call logic can also be independently integrated into the display, such as stored in the display's memory as software, or directly integrated into the display's circuit.

[0022] Next, the display can call the local decision module to determine the execution large language model that has a matching response capability to the user command through semantic understanding and intent recognition of the user command text. Specifically, the local decision module can be a locally deployed local large language model. The local decision module can also be other functional modules with natural language processing capabilities. The local decision module can determine whether the user command corresponding to the user command text matches the function integrated in the display based on the content of the user command text, and can output corresponding response pointing information, which can be used to indicate the execution large language model that has the corresponding response capability to the user command.

[0023] Specifically, when the content of the user instruction text matches the function integrated in the display, the local decision module can determine that the user instruction can be processed by the local large language model, and the execution large language model represented by the output response pointing information can be the local large language model. For example, for the large screen in the enterprise control center or data monitoring room, if the user wants to view the current occupancy data of the conference room, the corresponding user instruction can be matched with the data viewing or data expansion function integrated in the display, and the execution large language model represented by the response pointing information output by the local decision module can be the local large language model. Since the parameter scale of the local large language model is usually small, and the input text usually only involves content related to the function integrated in the display, the user instruction text matching the function is input into the local large language model for processing, and the response speed is faster.

[0024] Furthermore, according to the response direction information, the display can construct prompt instructions based on the user instruction text by calling the local large language model for input, and obtain output structured instruction information. According to the structured instruction information, the display can control the display to respond based on the integrated functions, such as displaying the current occupancy rate data of the conference room on the display screen, switching the display background, etc.

[0025] In the case where the content of the user command text does not match the functions integrated into the display, the local decision module can determine that the user command has exceeded the response capability of the local large language model, and the local large language model cannot make a correct response to the user command. At this time, the response pointing information output by the local decision module can represent the cloud-based large language model outside the display. The parameter scale of this large language model is larger than that of the local large language model, so it has a stronger response capability. For example, for the large screen in the enterprise control center or data monitoring room, if the user wants to know why the occupancy rate of the conference room is particularly high today, then it is obvious that for this demand, the corresponding user command text exceeds the response capability of the local large language model. The execution large language model represented by the response pointing information output by the local decision module can be an external cloud-based large language model.

[0026] Furthermore, according to the response pointing information, the display can first call the RAG (Retrieval-Augmented Generation) service based on the large model to obtain the material data related to the user instruction text. This service is an artificial intelligence technology that combines information retrieval and text generation. This service can enhance the processing power of the large language model, so that it can reference and utilize external knowledge sources when generating responses, thereby improving the accuracy and reliability of the generated content. Then, based on the user instruction text and the material data obtained by calling the RAG service, the user indication prompt instruction is constructed, and the answer text of the response is obtained by calling the cloud-based large language model, and fed back to the display through the RAG service, and finally the answer text is displayed on the screen of the display to respond to the user instruction.

[0027] See also Figure 2 An embodiment of the present specification provides a display control method, which can be applied to a display having network access capability and certain data computing and processing capability.

[0028] In this embodiment, the method for controlling a display may specifically include the following steps.

[0029] Step S110: Acquire a user instruction text corresponding to the user voice; wherein the user instruction text is used to express the user instruction carried by the user voice.

[0030] In some cases, the display can convert the user's voice into text form by acquiring the user command text corresponding to the user's voice, which is convenient for subsequent processing and use.

[0031] In this embodiment, user voice is used to issue user instructions to the display. Specifically, the user can issue user instructions by voice according to their own needs to control the display to respond according to the user instructions. The user voice can be formed into a corresponding audio data stream by the voice acquisition module. The user instruction text can correspond to the user voice and be in text form. The user voice and the corresponding user instruction text can express the same semantics, that is, express the same user instruction. For example, for some vehicle-mounted smart displays, if the user needs to check the current tire pressure status of the vehicle, the user instruction text can be "I want to check the current tire pressure status", and the voice acquisition module can collect the user voice "I want to check the current tire pressure status" to form corresponding audio data.

[0032] In this embodiment, the display can collect audio data corresponding to the user's voice through its internally integrated voice collection module, and convert the audio data corresponding to the user's voice into text form by calling a voice recognition service, such as a voice-to-text service, a voice synthesis service, etc., to obtain a user command text corresponding to the user's voice. In some embodiments, the process of collecting the user's voice and converting the user's voice into a user command text can also be performed by a voice service system that has communication interaction with the display, and then the voice service system sends the final user command text to the display to achieve the acquisition of the user command text.

[0033] Step S120: calling the locally deployed first large language model so that the first large language model outputs response pointing information; wherein the response pointing information is used to express the execution large language model for the user instruction text.

[0034] In some cases, the display can determine the execution large language model for processing the user command text according to the response direction information output by the first large language model by calling the first large language model. Since the first large language model is a locally deployed large language model, the response speed is fast, which can improve the overall response speed of the display to the user command text.

[0035] In this embodiment, the first large language model can be used as a local decision-making module to make decisions on the response subject of the user instruction. Specifically, the display can construct a prompt instruction based on the user instruction text and input it into the first large language model. After the first large language model can perform semantic understanding and intent recognition on the user instruction text, it can evaluate and identify the large language model suitable for processing the user instruction corresponding to the user instruction text, and then make a decision to determine the execution of the large language model and output the corresponding response direction information. For example, the user instruction text can be "I want to view the energy consumption distribution data of xxx company today", then the prompt instruction can be constructed as "I want to view the energy consumption distribution data of xxx company today, please select a large language model that is most suitable for processing the above requirements from the following multiple large language models", and after inputting it into the first large language model, the output response direction information can be obtained.

[0036] In this embodiment, the first language model can be deployed locally. Specifically, local deployment can refer to the local system environment to which the display belongs. For example, the first language model can be directly deployed in the display, or it can be deployed in a peripheral connected to the display, or the first language model can be deployed in a local server. In some embodiments, local deployment can also be understood as the first language model being deployed in a local network environment to which the display is connected. For example, the local network environment can be a local area network.

[0037] In some implementations, in order to improve the accuracy of the response information output by the first language model, the ReAct (Reasoning and Acting) mechanism can be used to let the first language model make a decision before the first language model outputs the response information. Specifically, the ReAct mechanism can introduce a "thinking step" to first retrieve the relevant dialogue history, background knowledge, and other information that is helpful for making better decisions for the input prompt instruction, and then reconstruct the prompt instruction and input it to the first language model for the second time to obtain the output response information.

[0038] In this embodiment, the response pointing information can be used to distinguish and identify different large language models to facilitate the deployment and calling of the display. Specifically, for example, the response pointing information may include model identifiers such as model name, version number, model tag, etc., and may also include the URL or API endpoint corresponding to the large language model.

[0039] In this embodiment, the execution large language model can be used as the main body for processing and responding to user instruction text. Specifically, a plurality of different large language models can be remotely deployed in the cloud outside the display to adapt to various types of user instructions. The execution large language model can be one of the large language models determined by the first large language model and having the response capability that best matches the user instruction. For example, if the user instruction involves questions and answers about professional medical data, the execution large language model can be a pre-trained medical large model. If the user instruction involves problems in common fields, the execution large language model can be an ordinary question-and-answer large model.

[0040] The large language model referred to in the implementation manner of this specification may be a general large language model with world knowledge trained based on ultra-large-scale data samples, so as to use the world knowledge learned to provide comprehensive and accurate answers or solutions. The large language model can be trained by a large number of parameters and large-scale data. The large language model may have a huge structure and complexity to capture a large number of features. Specifically, the large language model may be a general large language model with deep learning capabilities and natural language processing capabilities, and the large language model may have semantic understanding, analysis, reasoning, and natural language generation capabilities. Specifically, the large language model may be used to output the corresponding answer text based on the input question text and the large amount of general knowledge learned in various fields. The large language model may be a general large language model that has learned knowledge in various fields through training. The general large language model can not only learn common sense knowledge in fields such as geography, history, and science, but also learn professional knowledge in professional fields such as medicine, law, and finance. The large language model can be applied to multiple fields such as language translation, intelligent customer service, automatic writing, and speech recognition.

[0041] In some embodiments, the execution large language model may include the first large language model and a remotely deployed second large language model; the first large language model and the second large language model have different model identifiers respectively; the display may call the locally deployed first large language model so that the first large language model outputs response pointing information; wherein the response pointing information includes the model identifier of the execution large language model that executes the user instruction text.

[0042] In some cases, in addition to the locally deployed first language model, the display can call the remotely deployed second language model to make appropriate responses to various types of user instructions. Specifically, the number of the second language model can be one, or it can be multiple large language models with different performance or types. For example, according to different professional knowledge fields, multiple professional large language models for different professional knowledge fields can be deployed as the second language model, which can improve the accuracy of the response and thus improve the user's interactive experience.

[0043] In some embodiments, when the display is calling the locally deployed first language model, the first language model may output response pointing information if it determines that the user instruction text exceeds the response capability of the first language model; wherein the response pointing information includes the model identifier of the second language model.

[0044] In some cases, based on the user command text, when the first language model determines that the user command corresponding to the user command text does not exceed the scope of its own response capabilities, for example, the user command only involves functions that have been integrated in the display, or simple question-and-answer functions, etc., the first language model can directly process and respond; when it is determined that the user command has exceeded the scope of its own response capabilities, such as user commands involving data details or question-and-answer functions in professional fields, it can be processed by calling the externally deployed second language model with stronger response capabilities. In this way, not only the accuracy of the response results is improved, but also the response speed for some user commands can be taken into account, so that the user's interactive experience is better.

[0045] In some embodiments, the response capability scope can be used to indicate the capability boundary of the large language model to process and respond to input instructions. Specifically, the response capability can reflect the response performance of the large language model in a certain aspect, such as response speed, accuracy or comprehensiveness of the response result, etc., and can also reflect the comprehensive response performance of the large language model. The response capability scope can be preset based on the response capability of the large language model and combined with the actual application scenarios and requirements of the display. For example, the functional range integrated in the display can be used as the basis for setting the response capability scope of the first large language model. It is understandable that if an instruction that exceeds the functional range integrated in the display appears in the user instruction corresponding to the user instruction text, it can be determined that the user instruction text exceeds the response capability scope of the first large language model. Of course, the conditions for judging whether it exceeds the response capability scope of the first large language model can also be flexibly set according to needs, and this specification does not make specific restrictions on this.

[0046] In some embodiments, when the display is calling the locally deployed first language model, the first language model can also output response pointing information when it is determined that the user instruction text matches the function item integrated with the display; wherein the response pointing information includes the model identifier of the first language model.

[0047] In some cases, when the user command text matches the function items integrated into the display, the locally deployed first large language model is used directly as the execution large language model to respond. Since there is no need to call API and external interaction, the response speed can be improved and the user experience is better.

[0048] In some implementations, the function items integrated in the display can be used to implement specific functions of the display. Specifically, each different function item can correspond to an independent functional module, such as adjusting clarity, changing chart style, switching interfaces, chart drilling, etc.

[0049] In some embodiments, if the first language model understands the semantics expressed by the user instruction text, or recognizes the user's intention based on the user instruction text, that is, to use the function corresponding to the function item, then the user instruction text can be determined to match the function item integrated with the display. Specifically, for example, the user instruction text can be "I want to change a chart style", in which case the first language model can recognize that the user's intention is to change the display background, and then the user instruction text can be determined to match the function item of changing the chart style.

[0050] Step S130: Obtaining response result data for the user instruction by calling the execution large language model represented by the response pointing information based on the user instruction text to obtain model result data.

[0051] In some cases, the display calls the execution large language model to obtain response result data for the user command, and then hands the user command to the execution large language model with matching response capabilities for processing, which can broaden the overall response capability of the display and thus enhance the user's interactive experience.

[0052] In this embodiment, the display can construct a prompt instruction based on the user instruction text and input it into the execution large language model to obtain output model result data, and then based on the model result data, response result data for display in the display can be obtained.

[0053] In this embodiment, the model result data is used to represent the result outputted by executing the prompt instruction constructed by the large language model for the user instruction text. Since the content or type of different user instructions processed by the large language model are different, the output model result data may also be different. Specifically, for example, the model result data may be the answer text for answering the prompt instruction constructed for the user instruction text, or it may be the structured instruction information directly generated for the prompt instruction constructed for the user instruction text and can be executed by the display.

[0054] It can be understood that in order to improve the accuracy of the model result data output by the large language model, the ReAct mechanism is also introduced to enrich the content of the prompt instruction, and then it is input into the large language model for the second time to obtain the model result data.

[0055] In some embodiments, the parameter scale of the second largest language model is greater than the parameter scale of the first largest language model; the display may send the user instruction text to the interface service of the second largest language model when the model identifier included in the response pointing information indicates the second largest language model, so that the interface service constructs a user indication prompt instruction according to the user instruction text and inputs it into the second largest language model; wherein the interface service obtains matching material data according to the user instruction text, and the user indication prompt instruction includes the material data, so that the second largest language model generates an answer text corresponding to the user instruction text based on the material data; and receives the answer text fed back by the interface service; wherein the answer text may be used as model result data or directly as response result data.

[0056] In some cases, since the second largest language model has a larger parameter scale and can understand more input content, the output result is based on more input-related information and is more accurate. Therefore, by calling the interface service to obtain the material data related to the user instruction text, and constructing the second largest language model for the user instruction prompt instruction input based on the material data and the user instruction text, the answer text output by the second largest language model can be more accurate.

[0057] In some implementations, the interface service can be used as an intermediate service for communication between the second largest language model and the display. Specifically, the interface service can be configured with a communication interface for accessing the second largest language model, and the display can access the second largest language model by calling the communication interface.

[0058] In some implementations, an information retrieval service may also be integrated into the interface service. When the interface service receives a request, relevant information may be retrieved from an external knowledge base or database to enhance the response generated by the second language model and ensure the accuracy and timeliness of the output content. Such information may be documents, web pages, database records, etc. Specifically, for example, the information retrieval service may be a RAG service, a knowledge graph, a semantic search engine, etc.

[0059] Specifically, the interface service can obtain relevant information matching the user instruction text based on the information retrieval service according to the received user instruction text, and use the obtained relevant information as material data for constructing the user instruction prompt instruction, and construct the user instruction prompt instruction input second language model in combination with the user instruction text to obtain the answer text output by the second language model. Then, the interface service can return the answer text responded by the second language model to the display as the response result data in response to the user instruction.

[0060] In some embodiments, the display is integrated with a function identifier of a function item; the display can input a function instruction prompt word constructed based on the user instruction text into the first large language model to obtain a target function identifier output by the first large language model, and the target function identifier is used as model result data; wherein the target function identifier represents a function item matching the user instruction text; and response result data for executing the corresponding function item is obtained according to the target function identifier.

[0061] In some embodiments, the function instruction prompt is used to express the function instruction corresponding to the function item. The function instruction prompt may be consistent with the semantics expressed by the user instruction text. Specifically, the function instruction prompt may be obtained after polishing or adding relevant information to the user instruction text based on the content of the user instruction text. Of course, in some embodiments, the user instruction text may also be directly used as the function instruction prompt.

[0062] In some embodiments, the function identifier can be used to distinguish and identify the function items. Different function items have different function identifiers. Specifically, in the display, the function identifier can be obtained by assigning a label to each function item. For example, the function items may include multiple types such as adjusting clarity, changing chart style, switching interface, and chart drilling down. These function items can be assigned labels A, B, C, and D respectively. The function identifier corresponding to adjusting clarity is A, the function identifier corresponding to changing chart style is B, the function identifier corresponding to switching interface is C, and the function identifier corresponding to chart drilling down is D. Of course, the form of the function identifier may not be limited to this. For example, it may also be other forms such as address, memory size, etc. This specification does not make specific limitations on this, and it is sufficient to achieve the distinction and identification of function items.

[0063] In some embodiments, the first language model can generate structured instruction information that can be read and recognized by the display based on the target function identifier and combined with the content of the user instruction text, and the structured instruction information can also be used as model result data. Further, the display can call the corresponding function module according to the structured instruction information to obtain the response result data of executing the corresponding function item. Of course, in some embodiments, the display can also directly call the corresponding function module according to the target function identifier to obtain the response result data of executing the corresponding function item.

[0064] In some embodiments, the first large language model further outputs a target parameter corresponding to the target function identifier; the display may also obtain function data corresponding to the target parameter and the function item represented by the target function identifier, and use the function data as the response result data.

[0065] In some embodiments, the target parameter can be used to represent the display interface element configured in the display that matches the user instruction text. Specifically, a plurality of display interface elements can be configured in the display, including buttons, labels, texts, charts, etc., to form a display interface together and present it to the user. Among them, each display interface element can correspond to a preset interface parameter, and the interface parameter corresponding to the display interface element that matches the user instruction text can be used as the target parameter. For example, the user instruction text may be "I want to view the detailed energy consumption data corresponding to March in the energy consumption distribution chart", then the display interface element that matches it may be the display interface element involved in the content of the user instruction text, including the energy consumption distribution chart, and the text "March" in the energy consumption distribution chart, and the corresponding interface parameter is the target parameter. Regarding the specific form of the interface parameter and the target parameter, reference can be made to the form of the function identifier, which can distinguish and identify different display interface elements, and this specification does not make specific limitations on this.

[0066] In some embodiments, the function data may be used to control the display to execute the corresponding function item. Specifically, for example, the function data may include chart data, text data, etc. The display may control the display to display through a control signal based on the acquired function data and control data such as layout data, status data, and command data for controlling the display, so as to realize the corresponding function item.

[0067] In some embodiments, the first language model can generate structured instruction information based on the target function identifier and the target parameter. Further, the display can call the corresponding function module according to the structured instruction information to obtain the function item represented by the target function identifier and the function data corresponding to the target parameter. Compared with obtaining the response result data based only on the target function identifier, using the function item represented by the target function identifier and the function data corresponding to the target parameter as the response result data can make the response to the user instruction more accurate.

[0068] Step S140: providing the response result data to the user.

[0069] In some cases, the display completes the process of responding to the user's instructions by providing the user with response result data, which facilitates the user's subsequent further processing.

[0070] In this embodiment, the display can provide the response result data to the user by controlling the display to display in its display interface, and the user can view the response result data in the display interface to determine whether the response result meets the requirements. In some embodiments, the display can also provide the response result data to the user in other ways. For example, the response result data can be provided to the user by sound output through a speaker integrated in the display. For another example, for some response result data that may not be displayed in the display or have poor display effects due to software or hardware limitations, it can be provided by sending an access link to the user.

[0071] See also Figure 3 One embodiment of the present specification may also provide a control system for a display, and the control system may include the following units.

[0072] The text acquisition unit is used to acquire the user instruction text corresponding to the user voice; wherein the user instruction text is used to express the user instruction carried by the user voice.

[0073] A local model calling unit is used to call a locally deployed first large language model so that the first large language model outputs response pointing information; wherein the response pointing information is used to express the execution large language model for the user instruction text.

[0074] The execution model calling unit is used to obtain the response result data for the user instruction by calling the execution large language model represented by the response pointing information based on the user instruction text to obtain the model result data.

[0075] The result providing unit is used to provide the response result data to the user.

[0076] The specific functions and effects achieved by the control system of the display can be explained with reference to other embodiments of this specification, and will not be repeated here. Each unit in the control system of the display can be implemented in whole or in part by software, hardware, or a combination thereof. Each unit can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to the above units.

[0077] In this embodiment, the control system may also include: a second model calling unit, which is used to send the user instruction text to the interface service of the second large language model when the user instruction text exceeds the response capability of the first large language model, so that the interface service constructs a user indication prompt instruction according to the user instruction text and inputs it into the second large language model; wherein the interface service obtains matching material data according to the user instruction text, and the user indication prompt instruction includes the material data, so that the second large language model generates an answer text corresponding to the user instruction text based on the material data.

[0078] Regarding the specific functions and effects of the second model calling unit, please refer to other implementation methods of this specification for comparative explanation, and will not be repeated here.

[0079] An embodiment of the present specification may also provide a control system for a display, and the control system may include the following modules:

[0080] A local decision module is used to call the first language model locally deployed on the display according to the function instruction prompt words constructed by the user instruction text when the user instruction text corresponding to the user voice matches the function item integrated with the display; or, when the user instruction text corresponding to the user voice exceeds the response capability scope of the local large language model, call the second language model remotely deployed according to the user instruction prompt instruction constructed by the user instruction text;

[0081] A first large language model is used to output first model result data according to the function instruction prompt word, so that the display displays response result data obtained based on the first model result data;

[0082] The second large language model is used to output second model result data according to the user indication prompt instruction, so that the display displays the second model result data as response result data.

[0083] In this embodiment, the first model result data and the second model result data can be compared and explained with reference to the model result data introduced in the above-mentioned embodiment.

[0084] Regarding the specific functions and effects achieved by the control system of the display, please refer to other implementation methods of this specification for comparative explanation, and will not be repeated here.

[0085] The embodiments of the present specification also provide a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a computer, the computer executes the display control method in any of the above embodiments.

[0086] The embodiments of the present specification also provide a computer program product including instructions, which, when executed by a computer, enables the computer to execute the display control method in any of the above embodiments.

[0087] See also Figure 4 The present embodiment may provide a computer device, the computer device comprising: a memory, and one or more processors communicatively connected to the memory; the memory stores instructions executable by the one or more processors, the instructions are executed by the one or more processors, so that the one or more processors implement the display control method in any of the above embodiments.

[0088] In some embodiments, the computer device may include a processor, a non-volatile storage medium, an internal memory, a communication interface, a display device, and an input device connected by a system bus. The non-volatile storage medium may store an operating system and related computer programs.

[0089] The user information or user account information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, etc.) involved in multiple implementation methods of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0090] It should be understood that the specific examples herein are only intended to help those skilled in the art to better understand the embodiments of the present specification, rather than to limit the scope of the present invention.

[0091] It can be understood that in the various implementations of this specification, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the implementation methods of this specification.

[0092] It can be understood that the various embodiments described in this specification can be implemented individually or in combination, and the embodiments of this specification are not limited to this.

[0093] Unless otherwise specified, all technical and scientific terms used in the embodiments of this specification have the same meaning as those generally understood by those skilled in the art of the technical field of this specification. The terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the scope of this specification. The term "and / or" used in this specification includes any and all combinations of one or more related listed items. The singular forms of "a", "above", and "the" used in the embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0094] It can be understood that the processor of the embodiment of this specification can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method implementation can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiment of this specification can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of this specification can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined and executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0095] It is understood that the memory in the embodiments of this specification may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (programmable ROM, PROM), an erasable programmable read-only memory (erasablePROM, EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory may be a random access memory (RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0096] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this specification.

[0097] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method implementation methods and will not be repeated here.

[0098] In the several embodiments provided in this specification, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0099] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0100] In addition, each functional unit in each embodiment of the present specification may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0101] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this specification, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of this specification. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0102] The above is only a specific implementation of this specification, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in this specification, which should be included in the protection scope of this specification. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for controlling a display, It is characterized in that The method comprises: Acquire a user command text corresponding to the user voice; wherein the user command text is used to express the user command carried by the user voice; Calling a locally deployed first large language model so that the first large language model outputs response pointing information; wherein the response pointing information is used to express an execution large language model for the user instruction text; Obtaining response result data for the user instruction by calling the execution large language model represented by the response pointing information based on the user instruction text to obtain model result data; The response result data is provided to the user.

2. The method according to claim 1, It is characterized in that The execution large language model includes the first large language model and a remotely deployed second large language model; the first large language model and the second large language model have different model identifiers respectively; The step of calling the locally deployed first largest language model to output response pointing information by the first largest language model includes: The first large language model outputs response pointing information; wherein the response pointing information includes a model identifier representing an execution large language model for executing the user instruction text.

3. The method according to claim 2, It is characterized in that The step of calling the locally deployed first largest language model to output response pointing information by the first largest language model includes: When the first language model determines that the user instruction text exceeds the response capability of the first language model, response pointing information is output; wherein the response pointing information includes a model identifier of the second language model.

4. The method according to claim 3, It is characterized in that The parameter scale of the second largest language model is greater than the parameter scale of the first largest language model; The step of obtaining the response result data for the user instruction by calling the model result data obtained by executing the large language model represented by the response pointing information based on the user instruction text comprises: In the case where the model identifier included in the response pointing information indicates the second language model, the user instruction text is sent to the interface service of the second language model, so that the interface service constructs a user indication prompt instruction according to the user instruction text and inputs it into the second language model; wherein the interface service obtains matching material data according to the user instruction text, and the user indication prompt instruction includes the material data, so that the second language model generates an answer text corresponding to the user instruction text based on the material data; Receive the answer text fed back by the interface service; wherein the answer text serves as the response result data.

5. The method according to claim 2, It is characterized in that The display has multiple functional items integrated therein; The step of calling the locally deployed first largest language model to output response pointing information by the first largest language model includes: When the first language model determines that the user instruction text matches the function item integrated with the display, response pointing information is output; wherein the response pointing information includes a model identifier of the first language model.

6. The method according to claim 5, It is characterized in that The display has a function identifier integrated into it. The step of obtaining the response result data for the user instruction by calling the model result data obtained by executing the large language model represented by the response pointing information based on the user instruction text comprises: Inputting the function instruction prompt words constructed based on the user instruction text into the first large language model to obtain the target function identifier output by the first large language model; wherein the target function identifier represents the function item matching the user instruction text; Acquire response result data of executing the corresponding function item according to the target function identifier.

7. The method according to claim 6, It is characterized in that In the step of inputting the function instruction prompt words constructed based on the user instruction text into the first large language model to obtain the target function identifier output by the first large language model, the first large language model also outputs a target parameter corresponding to the target function identifier; The step of acquiring response result data of executing the corresponding function item according to the target function identifier comprises: The function item represented by the target function identifier and the function data corresponding to the target parameter are obtained, and the function data is used as the response result data.

8. A control system for a display, It is characterized in that The control system comprises: A text acquisition unit, used to acquire a user instruction text corresponding to the user voice; wherein the user instruction text is used to express the user instruction carried by the user voice; A first model calling unit, configured to call a locally deployed first large language model, so that the first large language model outputs response pointing information; wherein the response pointing information is used to express an execution large language model for the user instruction text; An execution model calling unit, configured to obtain response result data for the user instruction by calling the execution large language model represented by the response pointing information based on the user instruction text; The result providing unit is used to provide the response result data to the user.

9. The control system according to claim 8, It is characterized in that The control system further comprises: The second model calling unit is used to send the user instruction text to the interface service of the second largest language model when the user instruction text exceeds the response capability of the first largest language model, so that the interface service constructs a user indication prompt instruction according to the user instruction text and inputs it into the second largest language model; wherein the interface service obtains matching material data according to the user instruction text, and the user indication prompt instruction includes the material data, so that the second largest language model generates an answer text corresponding to the user instruction text based on the material data.

10. A computer device, It is characterized in that The computer device includes a memory and a processor, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

11. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores at least one computer program, and when the at least one computer program is executed by a processor, the method according to any one of claims 1 to 7 can be implemented.

12. A computer program product, It is characterized in that The computer program product is used to implement the method according to any one of claims 1 to 7.

13. A control system for a display, It is characterized in that The control system comprises: A local decision module is used to call the first language model locally deployed on the display according to the function instruction prompt words constructed by the user instruction text when the user instruction text corresponding to the user voice matches the function item integrated with the display; or, when the user instruction text corresponding to the user voice exceeds the response capability scope of the local large language model, call the second language model remotely deployed according to the user instruction prompt instruction constructed by the user instruction text; A first large language model is used to output first model result data according to the function instruction prompt word, so that the display displays response result data obtained based on the first model result data; The second large language model is used to output second model result data according to the user indication prompt instruction, so that the display displays the second model result data as response result data.

Images