Screen content generation method, display method, device, medium and program product

Abstract

An embodiment of the present application provides a method and a display method for generating screen content, a device, a medium, and a program product, which relate to the field of computer technologies. The method includes: receiving a processing instruction, where the processing instruction is used to indicate the content processing requirements of a target screen through natural language; in response to the processing instruction, using a preset recognition model to parse each task that needs to be executed from the content processing requirements; based on the task type to which each task belongs, calling a task execution tool corresponding to the task type to execute the corresponding task, and obtaining the screen display content corresponding to the content processing requirements. In the technical solution of the embodiment of the present application, the processing instruction of the target screen can be automatically parsed and the corresponding task execution tool can be called to implement the content processing of the target screen, which is beneficial to reducing the labor cost and time cost and efficiently and accurately implementing the content processing requirements of the target screen.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a method for generating screen content, a method for displaying the same, a device, a medium, and a program product. Background Art

[0002] Digital large screens are a data visualization technology that centrally displays various types of data information, such as sales data and implementation management information, on a large screen through a graphical interface. Digital large screens can provide intuitive and real-time data displays to help users better understand and manage complex data information. Currently, when constructing digital large screen content, it usually requires the operator to have certain programming knowledge or rely on a dedicated technical team to complete, and the entire construction process will consume a large amount of human and time costs. Summary of the Invention

[0003] Embodiments of this application provide a method for generating screen content, a method for displaying the same, a device, a medium, and a program product to alleviate or solve one or more technical problems existing in the prior art.

[0004] In a first aspect, an embodiment of this application provides a method for generating screen content. The method includes: receiving a processing instruction, where the processing instruction is used to indicate the content processing requirements of a target screen through natural language; in response to the processing instruction, using a preset recognition model to parse each task that needs to be executed from the content processing requirements; based on the task type to which each task belongs, calling a task execution tool corresponding to the task type to execute the corresponding task, and obtaining the screen display content corresponding to the content processing requirements.

[0005] In a second aspect, an embodiment of this application provides a method for displaying screen content, which is applied to a target screen. The method includes: receiving a display instruction for the target screen; in response to the display instruction, displaying the current screen display content of the target screen; receiving a processing instruction for the target screen, where the processing instruction is used to indicate the content processing requirements for the target screen through natural language; in response to the processing instruction, displaying new screen display content, where the new screen display content is the screen display content corresponding to the content processing requirements obtained by processing the processing instruction according to the screen content generation method in the first aspect.

[0006] In a third aspect, an embodiment of this application provides an electronic device, including a memory, a processor, and a computer program stored on the memory. The processor implements the method according to any one of the embodiments of this application when executing the computer program.

[0007] Fourthly, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the method according to any one of the embodiments of the present application is implemented.

[0008] Fifthly, an embodiment of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, the method according to any one of the embodiments of the present application is implemented.

[0009] According to the screen content generation method of the embodiment of the present application, the content processing requirements for the target screen indicated by natural language can be obtained according to the received processing instruction, and each task required to implement the content processing requirements can be parsed from the content processing requirements by using a preset recognition model, and the corresponding task can be executed by a task execution tool, so as to obtain the screen display content corresponding to the content processing requirements.

[0010] In this technical solution, the preset recognition model has the ability of natural language understanding and reasoning, which is beneficial to accurately parsing the content processing requirements of the operator for the target screen, so as to obtain tasks that more meet the processing requirements of the operator and improve the accuracy of the generated screen display content. Moreover, in this solution, the advantage of configuring specific task execution tools for different task types is that: since each task is classified and can be executed by the task execution tool corresponding to the task type, the probability that each task waits for the previous task to be completed before starting to execute is greatly reduced during the execution process, which is beneficial to reducing the waiting time before task execution and improving the response speed of task execution; each task execution tool has a high matching degree with a specific type of task, which is beneficial to improving the professionalism of task processing. During the processing of this solution, the operator does not need to deeply understand the technical details, and can automatically parse the processing instruction for the target screen and call the corresponding task execution tool to implement the content processing of the target screen without programming knowledge and a dedicated technical team, which is beneficial to efficiently and accurately implementing the content processing requirements of the target screen while reducing labor costs and time costs.

[0011] The above description is only an overview of the technical solution of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the specific embodiments of the present application are given below. Description of the Drawings

[0012] In the drawings, unless otherwise specified, the same reference numerals throughout the several views denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments according to the present application and should not be regarded as limiting the scope of the present application.

[0013] Figure 1 Shows a flowchart of the screen content generation method according to an embodiment of the present application;

[0014] Figure 2 Shows a task dependency graph of an exemplary embodiment of the present application;

[0015] Figure 3 Shows a detailed flowchart of the screen content generation method according to an exemplary embodiment of the present application;

[0016] Figure 4 Shows a flowchart of the screen content display method according to an embodiment of the present application;

[0017] Figure 5 Shows a schematic structural diagram of a screen content generation device according to an embodiment of the present application;

[0018] Figure 6 Shows a schematic structural diagram of a screen content display device according to an embodiment of the present application;

[0019] Figure 7 Shows a block diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0020] In the following, only some exemplary embodiments are briefly described. As those skilled in the art can recognize, the described embodiments can be modified in various different ways without departing from the concept or scope of the present application. Therefore, the drawings and the description are considered to be exemplary in nature and not restrictive.

[0021] To facilitate the understanding of the technical solutions of the embodiments of the present application, the related technologies of the embodiments of the present application are described below. The following related technologies can be arbitrarily combined with the technical solutions of the embodiments of the present application as optional solutions, and they all fall within the protection scope of the embodiments of the present application.

[0022] In actual application scenarios, with the accelerating advancement of digital transformation, digital large screens, as important carriers for information display, play a key role in multiple fields such as business analysis, real-time management, and data visualization. However, when constructing the content of digital large screens, it usually requires operators to have certain programming knowledge or rely on specialized technical teams to complete. Therefore, the current process of processing the screen content of digital large screens is highly dependent on programming skills or professional technical teams, which not only raises the operation threshold but also severely restricts the participation of non-technical background users and the content update efficiency.

[0023] In some related technologies, data visualization functions can be provided through business intelligence and analytics software (Tableau). However, this software is mainly targeted at users with a certain technical background and does not support directly modifying charts or layouts through natural language interaction. In related technologies, chart production is also carried out through data analysis tools (Power BI), but its natural language query function is mainly used for data retrieval rather than interface content processing. Also, report generation is carried out through enterprise-level data analysis applications (Looker). Although this software has simple natural language search options, custom requirements for visual presentation still need to be implemented by writing code.

[0024] In some other related technologies, a digital large screen can be controlled by a voice assistant to view the weather forecast, play music, etc. through voice commands. However, their functions are mainly used for data retrieval and cannot meet the configuration requirements of data visualization projects. Also, there are some Graphical User Interface (GUI) applications that can create a dashboard visualization system by dragging elements. However, this method often requires users to understand knowledge such as basic HyperText Markup Language (HTML) and Cascading Style Sheets (CSS), which poses a great challenge to those who do not understand the technology in the professional field.

[0025] It should be noted that the above application scenarios or application examples provided in the embodiments of the present application are for easy understanding, and the embodiments of the present application do not make specific limitations on the application of the technical solutions. In addition, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data that have been authorized by the user or fully authorized by all parties. And the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0026] The technical solutions of the present application and how the technical solutions of the present application solve the foregoing technical problems will be described in detail below with specific embodiments. The several specific embodiments listed can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0027] Figure 1 The flowchart of the screen content generation method according to the embodiment of the present application is shown, as Figure 1 shown, the method may include the following steps S101, step S102, and step S103.

[0028] Step S101: Receive a processing instruction, which is used to indicate the content processing requirements of the target screen through natural language.

[0029] Step S102: In response to the processing instruction, use a preset recognition model to parse out each task that needs to be executed from the content processing requirements.

[0030] Step S103: Based on the task types to which the tasks belong, call the task execution tools corresponding to the task types to execute the corresponding tasks, and obtain the screen display content corresponding to the content processing requirements.

[0031] According to the screen content generation method of the embodiments of the present application, the content processing requirements for the target screen indicated by natural language can be obtained according to the received processing instruction, each task required to implement the content processing requirements can be parsed out from the content processing requirements by using a preset recognition model, and the corresponding tasks can be executed by the task execution tools, so as to obtain the screen display content corresponding to the content processing requirements.

[0032] In this technical solution, using a preset recognition model helps to accurately parse the content processing requirements of the operator for the target screen, so as to obtain tasks that better meet the operator's processing requirements and improve the accuracy of the generated screen display content. Moreover, in this solution, the advantage of configuring specific task execution tools for different task types is that: since each task is classified and can be executed by the task execution tool corresponding to the task type, the probability that each task waits for the previous task to complete before it can start to execute is greatly reduced during the execution process, which is conducive to reducing the waiting time before task execution and improving the response speed of task execution; each task execution tool has a high degree of matching with a specific type of task, which is conducive to improving the professionalism of task processing. During the processing of this solution, the operator does not need to deeply understand the technical details, and can automatically parse the processing instruction for the target screen and call the corresponding task execution tool to implement the content processing of the target screen without programming knowledge and a dedicated technical team, which is conducive to efficiently and accurately implementing the content processing requirements of the target screen while reducing labor costs and time costs.

[0033] In step S101, the processing instruction is an instruction described in natural language. Exemplarily, natural language can be understood as the language naturally generated for communication between people. Different languages such as Chinese, English, French, etc. can be used, or it can be a dialect under a certain language, such as Cantonese, Minnan dialect, etc. The embodiments of the present application do not make specific limitations.

[0034] Exemplarily, the processing instruction can be at least one of the forms of text and voice.

[0035] As a specific example, a first interaction element may be included on the user terminal, and the first interaction element is used to trigger the reception of a processing instruction in text form. In response to an operation instruction for the first interaction element, a processing instruction for receiving text input is received through a text input device. Among them, the first interaction element includes, but is not limited to: a text input box, a drop-down list, a radio button, a checkbox, etc. The text input device includes at least one of the following: a keyboard, a mouse, a graphics tablet, a touch screen, etc.

[0036] As a specific example, a second interaction element may also be included on the target screen, and the second interaction element is used to trigger the reception of a processing instruction in voice form. In response to an operation instruction for the second interaction element, a processing instruction for collecting user voice input by an audio collection device mounted on the electronic device to which the target screen belongs is triggered. Among them, the second interaction element includes at least one of the following: a button, a menu item, etc. The operation instructions for the first interaction element and the second interaction element may be instructions triggered by operations such as a click operation and a selection operation. The audio collection device includes, but is not limited to, a microphone unit. The microphone unit includes, for example, a single microphone or a microphone array, and the number of microphones in the microphone array is at least 2.

[0037] Exemplarily, the content processing requirement is used to define the target for processing the target screen. The type of the content processing requirement may include at least one of creation, modification, and update. For example, when the processing instruction is "add a pie chart to show the proportion of each product category", it indicates a creation-type content processing requirement (create a new chart). For example, when the processing instruction is "replace the line chart of sales amount in the upper left corner with a bar chart", it indicates a modification-type content processing requirement (modify the existing chart). For another example, when the processing instruction is "update the statistical chart of sales amount in the fourth quarter", it indicates an update-type content processing requirement (update the chart data). For another example, when the processing instruction is "update the statistical chart of sales volume in the fourth quarter and add a bar chart to show the monthly change trend of sales volume", the content processing requirements indicated include two types: creation and update (update the chart data and create a new chart).

[0038] It should be understood that the above content processing requirements are only illustrative, and the specific content of the processing instruction can be customized according to the actual processing requirements for the target screen, and the embodiments of the present application do not make specific limitations.

[0039] In step S102, the preset recognition model is a model with natural language understanding ability and natural language reasoning ability. Exemplarily, the preset recognition model includes, but is not limited to, any one of the following models: Large Language Models (LLMs), and a joint model for intent recognition and semantic slot filling based on Bidirectional Encoder Representations from Transformers.

[0040] Among them, the large language model is abbreviated as the large model. Based on the natural language understanding ability and natural language inference (NLI) technology of the large model, the specific content processing requirements are parsed from the processing instructions in natural language form, and a task that better meets the operator's processing requirements is obtained, improving the accuracy of the generated screen display content.

[0041] Among them, the joint model for intent recognition and semantic slot filling based on the bidirectional encoder can encode the content processing requirements in natural language form through a pre-trained bidirectional encoder model to obtain the context representation of each word, so as to capture the context information of the content processing requirements, and thus perform intent recognition more accurately. Then, according to the intent recognition result and the structured information extracted by slot filling, a specific task is constructed.

[0042] It should be understood that the preset recognition model can also be other neural network models based on deep learning. This model has natural language understanding and reasoning abilities, and can parse the content processing requirements of the operator for the target screen based on the input content processing requirements in natural language form, so as to obtain each task that needs to be executed. The embodiments of the present application do not make specific limitations on the implementation form of the preset recognition model.

[0043] Specifically, natural language understanding includes, but is not limited to, using the preset recognition model to perform semantic analysis and intent recognition on the content processing requirements. Semantic analysis is used to understand the actual information conveyed in the content processing requirements, and intent recognition is used to identify the true intention of the user conveyed in the content processing requirements. Natural language reasoning can be understood as a process of using the preset recognition model to utilize natural language processing algorithms to understand and infer the logical relationships and meanings in natural language texts, so as to make reasonable judgments and conclusions. Usually, this logical reasoning not only involves the understanding of the literal meaning of the processing instructions, but also includes inferring implicit meanings and intentions.

[0044] In the embodiments of the present application, parsing the content processing requirements in the form of natural language for the target screen by using a preset recognition model helps to understand the true intention of the user in the content processing requirements. Based on the accurate understanding of the user's intention, it helps that each task to be executed can be accurately parsed from the content processing requirements, so as to help execute each task more accurately and efficiently and obtain a more accurate content processing result.

[0045] In some embodiments, the step of parsing each task to be executed from the content processing requirements by using a preset recognition model in step S102 may specifically include: performing intention recognition by using the preset recognition model based on the pre-stored historical conversation content and the content processing requirements; determining each operation to be executed and the parameters required for each operation according to the intention recognition result; creating at least one task according to each operation and parameter, and the task is used to execute the corresponding operation based on at least one parameter.

[0046] Exemplarily, the historical conversation content is used to indicate the pre-collected conversation history related to the content processing of the target screen. For example, the historical conversation content may include but is not limited to at least one of the following: historical processing instructions, feedback and evaluation on the historical screen display content. Based on the historical conversation content and the content processing requirements, it can help the preset recognition model perform more accurate intention recognition. For example, if the information of the content processing requirements indicated by the processing instructions is too simple or incomplete, the preset recognition model can, on the basis of the content processing requirements, combine the historical conversation content to perform more accurate intention recognition.

[0047] Suppose the received processing instruction is "update the sales chart". The content processing requirements indicated by this processing instruction do not provide more detailed information such as the update frequency and chart type. The historical conversation content includes historical processing instructions, such as "update the sales chart of each product monthly" and "update the sales chart in the upper left corner to a bar chart". In this way, the preset recognition model can use these historical conversation contents to improve and update the content processing requirements. Exemplarily, the updated content processing requirements may be, for example: "update the sales chart of each product in the upper left corner monthly and display the specific sales data of each product category in the form of a bar chart".

[0048] In some embodiments, after the step of performing intent recognition using a preset recognition model based on the pre-stored historical session content and content processing requirements, the following steps may further be included: displaying a predetermined interface on a target screen, where the display content of the predetermined interface includes an intent recognition result, a third interaction element, and a fourth interaction element. The third interaction element is used to trigger the generation of an approval message for the intent recognition result, and the fourth interaction element is used to trigger the generation of a rejection message for the intent recognition result. In response to the third interaction element being triggered, the following steps are performed: determining each operation to be performed and the parameters required for each operation according to the intent recognition result. In response to the fourth interaction element being triggered, triggering the reception of user feedback information, and generating a new content processing requirement according to the user feedback information.

[0049] As an example, the user feedback information may be, for example, a new processing instruction or an evaluation information for the intent recognition result. Suppose the user feedback information is "No, what I want is to update quarterly". Then the new content processing requirement may be, for example, "Update the sales charts of each product in the upper left corner quarterly and display the specific sales data of each product category in the form of a bar chart". It should be understood that the third interaction element and the fourth interaction element may be options or buttons provided by the predetermined interface. The operation instructions for the third interaction element and the fourth interaction element may be instructions triggered by operations such as click operations and selection operations. The predetermined interface may be in the form of a pop-up window or a page. The embodiments of the present application do not make specific limitations.

[0050] Exemplarily, in the step of determining each operation to be performed and the parameters required for each operation according to the intent recognition result, the parameters required for each operation may be understood as the input information required when performing each operation, and these parameters are used to indicate the specific execution manner of each operation. For example, for the processing instruction "Change the line chart of sales in the upper left corner to a bar chart", based on the historical session content and the content processing requirements indicated by this processing instruction, after performing intent recognition using a preset recognition model, the specific operation to be performed and the parameters required for the specific operation may be determined according to the intent recognition result. For example, the specific operation includes: changing the chart type. The parameters required for this specific operation include: chart position, current chart name, and target chart type, etc. Specifically, the chart position is "upper left corner", the current chart type is "line chart of sales", and the target chart type is "bar chart".

[0051] Exemplarily, at least one task is created according to this specific operation and the parameters required for the operation. These tasks may include, for example: generating a target position according to the position coordinates of the upper left corner where the line chart of sales is located; extracting the sales data used to generate this line chart from a predetermined data source in the case of confirming that the chart type is a line chart; generating a bar chart according to this sales data; removing the line chart of sales at the target position; rendering a bar chart of the sales data at the target position.

[0052] Exemplarily, one operation can correspond to one task, or multiple operations can be integrated into one task.

[0053] For example, in terms of operation independence, if any two operations can be executed independently, then one operation can correspond to one task; if the operations need to be executed in a specific order, then multiple operations that need to be executed in a specific order can be integrated into one task.

[0054] For another example, in terms of operation complexity, if an operation does not involve data interaction, such as reading data from a certain static file, then this operation can be regarded as a simple operation; if an operation involves data interaction, such as involving calling an external Application Programming Interface (API), or involving multiple data interactions with a database, etc., then this operation can be regarded as a complex operation. For a simple operation, an independent task can be created, which is beneficial to improving the flexibility and maintainability of task processing. For a complex operation, at least two complex operations can be integrated into one task to reduce the system overhead of task switching, which is beneficial to improving task processing efficiency.

[0055] In actual application scenarios, both the situation where one operation corresponds to one task and the situation where multiple operations are integrated into one task may often occur. It can be flexibly set according to needs, and the embodiments of the present application do not make specific limitations.

[0056] In this embodiment, when performing intention recognition, the historical conversation content helps to improve the content processing requirements, and the user's language style, word usage habits, and expression methods can be obtained from the historical conversation. Combining the historical conversation content and the current content processing requirements can help the preset recognition model understand the user's true intention more accurately, thereby helping to ensure that the created task meets the user's true processing requirements for the target screen, and is beneficial to reducing communication costs and improving the overall processing efficiency.

[0057] In some embodiments, determining each operation to be executed and the parameters required for each operation according to the intention recognition result includes: obtaining a preset operation range, where the operation range is used to indicate at least one operation to be performed on each screen component included in the target screen; based on the intention recognition result, extracting each operation to be executed from the operation range, and determining the parameters required for each operation.

[0058] As an example, the information presentation method of the target screen can be defined through the large-screen abstract structure. The information presentation method can be understood as: the specific presentation of each screen constituent content included in the target screen. Specifically, each screen constituent content may include each component part that makes up the target screen. Each screen constituent content includes, for example, but is not limited to at least one of the following items: global elements, screen components, component display content, screen layout structure, and screen style information.

[0059] Among them, global elements include the elements that are always displayed on the target screen. For example, elements such as titles, footers, and navigation bars.

[0060] Screen components include, but are not limited to, at least one of data visualization components, interactive components, and container components. Data visualization components include, but are not limited to, at least one of components such as charts, tables, and lists. Chart types include, but are not limited to, at least one of the following: line charts, bar charts, pie charts, and scatter plots. Interactive components are used to provide user interactions and include, but are not limited to, at least one of the following components: buttons, menus, input boxes, and sliders. Container components are used to create and manage containers on the target screen and include, but are not limited to, at least one of the following components: windows, panels, and dialog boxes.

[0061] Component display content at least includes the specific data or content displayed through data visualization components.

[0062] The screen layout structure is used to indicate the layout method of the target screen, that is, the layout method of each screen component included in each screen constituent content of the target screen. For example, the layout method includes, but is not limited to, the position information, size information of each screen component itself, and the data dependency relationship between each screen component. Among them, the data dependency relationship between screen components describes the data flow between components, that is, how data is transmitted between different screen components.

[0063] Screen style information is used to indicate the visual display style of the target screen. As an example, the visual display style includes, but is not limited to, at least one of the following: the background color and font style of the target screen, etc.

[0064] As an example, the limitation of the operation range by the large-screen abstract structure can be reflected in: at least one operation performed on each screen constituent content. For the sake of easy understanding, the following is illustrated by examples.

[0065] For example, in terms of the information presentation method, the limitation of the operation range includes, but is not limited to, at least one of the following operations: adding or replacing data visualization components of the corresponding type according to the data type, modifying the layout method of the data visualization component (that is, modifying the position information and size information of the corresponding component), adjusting the title on the target screen, updating the display data of any chart, and adjusting the background color of the target screen.

[0066] As a specific example, when the data type is sales performance data, the types of data visualization components may include, but are not limited to: bar charts of sales amount, line charts of sales amount, etc. When the data type is traffic data, the types of data visualization components may include, but are not limited to: bar charts of traffic volume, line charts of vehicle speed, distribution maps of traffic accidents, etc.

[0067] As a specific example, when modifying the layout of a data visualization component, such as a certain chart, the parameter operation range of the chart can be preset, for example: position adjustment range, size adjustment range, etc.

[0068] It should be understood that the large-screen abstract structure can be summarized in advance according to actual usage experience and / or the needs and usage scenarios of the target user group. The operations on the components of each screen included in the above operation range are only illustrative. In actual applications, the target operation range can be set according to actual needs for the user to process instructions, and the embodiments of the present application do not make specific limitations.

[0069] In this embodiment, the operation range for performing operations on the components of each screen can be determined through the large-screen abstract structure. Through this operation range, further screening of each operation that needs to be performed determined according to the intention recognition result is beneficial to ensuring that each operation that needs to be performed is an operation supported by the target screen, which is beneficial to improving the effectiveness of performing operations on the target screen, and thus helps to improve the processing efficiency.

[0070] In some embodiments, in the above step S103, the step of calling a task execution tool corresponding to the task type to execute the corresponding task and obtaining the screen display content corresponding to the content processing requirement may specifically include: generating the dependency relationship between tasks based on a preset screen construction process and a preset screen layout structure. The screen construction process includes the operations required to construct each component of the target screen, and the required operations include each step required in the process of generating the corresponding screen component. The screen layout structure is used to indicate the layout method of the target screen; determining the call order of each task based on the dependency relationship; and calling the task execution tools corresponding to each task in the call order to execute each task to obtain the screen display content corresponding to the content processing requirement.

[0071] Exemplarily, the screen construction process refers to the overall process of creating and deploying a screen display system based on the target screen, and this overall process includes, but is not limited to, the construction processes of each screen component described in the above embodiments. The meaning of the screen layout structure can be referred to the description of the above embodiments and will not be elaborated here.

[0072] Exemplarily, an operation can be understood as a specific step. Taking the screen components in the screen composition content as an example, when the screen components include chart components, the operations required to construct the screen composition content may include, for example, but are not limited to, the following steps: obtaining data from a specified data source (such as a database, a data file, etc.); cleaning the obtained data (removing duplicates, handling missing values, etc.); rendering the chart components according to the processed data; adjusting the position and size of the chart components on the screen. By pre-defining the operations required to construct each screen composition content, it is beneficial to ensure the accurate implementation of the corresponding screen construction content.

[0073] Exemplarily, for the screen construction process, the dependency relationships between tasks may involve the logical order of the operations required to construct each screen composition content. For example, when a countdown time component needs to be added to a target screen, and this component is used to display a countdown on the target screen, it includes at least the following three tasks. Task 1: Set the countdown end time, such as the start time of sales settlement. Task 2: After setting the countdown end time, start the countdown. Task 3: Update the countdown in real time. Among them, Task 1 defines the target time point of the countdown and needs to be completed first; for Task 2, it needs to know when to start the countdown, so this task depends on the completion of Task 1; in Task 3, only after the countdown is started does it need to update the time, so Task 3 depends on the completion of Task 2. Another example is that when relevant data on enterprise operation management needs to be displayed on a target screen, different levels of management personnel have different data access rights. Grassroots management personnel can view daily operation data, while senior management personnel can access more comprehensive operation analysis data. Therefore, when a bar chart needs to be added to the target screen to display operation analysis data, the user permission verification task needs to be executed first, and when the user permission verification is passed, the bar chart data display task is then executed.

[0074] Exemplarily, for the screen layout structure, the data dependency relationships between screen components determine that the output data of one screen component may be the input data of another screen component. For example, a target screen contains a sales amount data component and a profit calculation component, such as a sales amount bar chart and a profit amount pie chart. When calculating the profit, the profit calculation component requires the sales amount data as input data. Therefore, the output (sales amount data) of the sales amount data component is one of the inputs of the profit calculation component (cost data also needs to be obtained). Thus, when the dependency relationships between tasks involve the data dependency relationships between components, the data dependency relationships between these components can determine the dependency relationships between the corresponding tasks.

[0075] Exemplarily, during the processing of the target screen, the dependency relationships between tasks can be generated based on the screen construction process and the screen layout structure. The dependency relationships between tasks are beneficial to ensuring the correctness of the screen construction process and the screen layout structure.

[0076] Exemplarily, in the screen construction process, operations such as data acquisition, data processing, and view rendering are involved. Based on the screen layout structure, the operations involved include, for example: determining the position and size of the chart component. When the user needs to change the line chart of sales amount in the upper left corner to a bar chart, the tasks involved include, for example: the task of acquiring sales amount data, the data processing task (processing the data to be suitable for the bar chart format), the rendering task, the layout task (setting the position and size of the bar chart), and the data display task. By analyzing the screen construction process and the screen layout structure, the following task dependency relationships can be determined: the rendering task of the bar chart depends on the data processing task and the layout task of the sales amount data; the data display task of the bar chart depends on the rendering task of the bar chart.

[0077] In this example, according to the above task dependency relationships, the call order of each task can be determined as follows: acquire sales amount data; process the sales amount data to be suitable for the bar chart format; define the layout structure of the bar chart in the upper left corner area of the target screen, for example: the area where the line chart in the upper left corner is located is used as the predetermined position of the bar chart; render the bar chart at the predetermined position of the bar chart.

[0078] In this embodiment, the dependency relationships between tasks are generated based on the screen construction process and the screen layout structure, the task call order is determined based on the dependency relationships, and then each task is executed according to the task call order. The generation of the dependency relationships ensures that the execution of each task is based on the completion of its previous tasks, thereby ensuring the correctness and processing efficiency of the entire screen construction process and the screen layout.

[0079] Exemplarily, a task execution tool refers to a tool component used to execute corresponding types of tasks. Each tool component can be an agent capable of executing corresponding types of tasks. In the field of artificial intelligence, an agent is defined as an entity that can perceive the environment, make decisions, and take actions, and has learning and decision-making capabilities for autonomously executing specific tasks.

[0080] A model agent is a type of agent, which is constructed based on a preset recognition model. Based on the powerful computing power and learning ability of the preset recognition model, it can provide more advanced functions such as natural language understanding, natural language reasoning, task generation, and task planning. Among them, task planning includes the generation of dependency relationships between tasks and the determination of the call order. Exemplarily, when the model agent is an agent constructed based on a large language model, it can be abbreviated as a large model agent.

[0081] In the above execution steps of the embodiments of the present application, a model agent or other deep learning-based intelligent agent application can be used to generate the dependency relationships between tasks according to a preset screen construction process and a preset screen layout structure, and determine the invocation order of each task based on the dependency relationships. The type of the intelligent agent application can be custom-selected according to actual needs, and the embodiments of the present application do not make specific limitations.

[0082] In this embodiment, by generating the task dependency relationships and determining the task invocation order, manual intervention can be reduced, which is beneficial to improving the efficiency of screen content processing. Moreover, each task execution tool is used to execute its corresponding task, so that the task execution tool has a high degree of matching with a specific task, which is thus beneficial to improving the professionalism during multitasking processing and the task processing efficiency.

[0083] In some embodiments, the step of generating the dependency relationships between tasks based on the preset screen construction process and the preset screen layout structure may specifically include: determining the first dependency relationship between tasks based on the operations required to be performed for the screen composition content involved in each task; determining the second dependency relationship between tasks according to the dependency relationships between each screen component included in the screen composition content involved in each task; and determining the dependency relationships between tasks by combining the first dependency relationship and the second dependency relationship.

[0084] Exemplarily, from the content of the above embodiments, the first dependency relationship can be understood as the dependency relationship determined based on the logical order of the operations required to construct each screen composition content, which is abbreviated as the logical dependency relationship. For example, it can be used to indicate which tasks need to be executed before other tasks logically. The second dependency relationship can be understood as the dependency relationship determined based on the data flow between each screen component, which is abbreviated as the data dependency relationship. For example, it can be used to indicate which tasks' outputs are the inputs of other tasks.

[0085] In this example, the first dependency relationship and the second dependency relationship can be combined in the following way to obtain the dependency relationships between tasks. For example: create a task dependency graph and visualize the above logical dependency relationship and data dependency relationship in the task dependency graph. For each node in the task dependency graph, any node can represent a task. In the task dependency graph, if there is a directed edge between the nodes corresponding to any two tasks, for example, between the node corresponding to the first task (denoted as the first node) and the node corresponding to the second task (denoted as the second node), there is a directed edge from the first node to the second node, it means that the second task depends on the first task, and the second task can start to execute only after the first task is completed.

[0086] In some scenarios, if there is only a logical dependency between the first task and the second task; or, if there is only a data dependency between the first task and the second task; or, if there is both a logical dependency and a data dependency between the first task and the second task, and the requirements for the task execution order are the same for both, then the task execution order between the first task and the second task is clear, and the dependency between these two tasks can be directly determined.

[0087] In other scenarios, if there is both a logical dependency and a data dependency between the first task and the second task, and if there is a conflict between them, for example, the task execution orders indicated by the logical dependency and the data dependency are inconsistent, then it can be resolved in any of the following ways.

[0088] One way is to resolve the conflict through a predefined priority rule. For example, it is preset that the logical dependency takes precedence over the data dependency. If there is a conflict between them, then the dependency between tasks is determined preferentially according to the logical dependency. Or, it is preset that the data dependency takes precedence over the logical dependency. If there is a conflict between them, then the dependency between tasks is determined preferentially according to the data dependency. Specifically, it can be customized according to actual needs, and the embodiments of the present application do not make specific limitations.

[0089] Another way is to resolve the conflict through user indication. For example, in the process of determining the dependencies between tasks by combining the logical dependency and the data dependency, if the above conflict occurs, the conflict description information, at least one proposed solution, and the interaction elements corresponding to each solution can be displayed through a message pop-up window on the user terminal. In response to the triggering of the interaction element for any proposed solution, the proposed solution selected by the user is obtained, so that the dependency selected by the user can be determined, and the dependency between the corresponding tasks can be determined.

[0090] Specifically, the conflict description information is used to explain the tasks in conflict and the reasons for the conflict. For example, the conflict description information is "There is a dependency conflict between the first task and the second task. The logical dependency requires the first task to be executed before the second task, but the data dependency requires the second task to be executed before the first task".

[0091] Specifically, the proposed solutions are used to provide conflict solutions for the user to choose according to the actual situation. For example: The proposed solutions are "Proposed Solution 1: Give priority to satisfying the logical dependency and ensure that the first task is executed before the second task. Proposed Solution 2: Give priority to satisfying the data dependency and ensure that the second task is executed before the first task".

[0092] Accordingly, the message pop-up window contains a fifth interaction element corresponding to Suggestion Scheme 1 and a sixth interaction element corresponding to Suggestion Scheme 2. In response to the fifth interaction element being triggered, if the obtained suggested solution selected by the user is "Suggestion Scheme 1", the dependency relationship between the two tasks can be determined according to the logical dependency relationship. In response to the sixth interaction element being triggered, if the obtained suggested solution selected by the user is "Suggestion Scheme 2", the dependency relationship between the two tasks can be determined according to the data dependency relationship.

[0093] It should be noted that the fifth interaction element and the sixth interaction element can be options or buttons provided in the message pop-up window, and the embodiments of the present application do not make specific limitations.

[0094] In this embodiment, through the logical dependency relationship and data dependency relationship between the tasks shown in the task dependency relationship diagram, and the solution in the case of conflicts between the logical dependency relationship and data dependency relationship described above, the dependency relationship between the tasks can be determined.

[0095] As an example, there can be multiple implementation manners for determining the execution order of each task according to the dependency relationship between the tasks. For example, the execution order of the tasks can be sorted through the following steps A1 - A6.

[0096] Step A1, create an initially empty queue (which can be called a node queue) and an initially empty list (a node sorted list). Step A2, add all nodes with an in-degree of zero to the node queue. An in-degree of zero means that the task corresponding to the node is an independent task, and the number of tasks on which this task depends is zero. Step A3, obtain a node with an in-degree of zero from the node queue as the current node, remove the current node from the node queue, and add it to the node sorted list. In the node sorted list, the nodes are arranged in the order of addition, and the node sorting represents the sorting of the task execution order. Step A4, update the in-degree of all successor nodes of the current node. Among them, a successor node refers to a node that can be directly reached from the current node through a directed edge. The update of the in-degree means reducing the in-degree of the corresponding successor node. If the in-degree of a certain successor node after update is zero, this successor node can be added to the above node queue. Step A5, repeat the above steps A3 to A4 until the node queue is empty, indicating that all nodes have been processed, and obtain the nodes arranged in sequence in the node sorted list. Step A6, according to the nodes arranged in sequence in the node sorted list, obtain the execution order of the corresponding tasks.

[0097] Through the above steps, it can be ensured that the task corresponding to each node can be executed independently when it is an independent task, and can start execution after all the tasks it depends on have been completed when it is a non-independent task, so as to obtain a task execution order that conforms to the dependency relationship, which is beneficial to ensuring that each task is executed in the correct order.

[0098] For another example, a task management tool can be called to process the dependency relationships between the above-determined tasks to obtain the execution order of each task. The task management tool includes, for example, but is not limited to any one of the following: a task hierarchy management tool (Asana), a project management tool (Zoho Projects), etc. These task management tools have task management and dependency relationship processing functions, and can generate the task execution order according to the dependency relationships between tasks to ensure the smooth execution of tasks. In practical applications, a suitable task management tool can be selected according to actual needs, and the embodiments of the present application do not make specific limitations.

[0099] In this embodiment, the screen content construction process and the screen content layout structure jointly determine the dependency relationships between tasks, and determining the call order of each task according to the dependency relationships between tasks is beneficial to ensuring the correct construction of the display content included in the target screen.

[0100] In some embodiments, the step of calling the task execution tools corresponding to each task according to the call order to execute each task to obtain the screen display content corresponding to the content processing requirement may specifically include: generating a task sequence including each task according to the call order; for each independent task in the task sequence, asynchronously calling each independent task by using the task execution tool corresponding to each independent task, where any independent task is a task that does not depend on other tasks among the tasks; for any non-independent task in the task sequence, obtaining the output result of the task on which the non-independent task depends, and calling the corresponding task execution tool to execute the non-independent task based on the output result; in the case where all tasks in the task sequence have been executed, generating the screen display content corresponding to the content processing requirement according to the output result of the task sequence.

[0101] For ease of understanding, the following combines Figure 2 to describe the call order of each task. Figure 2 FIG. shows a task dependency relationship diagram of an exemplary embodiment of the present application. Figure 2 schematically shows a plurality of tasks, including task A, task B, task C, and task D. The connecting lines with arrows between the tasks indicate that there are dependency relationships between the corresponding tasks. Among them, the task located at the end point of the connecting line depends on the task located at the starting point of the same connecting line. Based on this, in Figure 2 , the execution of task C depends on the outputs of task A and task B, and the execution of task D depends on the output of task C.

[0102] In some embodiments, if one task depends on other tasks, placeholder variables can be used in that one task to represent the outputs of the other dependent tasks. One placeholder variable can correspond to one output. When the other tasks on which that one task depends generate actual output data, the actual output data can be used to replace the corresponding placeholder variables until all the placeholder variables in that one task are replaced by the actual output data corresponding to the outputs.

[0103] Continuing to refer to Figure 2 , Task A and Task B do not depend on other tasks and can be referred to as independent tasks. Each independent task can be asynchronously invoked. Asynchronous invocation means that tasks can be executed concurrently, without being restricted by a specific execution order, and is applicable to scenarios where tasks can be processed in parallel and there are no dependencies between tasks. Asynchronous invocation can ensure the independent execution of tasks. For example, when an independent task is obtained, it can be asynchronously invoked without waiting for the acquisition or execution of any other task, which is beneficial to improving the task execution efficiency.

[0104] Correspondingly, Task C and Task D each depend on other tasks and can be referred to as non - independent tasks. The output results of the other tasks on which the non - independent tasks depend can be used to execute the non - independent tasks.

[0105] For Figure 2 the tasks shown in

[0106] In this embodiment, during the process of invoking the task execution tools corresponding to each task in the call order to execute each task, by asynchronously invoking independent tasks, multiple independent tasks can be executed simultaneously, which is beneficial to improving the overall processing efficiency of each task. Executing non - independent tasks in a specific order based on the task dependency relationship, with a clear task sequence and dependency relationship, makes the processing requirements for the target screen easier to implement, thereby being beneficial to improving the overall performance of task processing.

[0107] In some embodiments, after the step of asynchronously invoking each independent task using the task execution tool corresponding to each independent task, the following steps may further be included: For any independent task, store the output result of the independent task at a first storage address, where the first storage address is a storage address pre-configured for the task execution tool corresponding to the independent task, and different task execution tools are configured with different storage addresses; Obtain the output results of the tasks upon which the non-independent task depends, including: obtain the output results of the tasks upon which the non-independent task depends from a second storage address, where the second storage address is a storage address pre-configured for the task execution tool corresponding to the task upon which the non-independent task depends.

[0108] Exemplarily, different memory address spaces may be allocated for the first storage address and the second storage address in the memory, so that the task execution tools corresponding to each task can access their own memory address spaces, which is conducive to ensuring the isolation and security of the data processed between the task execution tools.

[0109] In this embodiment, each task execution tool is configured with a dedicated memory address space for storing the intermediate results of task processing (different from the final task processing results obtained after all tasks are completed). After a task is completed, the intermediate results will be forwarded as input to other tasks that depend on this task, which is conducive to ensuring the safe and efficient transfer of data between tasks.

[0110] In some embodiments, the composition content of each screen includes at least one of the following: global elements, data visualization components, screen layout structures, and screen style information; where the global elements include the elements that are always displayed on the target screen; the data visualization components are used for data visualization display; the screen style information is used to indicate the visual display style of the target screen.

[0111] Exemplarily, the data visualization components include but are not limited to at least one of components such as charts, tables, lists, etc. For different types of components included in the data visualization components, different data display formats are adopted when displaying the same data content.

[0112] In this embodiment, the composition content of each screen includes the objects to be processed on the target screen. It should be understood that in actual application scenarios, the composition content of each screen may further include more other content, which can be specifically determined according to actual processing requirements. By flexibly processing these composition contents of the screen, the screen display system to which the target screen belongs can maintain its functionality to meet the changing usage requirements.

[0113] In some embodiments, the steps of obtaining the screen display content corresponding to the content processing requirement may specifically include: extracting predetermined characteristic indicators based on the generated task output result, where the predetermined characteristic indicators include: each indicator for measuring whether the task is successfully executed; when at least one of the indicators meets a predetermined condition, determining that the task output result is a successful output result, and the predetermined condition is used to indicate the predetermined value range of each indicator; generating corresponding screen display content based on the successful output result.

[0114] Exemplarily, the extraction of the predetermined characteristic indicators is based on the characteristic indicators extracted by the sample extraction logic. In some scenarios, if the task output result contains all relevant information of the task execution, such as output value, error rate, execution time, etc., then the task output result can be regarded as a sample. Correspondingly, the characteristic indicators may include, for example, at least one of the following: output value, error rate, and execution time.

[0115] Taking the processing instruction "replace the line chart of sales amount in the upper left corner with a bar chart" as an example, the output value may refer to: the result expected to be obtained in the later stage of task execution, that is, the line chart of sales amount in the upper left corner is successfully replaced with a bar chart. If the task is successfully executed, the output value can be expressed as "the bar chart of sales amount has been displayed in the upper left corner". The error rate may refer to: the proportion of errors in the task output result. The value of the error rate can be calculated by the ratio of the number of task execution failures to the total number of task executions. Among them, the number of task execution failures refers to: the number of times when the corresponding output value does not match the expected result, and the total number of task executions is equal to the total number of times the output value is obtained. Assuming that the total number of task executions is 10 and each obtained output value matches the expected result, the error rate is 0%; if 3 of the obtained output values do not match the expected result, the error rate is 3%. It can be seen that the error rate is inversely proportional to the probability of successful task execution. The execution time may refer to the time required for all tasks in each task to be executed to obtain the task output result.

[0116] Exemplarily, the predetermined conditions corresponding to the output value include: the output value matches the result expected to be obtained in the later stage of task execution, where the match means that the output value is equal to the expected result or the semantic similarity between the two is greater than or equal to a predetermined similarity threshold. The predetermined conditions corresponding to the error rate include: the error rate is less than or equal to a predetermined error rate threshold. The predetermined conditions corresponding to the execution time include: the execution time is less than or equal to a predetermined duration threshold.

[0117] It should be understood that the predetermined similarity threshold, the predetermined error rate threshold, and the predetermined duration threshold can be custom-set according to actual needs, and the embodiments of the present application do not make specific limitations.

[0118] In this embodiment, feature metrics can be extracted and utilized based on sample extraction logic, and it can be determined whether the task output result meets the expected goal according to the extracted feature metrics. The feature metrics can reflect the objective attributes of the task output result, which helps to accurately judge whether the output meets the expected goal, thereby improving the accuracy of the evaluation and making the evaluation result more objective and reliable.

[0119] In some embodiments, after extracting the predetermined feature metrics based on the generated task output result, the following steps may further be included: evaluating the task output result using a language model to obtain a model evaluation result; and determining that the task output result is a successful output result when at least one of the metrics meets the predetermined condition and the model evaluation result is a passed evaluation.

[0120] Exemplarily, evaluating the task output result using a language model can be achieved in the following manner. For example, a rule set is pre-constructed, and the predetermined output result in the case of successful task execution is defined in the rule set. Taking the received processing instruction "change the sales line chart in the upper left corner to a bar chart" as an example, if the sales data in the task data result is in the bar chart format, it can be considered that the task has been completed.

[0121] For the bar chart format, for example, the differences between the data in the bar chart format and the data in the line chart format include, but are not limited to, at least one of the following: the bar chart shows the data volume through the height of the bars, while the line chart shows the data change trend through the ups and downs of the lines; the bar chart can be used to display categorical data, and the line chart is suitable for displaying continuous data and time series data.

[0122] In this embodiment, for judging whether the task output result meets the expected goal by combining the sample extraction logic and the model evaluation method, the advanced reasoning capabilities of the sample extraction logic and the language model can be comprehensively utilized to more effectively judge whether the task meets the expected goal, thereby adapting to complex task evaluation requirements and providing a more flexible and comprehensive evaluation result.

[0123] The screen content generation method according to the embodiments of the present application uses a preset recognition model to parse the content processing requirements for the target screen in the form of natural language, which helps to accurately parse the content processing requirements of the operator for the target screen, so as to obtain a task that better meets the processing requirements of the operator and improve the accuracy of the generated screen display content. Moreover, in this solution, the advantage of configuring specific task execution tools for different task types is that: since each task is classified and can be executed by the task execution tool corresponding to the task type, the probability that each task waits for the previous task to complete before it can start execution is greatly reduced during the execution process, which is conducive to reducing the waiting time before task execution and improving the response speed of task execution; each task execution tool has a high degree of matching with a specific type of task, which is conducive to improving the professionalism of task processing. During the processing of this solution, the operator does not need to deeply understand the technical details, and can automatically parse the processing instructions for the target screen and call the corresponding task execution tool to implement the content processing of the target screen without programming knowledge and a dedicated technical team. While reducing labor costs and time costs, it is conducive to efficiently and accurately implementing the content processing requirements of the target screen.

[0124] The embodiments of the present application can provide a faster and more controllable natural language interaction method for building a digital large screen, assist users in creating and modifying the generated large screen, expand the application examples of the model in the construction of data visualization systems, provide a more intelligent data large screen construction and local modification ability for the data visualization industry, and also provide a new solution for the traditional graphical user interface application interaction method. The overall processing flow does not require the model to repeatedly confirm the tasks to be executed, effectively reducing the number of model calls and making the processing process more refined; moreover, the method of the present application can automatically perform task planning and evaluation of task execution results, enhancing the controllability of the generated large screen content; and, by concurrently calling each independent task through the task execution tool, the task execution waiting time under the multi-task tool call can be effectively reduced, improving the task processing efficiency.

[0125] Figure 3 Shows a detailed flowchart of the screen content generation method according to an exemplary embodiment of the present application. As Figure 3 shown, in some embodiments, the screen content generation method includes the following steps.

[0126] Step S301, receiving a user instruction.

[0127] In this step, the user instruction can be at least one of the text or voice form.

[0128] Step S302, creating a task.

[0129] In this step, by using the natural language understanding ability and natural language inference technology of the preset recognition model, the content processing requirements are mined from the historical conversation content and the current user instruction input, and the specific operations to be completed and the corresponding parameters required for the operations (also known as: actions and the parameters required for the actions) are extracted in combination with the preset operation scope, thus completing the creation of a new task.

[0130] Step S303, task planning.

[0131] In this step, after the creation of the new task is completed, all the current tasks can be screened according to the preset screen construction process and the pre-summarized screen layout structure to obtain independent tasks and non-independent tasks.

[0132] Among them, the screen construction process can be a construction process summarized based on the experience of building digital large screens. The screen layout structure can also be referred to as the large screen abstract structure. Independent tasks can be isolated from each other. Isolation means that there is no direct connection between independent tasks, and the progress or result of one task will not affect other tasks. Non-independent tasks are also called associated tasks, that is, tasks with a dependency relationship. Associated tasks can be sorted by priority. For example, based on the dependency relationship, the call order of each task and the execution order of each associated task are determined.

[0133] As an example, for each task that needs to be executed parsed from the content processing requirements, before task planning, it can be understood as an unordered task set. To ensure the accuracy of subsequent task completion, a task sequence and its dependency relationship can be generated according to the fixed order call requirements caused by the dependencies between tasks, forming a directed acyclic graph. Using the pre-summarized ordered process of large screen construction (i.e., the screen construction process) and the abstract structure (i.e., the screen layout structure), the mutual dependency relationships between tasks are identified. If a non-independent task depends on another task, a placeholder variable can be used to replace the output of the other task, and the actual output of the other task can be used to replace the placeholder variable later. After the above task planning, the corresponding actions of each task to be completed can be added to the action group in an orderly manner waiting to be called. The corresponding actions of each task are at least one operation required to execute each task.

[0134] Step S304, generate a task queue.

[0135] In this step, after determining the call order of each task through the mutual dependency relationships between tasks, a corresponding task queue can be generated according to this call order. Through this task queue, it can be ensured that the tasks that are depended on are executed first, and then the tasks that depend on other tasks are executed, ensuring that each task can proceed smoothly and efficiently.

[0136] Step S305, concurrently call the task execution tool to execute each task.

[0137] In this step, tasks in the task queue are obtained in sequence, and the task execution tool corresponding to the obtained task is used to execute the task. After all tasks are executed, screen display content corresponding to the content processing requirement is generated according to the task output result. Specifically, according to the mutual dependencies of the tasks, independent tasks can be asynchronously and concurrently called.

[0138] Exemplarily, according to the following four types included in the composition of the target screen: global elements, data visualization components (such as charts and their display content), screen layout structure (i.e., large screen structure), and screen style information (i.e., large screen style), and each is equipped with its corresponding intelligent agent tool, i.e., task execution tool, for the execution of each task.

[0139] In some scenarios, each intelligent agent tool can be configured with a dedicated memory address space to store the output result of the current task, and the output result of this task can be forwarded as an intermediate result to other tasks that depend on this task.

[0140] Exemplarily, in Figure 3 it, the task execution tool for modifying global elements can be named the large screen global element modification intelligent agent. The task execution tool for modifying charts and their display content can be named the large screen chart recommendation intelligent agent; the task execution tool for modifying the screen layout structure can be named the large screen layout intelligent agent; and the task execution tool for modifying screen style information can be named the stylization intelligent agent.

[0141] In this example, the large screen global element modification intelligent agent can modify the global elements in the target screen. The large screen chart recommendation intelligent agent can be used to execute the view display task. In the data visualization process of the target screen, the view can include but is not limited to data display forms such as charts, tables, and lists. The stylization intelligent agent can be used to modify the style information in the target screen. It should be understood that in the description of the embodiments of the present application, the view node can be understood as a screen component.

[0142] In this example, when the large screen layout intelligent agent modifies the screen layout, it can determine the layout method of the screen components through the layout tree. Specifically, the layout tree includes multiple view nodes. A view node can represent the chart type of a chart view and the size information of the chart view in the target screen. The multiple view nodes are distributed at multiple levels; mapping the multiple view nodes in the layout tree to different positions in the target screen can determine the layout method of the chart view in the target screen.

[0143] Step S306, result evaluation.

[0144] In this step, feature indicators can be extracted and utilized based on the sample extraction logic, and it can be determined whether the task output result meets the expected goal according to the extracted feature indicators. Refer to Figure 3 , in some embodiments, if it is determined that the task output result does not meet the expected goal, step S303 can be returned, that is, the task planning is re-executed.

[0145] Exemplarily, during the process of re-executing the task planning, in response to the new user instruction received, the operations to be modified and / or the parameters required to modify at least one operation can be obtained from the new user instruction. According to the modified operations and / or the parameters required for each modified operation, at least one new task is created, and then task planning is performed on each new task.

[0146] Step S306, construct a large screen.

[0147] In this step, the task output result can be used to construct the target screen according to the abstract syntax. The abstract syntax is a way to describe the code structure in a programming language. Using the abstract syntax, the layout method of the screen components can be determined through the above layout tree, so as to obtain the corresponding large screen layout structure.

[0148] Through the above steps S301 - S306, the natural language understanding and reasoning technology based on the preset recognition model can be utilized, and combined with the calling ability of the task execution tool, so that the end user can directly command the configuration or content change of the chart components on the digital large screen in an oral way. According to this method, the end user does not need to deeply understand the technical details. Without programming knowledge and a dedicated technical team, the processing instructions for the target screen can be automatically parsed and the corresponding task execution tool can be called to implement the content processing of the target screen, which reduces the labor cost and time cost, and is conducive to efficiently and accurately realizing the content processing requirements of the target screen.

[0149] In the embodiments of the present application, when interacting with a preset recognition model, specific functions or instructions can be called to make the model execute processing instructions or provide the required information. Utilize the capabilities of the preset recognition model to solve complex problems or obtain customized responses. According to the method of the embodiments of the present application, by using the natural language processing technology of the preset recognition model and combining the ability of agent tool invocation, a new agent tool invocation framework is proposed and developed, and an overall process from intent recognition, task planning and sorting, to concurrent invocation of task execution tools is built, which is beneficial to reducing the learning cost of users for professional technologies and enabling users to easily manage and quickly update the configuration content of digital dashboards. In this solution, through the preset recognition model agent, the user requirements in a complex context can be understood to propose task requirements, and then the processing flow of each task can be automatically planned through the processing of task planning and sorting, and multiple task execution tools can be coordinated to perform concurrent execution of each independent task to ensure the efficient completion of tasks.

[0150] In the agent tool invocation framework according to the embodiments of the present application, the model agent and the task execution tool agent included can manage the digital dashboard content more intelligently. Non-technical users can easily update and customize the digital dashboard content without programming knowledge, greatly reducing the operation cost and improving the timeliness and personalization level of content updates.

[0151] Figure 4 The flowchart of the screen content display method according to the embodiments of the present application is shown, as Figure 4 shown, this method may include the following steps S401 - S404.

[0152] S401, Receive a display instruction for the target screen.

[0153] S402, In response to the display instruction, display the current screen display content of the target screen.

[0154] S403, Receive a processing instruction for the target screen, where the processing instruction is used to indicate the content processing requirement for the target screen through natural language.

[0155] S404, In response to the processing instruction, display new screen display content, where the new screen display content is the screen display content corresponding to the content processing requirement obtained by processing the processing instruction according to the screen content generation method of the embodiments of the present application.

[0156] According to the screen content display method of the embodiments of the present application, it can respond to the received display instruction for the target screen to display the screen display content, and can respond to the received processing instruction for the target screen to display the new screen display content, where the new screen display content is the screen display content corresponding to the content processing requirement obtained by processing the processing instruction according to any of the screen content generation methods of the above embodiments of the present application. Through this method, the content of the target screen can be dynamically displayed according to the received display instruction, and the new screen display content corresponding to the content processing requirement can be generated in response to the processing instruction, so that real-time processing can be flexibly performed according to the instruction.

[0157] For the screen content generation method in the embodiments of the present application, reference can be made to the corresponding description in the screen content generation method described in combination with the above embodiments Figures 1 - 3 and has the corresponding beneficial effects, which will not be elaborated here.

[0158] Corresponding to the application scenario and method of the method provided by the embodiments of the present application, the embodiments of the present application also provide a screen content generation device.

[0159] Figure 5 The structural schematic diagram of a screen content generation device according to an embodiment of the present application is shown. This device is used to execute the screen content generation method provided in any of the above embodiments, as Figure 5 shown, this screen content generation device includes:

[0160] A receiving module 510 that receives a processing instruction, where the processing instruction is used to indicate the content processing requirement of the target screen through natural language.

[0161] An analysis module 520 that, in response to the processing instruction, uses a preset recognition model to analyze and obtain each task that needs to be executed from the content processing requirement.

[0162] An execution module 530 that, based on the task type to which each task belongs, calls a task execution tool corresponding to the task type to execute the corresponding task, and obtains the screen display content corresponding to the content processing requirement.

[0163] In some embodiments, when the analysis module 520 is used to analyze and obtain each task that needs to be executed from the content processing requirement by using a preset recognition model, it is specifically used for: performing intention recognition by using a preset recognition model based on the pre-stored historical session content and the content processing requirement; determining each operation that needs to be executed and the parameters required for each operation according to the intention recognition result; creating at least one task according to each operation and parameter, where the task is used to execute the corresponding operation based on at least one parameter.

[0164] In some embodiments, when the parsing module 520 is used to determine each operation to be performed and the parameters required for each operation according to the intention recognition result, it is specifically configured to: obtain a preset operation scope, where the operation scope is used to indicate at least one operation to be performed on each screen component included in the target screen; based on the intention recognition result, extract each operation to be performed from the operation scope, and determine the parameters required for each operation.

[0165] In some embodiments, when the execution module 530 is used to call a task execution tool corresponding to the task type to execute the corresponding task and obtain the screen display content corresponding to the content processing requirement, it is specifically configured to: generate the dependency relationship between each task based on the preset screen construction process and the preset screen layout structure, where the screen construction process includes the operations required to construct each screen component included in the target screen, and the operations required to be performed include each step required to generate the corresponding screen component, and the screen layout structure is used to indicate the layout mode of the target screen; determine the call order of each task based on the dependency relationship; according to the call order, call the task execution tools corresponding to each task to execute each task, and obtain the screen display content corresponding to the content processing requirement.

[0166] In some embodiments, when the execution module 530 is used to generate the dependency relationship between each task based on the preset screen construction process and the preset screen layout structure, it is specifically configured to: determine the first dependency relationship between each task based on the operations required to be performed on the screen components involved in each task; determine the second dependency relationship between each task according to the dependency relationship between each screen component included in the screen components involved in each task; combine the first dependency relationship and the second dependency relationship to determine the dependency relationship between each task.

[0167] In some embodiments, when the execution module 530 is used to, according to the call order, call the task execution tools corresponding to each task to execute each task and obtain the screen display content corresponding to the content processing requirement, it is specifically configured to: generate a task sequence including each task according to the call order; for each independent task in the task sequence, asynchronously call each independent task by using the task execution tool corresponding to each independent task, where any independent task is a task that does not depend on other tasks among each task; for any non-independent task in the task sequence, obtain the output result of the task on which the non-independent task depends, and call the corresponding task execution tool to execute the non-independent task based on the output result; when all the tasks in the task sequence are executed, generate the screen display content corresponding to the content processing requirement according to the output result of the task sequence.

[0168] In some embodiments, the screen content generation device further includes a storage module, which is configured to, after asynchronously invoking each independent task by using the task execution tool corresponding to each independent task, for any independent task, store the output result of the independent task to a first storage address, where the first storage address is a storage address pre-configured for the task execution tool corresponding to the independent task, and different task execution tools are configured with different storage addresses; when the execution module 530 is used to obtain the output result of the task on which the non-independent task depends, it is specifically configured to: obtain the output result of the task on which the non-independent task depends from a second storage address, where the second storage address is a storage address pre-configured for the task execution tool corresponding to the task on which the non-independent task depends.

[0169] In some embodiments, each screen composition content includes at least one of the following: global elements, data visualization components, screen layout structures, and screen style information; where the global elements include the elements that are always displayed on the target screen; the data visualization components are used for data visualization display; and the screen style information is used to indicate the visual display style of the target screen.

[0170] In some embodiments, when the execution module 530 is used to obtain the screen display content corresponding to the content processing requirement, it is specifically configured to: extract predetermined characteristic indicators based on the generated task output result, where the predetermined characteristic indicators include: each indicator used to measure whether the task is executed successfully; in the case where at least one of the indicators meets a predetermined condition, determine the task output result as a successful output result, and the predetermined condition is used to indicate the predetermined value range of each indicator; generate the corresponding screen display content based on the successful output result.

[0171] The functions of the modules in each device in the embodiments of the present application can be referred to the corresponding descriptions in the above methods, and have the corresponding beneficial effects, which will not be elaborated here.

[0172] Figure 6 The structural schematic diagram of a screen content display device according to an embodiment of the present application is shown, and the device is used to execute the screen content display method provided in any of the above embodiments, such as Figure 6 As shown, the screen content display device includes:

[0173] A receiving module 610, which receives a display instruction for a target screen.

[0174] A display module 620, which, in response to the display instruction, displays the current screen display content of the target screen.

[0175] The receiving module 610 is further configured to receive a processing instruction for the target screen, and the processing instruction is used to indicate the content processing requirement for the target screen through natural language.

[0176] The display module 620 is further configured to display new screen display content in response to a processing instruction, where the new screen display content is the screen display content corresponding to the content processing requirement obtained by processing the processing instruction according to any one of the above screen content generation methods.

[0177] For the functions of the modules in each device of the embodiments of the present application, reference may be made to the corresponding descriptions in the above methods, and they have the corresponding beneficial effects, which will not be elaborated here.

[0178] Figure 7 The block diagram of the electronic device provided by the embodiments of the present application is shown. As Figure 7 shown, the electronic device includes: a memory 701 and a processor 702, and a computer program that can run on the processor 702 is stored in the memory 701. When the processor 702 executes the computer program, the method in the above embodiments is implemented. The number of the memory 701 and the processor 702 can be one or more. In a specific implementation, the electronic device may further include a communication interface 703 for communicating with external devices and performing data interaction and transmission.

[0179] In a specific implementation, if the memory 701, the processor 702, and the communication interface 703 are independently implemented, the memory 701, the processor 702, and the communication interface 703 can be interconnected through a bus and communicate with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 7 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0180] Optionally, in a specific implementation, if the memory 701, the processor 702, and the communication interface 703 are integrated on a chip, the memory 701, the processor 702, and the communication interface 703 can communicate with each other through an internal interface.

[0181] The embodiments of the present application provide a computer-readable storage medium that stores a computer program, and when the program is executed by a processor, any one of the methods provided in the embodiments of the present application is implemented.

[0182] The embodiments of the present application provide a computer program product, including a computer program, and when the program is executed by a processor, any one of the methods provided in the embodiments of the present application is implemented.

[0183] An embodiment of the present application further provides a chip, which includes a processor for calling and running instructions stored in a memory, so that a communication device equipped with the chip executes any method provided by the embodiment of the present application.

[0184] An embodiment of the present application further provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, the output interface, the processor, and the memory are connected through an internal connection path. The processor is configured to execute code in the memory. When the code is executed, the processor is configured to execute any method provided by the embodiment of the application.

[0185] It should be understood that the above-mentioned processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the advanced risc machines (ARM) architecture.

[0186] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DR RAM).

[0187] In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions in accordance with the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0188] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0189] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In the description of the present application, "a plurality of" means two or more unless otherwise specifically defined.

[0190] Any process or method described in the flowchart or otherwise described herein can be understood to represent a module, segment, or portion of code including one or more executable instructions for implementing a specific logical function or process. And the scope of the preferred embodiments of the present application includes additional implementations, where the functions can be executed in a substantially simultaneous manner or in a reverse order according to the involved functions, rather than in the order shown or discussed.

[0191] The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing a logical function, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or in connection with such instruction execution systems, apparatus, or devices.

[0192] It should be understood that each part of the present application can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the method in the above embodiments can be completed by a program instructing relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0193] In addition, each functional unit in various embodiments of the present application may be integrated into one processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The storage medium may be a read-only memory, a magnetic disk, an optical disc, or the like.

[0194] As mentioned above, the above are only exemplary embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope recorded in the present application can easily think of various changes or substitutions thereof, and these should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for generating screen content, characterized in that: include: receiving a processing instruction, wherein the processing instruction is used to indicate a content processing requirement of a target screen through natural language; In response to the processing instruction, the preset recognition model is used to parse the content processing requirements to obtain tasks that need to be performed; The execution order of each task is determined by a task dependency relationship, wherein the task dependency relationship includes: a dependency relationship determined based on a predetermined priority order of the first dependency relationship and the second dependency relationship when the task execution orders indicated by the first dependency relationship and the second dependency relationship are inconsistent; The first dependency relationship is used to indicate: the logical order of operations required to construct each screen component content; each screen component content includes a screen layout structure, and the screen layout structure is used to indicate the layout mode of the target screen; the second dependency relationship is used to indicate data transfer between each screen component; The task execution tool is called to execute the corresponding task, and the screen display content corresponding to the content processing requirement is obtained.

2. The method according to claim 1, characterized in that The using of a preset recognition model to parse the content processing requirements to obtain the tasks to be performed includes: Based on the pre-stored historical conversation content and the content processing requirements, the preset recognition model is used to perform intent recognition, wherein the historical conversation content is used to indicate the pre-collected conversation history related to the content processing of the target screen; Determine each operation to be performed and parameters required for each operation according to the intention recognition result; At least one task is created according to the operations and the parameters, and the task is used to perform the corresponding operation based on at least one parameter.

3. The method according to claim 2, characterized in that The determining of each operation to be performed and the parameters required for each operation according to the intention recognition result includes: Acquire a preset operation range, where the operation range is used to indicate at least one operation to be performed on each screen component content included in the target screen; Based on the intention recognition result, each operation to be performed is extracted from the operation range, and parameters required for each operation are determined.

4. The method according to claim 1, characterized in that: The calling of the task execution tool to execute the corresponding task and obtain the screen display content corresponding to the content processing requirement includes: Based on a preset screen construction process and a preset screen layout structure, the dependency relationship between the tasks is generated, the screen construction process includes operations required to be performed to construct each screen component content included in the target screen, the operations required to be performed include each step required to be performed in the process of generating the corresponding screen component content, and the screen layout structure is used to indicate the layout mode of the target screen; Based on the dependency relationship, determining the calling order of the tasks; According to the calling sequence, the task execution tools corresponding to the tasks are called to execute the tasks, so as to obtain the screen display content corresponding to the content processing requirement.

5. The method according to claim 4, characterized in that The generating of the dependency relationship between the tasks based on the preset screen building process and the preset screen layout structure includes: Determining a first dependency relationship between the tasks based on operations required to be performed for screen composition contents involved in the tasks; Determining a second dependency relationship between the tasks according to a dependency relationship between the screen components included in the screen composition content involved in the tasks; Dependencies between the tasks are determined by combining the first dependency and the second dependency.

6. The method according to claim 4, characterized in that The calling of the task execution tools corresponding to the tasks respectively according to the calling sequence to execute the tasks and obtain the screen display content corresponding to the content processing requirements includes: Generate a task sequence including the tasks according to the calling order; For each independent task in the task sequence, asynchronously call each independent task using the task execution tool corresponding to each independent task, and any independent task is a task that does not depend on other tasks among the tasks; For any dependent task in the task sequence, obtaining an output result of a task on which the dependent task depends, and calling a corresponding task execution tool to execute the dependent task based on the output result; When all the tasks in the task sequence are completed, screen display content corresponding to the content processing requirement is generated according to the output results of the task sequence.

7. The method according to claim 6, characterized in that After the task execution tools corresponding to the independent tasks are used to asynchronously call the independent tasks, the method further includes: For any independent task, the output result of the independent task is stored in a first storage address, where the first storage address is a storage address pre-configured for a task execution tool corresponding to the independent task, and different task execution tools are configured with different storage addresses; The obtaining of the output result of the task on which the non-independent task depends includes: obtaining the output result of the task on which the non-independent task depends from a second storage address, wherein the second storage address is a storage address pre-configured for a task execution tool corresponding to the task on which the non-independent task depends.

8. The method according to claim 4, characterized in that The contents of each screen also include at least one of the following: global elements, data visualization components and screen style information; wherein the global elements include elements that are always displayed in the target screen; the data visualization components are used for data visualization display; and the screen style information is used to indicate the visual display style of the target screen.

9. The method according to claim 1, characterized in that: The obtaining of the screen display content corresponding to the content processing requirement includes: Extracting predetermined characteristic indicators based on the generated task output result, wherein the predetermined characteristic indicators include: indicators for measuring whether the task is successfully executed; When at least one of the indicators satisfies a predetermined condition, determining that the task output result is a successful output result, wherein the predetermined condition is used to indicate a predetermined value range of the indicators; Generate corresponding screen display content based on the successful output result.

10. A method for displaying screen content, characterized in that: Applied to the target screen, the method comprises: receiving a display instruction for the target screen; In response to the display instruction, display the current screen display content of the target screen; receiving a processing instruction for the target screen, wherein the processing instruction is used to indicate a content processing requirement for the target screen through natural language; In response to the processing instruction, new screen display content is displayed, where the new screen display content is the screen display content corresponding to the content processing requirement obtained by processing the processing instruction according to the method described in any one of claims 1-9.

11. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the method of any one of claims 1 to 9 or claim 10 when executing the computer program.

12. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 or claim 10 is implemented.

13. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9 or claim 10.

Images