Screen content generation method, display method, equipment, medium and program product

Abstract

The embodiment of the invention provides a screen content generation method, a display method, equipment, a medium and a program product, and relates to the technical field of computers, the method comprises the following steps: receiving a processing instruction, the processing instruction being used for indicating a content processing demand of a target screen through a natural language; in response to the processing instruction, analyzing the content processing demand by using a preset identification model to obtain each task needing to be executed; and based on the task type to which each task belongs, calling a task execution tool corresponding to the task type to execute the corresponding task, and obtaining screen display content corresponding to the content processing demand. According to the technical scheme, the processing instruction of the target screen can be automatically analyzed, and the corresponding task execution tool can be called to process the content of the target screen, so that the labor cost and the time cost can be reduced, and the content processing requirement of the target screen can be efficiently and accurately realized.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a screen content generation method, display method, device, medium and program product. Background Art

[0002] Digital large screen is a data visualization technology that displays various data information such as sales data and implementation management information on a large screen through a graphical interface. Digital large screen can provide intuitive and real-time data display to help users better understand and manage complex data information. Currently, when building digital large screen content, operators are usually required to have certain programming knowledge or rely on a dedicated technical team to complete it. The entire construction process will consume a lot of manpower and time costs. Summary of the invention

[0003] The embodiments of the present application provide a screen content generation method, display method, device, medium and program product to alleviate or solve one or more technical problems existing in the prior art.

[0004] In a first aspect, an embodiment of the present application provides a method for generating screen content, the method comprising: receiving a processing instruction, the processing instruction being used to indicate a content processing requirement of a target screen through natural language; in response to the processing instruction, using a preset recognition model to parse the content processing requirement to obtain each task that needs to be performed; based on the task type to which each task belongs, calling a task execution tool corresponding to the task type to execute the corresponding task, and obtaining the screen display content corresponding to the content processing requirement.

[0005] In a second aspect, an embodiment of the present application provides a screen content display method, which is applied to a target screen, the method comprising: receiving a display instruction for the target screen; in response to the display instruction, displaying the current screen display content of the target screen; receiving a processing instruction for the target screen, the processing instruction being used to indicate content processing requirements for the target screen through natural language; in response to the processing instruction, displaying new screen display content, the new screen display content being screen display content corresponding to the content processing requirements obtained by processing the processing instruction according to the screen content generation method of the first aspect.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, and the processor implements any method of the embodiment of the present application when executing the computer program.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method of any one of the embodiments of the present application is implemented.

[0008] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements any method of the embodiments of the present application when executed by a processor.

[0009] According to the screen content generation method of the embodiment of the present application, the content processing requirements for the target screen indicated by natural language can be obtained according to the received processing instructions, and the tasks required to implement the content processing requirements can be parsed from the content processing requirements using a preset recognition model, and the corresponding tasks can be executed through a task execution tool to obtain the screen display content corresponding to the content processing requirements.

[0010] In this technical solution, the preset recognition model has the ability of natural language understanding and reasoning, which is conducive to accurately analyzing the operator's content processing requirements for the target screen, so as to obtain tasks that better meet the operator's processing requirements and improve the accuracy of the generated screen display content. In addition, in this solution, the advantage of configuring specific task execution tools for different task types is that: since each task is classified and can be executed by the task execution tool corresponding to the task type, the probability of waiting for the previous task to be completed before starting to execute each task during the execution process is greatly reduced, which is conducive to reducing the waiting time before task execution and improving the response speed of task execution; each task execution tool has a high degree of matching with a specific type of task, which is conducive to improving the professionalism of task processing. In the processing process of this solution, the operator does not need to have a deep understanding of the technical details. Without programming knowledge and a dedicated technical team, the processing instructions for the target screen can also be automatically parsed and the corresponding task execution tool can be called to realize the content processing of the target screen, which is conducive to efficiently and accurately realizing the content processing requirements of the target screen while reducing labor costs and time costs.

[0011] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments according to the present application and should not be regarded as limiting the scope of the present application.

[0013] Figure 1 A flowchart of a method for generating screen content according to an embodiment of the present application is shown; Figure 2 A task dependency diagram of an exemplary embodiment of the present application is shown; Figure 3 A detailed flow chart of a method for generating screen content according to an exemplary embodiment of the present application is shown; Figure 4 A flow chart showing a method for displaying screen content according to an embodiment of the present application is shown; Figure 5 A schematic diagram showing the structure of a screen content generating device according to an embodiment of the present application is shown; Figure 6 A schematic structural diagram of a screen content display device according to an embodiment of the present application is shown; Figure 7 A block diagram of an electronic device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0014] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the concept or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary in nature and not restrictive.

[0015] To facilitate understanding of the technical solutions of the embodiments of the present application, the following describes the related technologies of the embodiments of the present application. The following related technologies can be combined with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

[0016] In actual application scenarios, with the acceleration of digital transformation, digital large screens, as an important carrier of information display, play a key role in business analysis, real-time management, data visualization and other fields. However, when building digital large screen content, operators are usually required to have certain programming knowledge or rely on a dedicated technical team to complete it. Therefore, the current digital large screen screen content processing process is very dependent on programming skills or professional technical teams, which not only increases the operating threshold, but also seriously restricts the participation of non-technical background users and the efficiency of content updates.

[0017] In some related technologies, data visualization can be provided through business intelligence and analysis software (Tableau), but the software is mainly aimed at users with a certain technical background and does not support direct modification of charts or layouts through natural language interaction. In related technologies, charts are also made through data analysis tools (Power BI), but its natural language query function is mainly used for data retrieval rather than interface content processing. Also, report generation is performed through enterprise-level data analysis applications (Looker). Although this software has a simple natural language search option, custom requirements for visual presentation still need to be implemented by writing code.

[0018] In other related technologies, voice assistants can be used to control digital large screens to view weather forecasts and play music through voice commands, but their functions are mainly used for data retrieval and cannot meet the configuration requirements of data visualization projects. In addition, there are some graphical user interface (GUI) applications that can create dashboard visualization systems by dragging elements, but this method often requires users to understand basic knowledge such as HyperText Markup Language HTML and Cascading Style Sheets CSS, which is a big challenge for people who do not understand professional technology. It should be noted that the above application scenarios or application examples provided in the embodiments of the present application are for ease of understanding, and the embodiments of the present application do not specifically limit the application of the technical solution. In addition, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0019] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The several specific embodiments listed can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0020] Figure 1 A flow chart of a method for generating screen content according to an embodiment of the present application is shown. Figure 1 As shown, the method may include the following steps S101, S102 and S103.

[0021] Step S101 : receiving a processing instruction, where the processing instruction is used to indicate content processing requirements of a target screen through natural language.

[0022] Step S102, in response to the processing instruction, the preset recognition model is used to parse the content processing requirements to obtain the tasks that need to be executed.

[0023] Step S103: based on the task type to which each task belongs, calling the task execution tool corresponding to the task type to execute the corresponding task, and obtaining the screen display content corresponding to the content processing requirement.

[0024] According to the screen content generation method of the embodiment of the present application, the content processing requirements for the target screen indicated by natural language can be obtained according to the received processing instructions, and the tasks required to implement the content processing requirements can be parsed from the content processing requirements using a preset recognition model, and the corresponding tasks can be executed through a task execution tool to obtain the screen display content corresponding to the content processing requirements.

[0025] In this technical solution, the use of a preset recognition model helps to accurately analyze the operator's content processing requirements for the target screen, so as to obtain tasks that better meet the operator's processing requirements and improve the accuracy of the generated screen display content. In addition, in this solution, the advantage of configuring specific task execution tools for different task types is that: since each task is classified and can be executed by the task execution tool corresponding to the task type, the probability of each task waiting for the previous task to be completed before it can be started during execution is greatly reduced, which is conducive to reducing the waiting time before task execution and improving the response speed of task execution; each task execution tool has a high degree of matching with a specific type of task, which is conducive to improving the professionalism of task processing. In the processing process of this solution, the operator does not need to have a deep understanding of the technical details. Without programming knowledge and a dedicated technical team, the operator can also automatically analyze the processing instructions for the target screen and call the corresponding task execution tool to realize the content processing of the target screen, which is conducive to efficiently and accurately realizing the content processing requirements of the target screen while reducing labor costs and time costs.

[0026] In step S101, the processing instruction is an instruction described in natural language. For example, natural language can be understood as a naturally generated language used for communication between people. For example, languages ​​of different countries such as Chinese, English, French, etc. can also be dialects of a certain language, such as Cantonese, Minnan dialect, etc. This application embodiment is not specifically limited.

[0027] Exemplarily, the processing instruction may be in at least one of text and voice form.

[0028] As a specific example, the user terminal may include a first interactive element, which is used to trigger the receipt of a processing instruction in text form. In response to the operation instruction of the first interactive element, the text input processing instruction is received through a text input device. The first interactive element includes but is not limited to: a text input box, a drop-down list, a radio button or a check box, etc. The text input device includes but is not limited to at least one of the following: a keyboard, a mouse, a handwriting tablet, a touch screen, etc.

[0029] As a specific example, the target screen may also include a second interactive element, which is used to trigger a processing instruction for receiving a voice form. In response to an operation instruction on the second interactive element, a processing instruction for collecting user voice input through an audio acquisition device mounted on the electronic device to which the target screen belongs is triggered. Among them, the second interactive element includes but is not limited to at least one of the following: a button, a menu item, etc. The operation instructions for the first interactive element and the second interactive element may be instructions triggered by operations such as a click operation and a selection operation. The audio acquisition device includes but is not limited to a microphone unit. The microphone unit, for example, includes a single microphone or a microphone array, and the number of microphones in the microphone array is at least 2.

[0030] Exemplarily, content processing requirements are used to define the target for processing the target screen. The types of content processing requirements may include at least one of creation, modification, and update. For example, when the processing instruction is "add a pie chart to show the proportion of each product category", it indicates a content processing requirement of the creation type (create a new chart). For example, when the processing instruction is "change the sales line chart in the upper left corner to a bar chart", it indicates a content processing requirement of the modification type (modify the existing chart). For another example, when the processing instruction is "update the sales statistics chart for the fourth quarter", it indicates an update type of content processing requirement (update chart data). For another example, when the processing instruction is "update the sales statistics chart for the fourth quarter and add a bar chart to show the monthly sales trend", the indicated content processing requirements include two types of creation and update (update chart data and create new charts).

[0031] It should be understood that the above-mentioned content processing requirements are merely schematic illustrations, and the specific content of the processing instructions can be customized according to the actual processing requirements of the target screen, and the embodiments of the present application do not make specific limitations.

[0032] In step S102, the preset recognition model is a model with natural language understanding and natural language reasoning capabilities. Exemplarily, the preset recognition model includes but is not limited to any of the following models: Large Language Models (LLM), intent recognition based on bidirectional encoder representations from transformers and semantic slot filling joint model.

[0033] The big language model is referred to as the big model. Based on the natural language understanding ability and natural language inference (NLI) technology of the big model, the specific content processing requirements are parsed from the processing instructions in the form of natural language, and tasks that better meet the processing requirements of the operator are obtained, thereby improving the accuracy of the generated screen display content.

[0034] Among them, the joint model of intent recognition and semantic slot filling based on the bidirectional encoder can encode the content processing requirements in natural language form through the pre-trained bidirectional encoder model, and obtain the contextual representation of each word to capture the contextual information of the content processing requirements, so as to more accurately recognize the intent, and then construct a specific task based on the intent recognition results and the structured information extracted by slot filling.

[0035] It should be understood that the preset recognition model can also be other neural network models based on deep learning. The model has natural language understanding and reasoning capabilities, and can parse the operator's content processing requirements for the target screen based on the content processing requirements in the form of input natural language to obtain the tasks that need to be performed. The embodiment of the present application does not specifically limit the implementation form of the preset recognition model.

[0036] Specifically, natural language understanding includes but is not limited to semantic analysis and intent recognition of content processing requirements using preset recognition models. Semantic analysis is used to understand the actual information conveyed in content processing requirements, and intent recognition is used to identify the user's true intent conveyed in content processing requirements. Natural language reasoning can be understood as the process of using preset recognition models and natural language processing algorithms to understand and infer the logical relationships and meanings in natural language texts, thereby making reasonable judgments and conclusions. Usually, this logical reasoning involves not only understanding the literal meaning of processing instructions, but also inferring implicit meanings and intentions.

[0037] In an embodiment of the present application, a preset recognition model is used to parse the content processing requirements in the natural language form for the target screen, which helps to understand the user's true intention in the content processing requirements. Based on the accurate understanding of the user's intention, the tasks that need to be performed can be accurately parsed from the content processing requirements, thereby helping to perform the tasks more accurately and efficiently and obtain more accurate content processing results.

[0038] In some embodiments, the step of using a preset recognition model to parse the content processing requirements to obtain the tasks that need to be performed in step S102 may specifically include: based on pre-stored historical session content and content processing requirements, using a preset recognition model to perform intent recognition; determining the operations that need to be performed and the parameters required for each operation based on the intent recognition results; creating at least one task based on each operation and parameter, and the task is used to perform the corresponding operation based on at least one parameter.

[0039] Exemplarily, historical conversation content is used to indicate pre-collected conversation history related to content processing of the target screen. For example, historical conversation content may include, but is not limited to, at least one of the following: historical processing instructions, feedback and evaluation of historical screen display content. Based on historical conversation content and content processing requirements, the preset recognition model can be helped to more accurately identify intent. For example, if the information on the content processing requirements indicated by the processing instructions is too simple or incomplete, the preset recognition model can be based on the content processing requirements and combined with the historical conversation content to more accurately identify intent.

[0040] Assume that the received processing instruction is "update the sales chart", the content processing requirement indicated by this processing instruction does not provide more detailed information such as update frequency, chart type, etc. The historical session content includes historical processing instructions, and historical processing instructions include, for example, "update the sales chart of each product on a monthly basis", "update the sales chart in the upper left corner to a bar chart", etc. In this way, the preset recognition model can use these historical session contents to improve and update the content processing requirements. Exemplarily, the updated content processing requirements may be, for example: "Update the sales chart of each product in the upper left corner on a monthly basis, and display the specific sales data of each product category in the form of a bar chart."

[0041] In some embodiments, after the above-mentioned step of performing intent recognition using a preset recognition model based on pre-stored historical conversation content and content processing requirements, it may also include: displaying a predetermined interface on the target screen, the display content of the predetermined interface includes the intent recognition result, the third interactive element and the fourth interactive element, the third interactive element is used to trigger the generation of an approval message for the intent recognition result, and the fourth interactive element is used to trigger the generation of a rejection message for the intent recognition result. In response to the third interactive element being triggered, the following steps are performed: determining the operations to be performed and the parameters required for each operation based on the intent recognition result. In response to the fourth interactive element being triggered, triggering the reception of user feedback information, and generating new content processing requirements based on the user feedback information.

[0042] As an example, user feedback information may be, for example, a new processing instruction or evaluation information of the intent recognition result. Assuming that the user feedback information is "No, what I want is quarterly updates", the new content processing requirement may be, for example, "Update the sales chart of each product in the upper left corner quarterly, and display the specific sales data of each product category in the form of a bar chart". It should be understood that the third interactive element and the fourth interactive element may be options or buttons provided by a predetermined interface. The operation instructions for the third interactive element and the fourth interactive element may be instructions triggered by operations such as clicking operations and selecting operations. The predetermined interface may be in the form of a pop-up window or a page. The embodiments of the present application are not specifically limited.

[0043] Exemplarily, in the step of determining the operations to be performed and the parameters required for each operation according to the intention recognition result, the parameters required for each operation can be understood as the input information required to perform each operation, and these parameters are used to indicate the specific execution method of each operation. For example, for the processing instruction "change the sales line chart in the upper left corner to a bar chart", based on the historical session content and the content processing requirements indicated by the processing instruction, after performing intent recognition using a preset recognition model, the specific operations to be performed and the parameters required for the specific operations can be determined according to the intention recognition result. For example, specific operations include: changing the chart type. The parameters required for the specific operation include: chart position, current chart name, target chart type, etc. Specifically, the chart position is "upper left corner", the current chart type is "sales line chart", and the target chart type is "bar chart".

[0044] Exemplarily, at least one task is created based on the specific operation and the parameters required for the operation. These tasks may include, for example: generating a target position based on the position coordinates of the upper left corner of the sales line chart; when confirming that the chart type is a line chart, extracting sales data used to generate the line chart from a predetermined data source; generating a bar chart based on the sales data; removing the sales line chart at the target position; and rendering a bar chart of the sales data at the target position.

[0045] For example, one operation may correspond to one task, or multiple operations may be integrated into one task.

[0046] For example, in terms of operation independence, if any two operations can be executed independently, then one operation can correspond to one task; if operations need to be executed in a specific order, then multiple operations that need to be executed in a specific order can be integrated into one task.

[0047] For another example, from the perspective of operation complexity, if an operation does not involve data interaction, such as reading data from a static file, then the operation can be considered a simple operation; if an operation involves data interaction, such as calling an external application programming interface (API), or involving multiple data interactions with a database, then the operation can be considered a complex operation. For simple operations, independent tasks can be created, which helps improve the flexibility and maintainability of task processing. For complex operations, at least two complex operations can be integrated into one task to reduce the system overhead of task switching, which helps improve task processing efficiency.

[0048] In actual application scenarios, both the case where one operation corresponds to one task and the case where multiple operations are integrated into one task may occur. The configuration can be flexibly performed as required, and the embodiments of the present application do not specifically limit this.

[0049] In this embodiment, when performing intent recognition, historical conversation content helps to improve content processing requirements, and the user's language style, word usage habits, and expressions can be obtained from historical conversations. Combining historical conversation content and current content processing requirements can help the preset recognition model more accurately understand the user's true intentions, thereby helping to ensure that the created task meets the user's actual processing requirements for the target screen, and is conducive to reducing communication costs and improving overall processing efficiency.

[0050] In some embodiments, the operations to be performed and the parameters required for each operation are determined based on the intention recognition results, including: obtaining a preset operation range, the operation range is used to indicate at least one operation performed on the screen contents included in the target screen; based on the intention recognition results, extracting the operations to be performed from the operation range, and determining the parameters required for each operation.

[0051] As an example, the information presentation mode of the target screen can be defined by the large screen abstract structure. The information presentation mode can be understood as: the specific presentation of each screen component content included in the target screen. Specifically, each screen component content may include each component constituting the target screen. Each screen component content, for example, includes but is not limited to at least one of the following items: global elements, screen components, component display content, screen layout structure, and screen style information.

[0052] Global elements include elements that are always displayed on the target screen, such as titles, footers, and navigation bars.

[0053] Screen components include but are not limited to at least one of data visualization components, interactive components, and container components. Data visualization components include but are not limited to at least one of components such as charts, tables, and lists. Chart types include but are not limited to at least one of the following: line charts, bar charts, pie charts, and scatter charts. Interactive components are used to provide user interaction, including but not limited to at least one of the following components: buttons, menus, input boxes, and sliders. Container components are used to create and manage containers in the target screen, including but not limited to at least one of the following components: windows, panels, and dialog boxes.

[0054] The component display content at least includes the specific data or content displayed by the data visualization component.

[0055] The screen layout structure is used to indicate the layout mode of the target screen, that is, the layout mode of each screen component contained in each screen component content of the target screen. For example, the layout mode includes but is not limited to the position information and size information of each screen component in the target screen and the data dependency relationship between each screen component. Among them, the data dependency relationship between screen components describes the data flow between components, that is, how data is transferred between different screen components.

[0056] The screen style information is used to indicate the visual display style of the target screen. As an example, the visual display style includes but is not limited to at least one of the following: background color and font style of the target screen.

[0057] As an example, the limitation of the operation scope by the large screen abstract structure may be reflected in: at least one operation performed on the content of each screen component. For ease of understanding, the following is explained by giving examples.

[0058] For example, in terms of information presentation, the limitation on the scope of operation includes but is not limited to at least one of the following operations: adding or replacing data visualization components of corresponding types according to data types, modifying the layout of the data visualization components (i.e., modifying the position information and size information of the corresponding components), adjusting the title in the target screen, updating the display data of any chart, and adjusting the background color of the target screen.

[0059] As a specific example, when the data type is sales performance data, the types of data visualization components may include but are not limited to: sales bar chart, sales line chart, etc. When the data type is traffic data, the types of data visualization components may include but are not limited to: traffic flow bar chart, vehicle speed line chart, traffic accident distribution map, etc.

[0060] As a specific example, when modifying the layout of a data visualization component such as a chart, the parameter operation range of the chart may be preset, such as a position adjustment range, a size adjustment range, and the like.

[0061] It should be understood that the large screen abstract structure can be obtained in advance based on actual usage experience and / or the needs and usage scenarios of the target user group. The operations on the contents of each screen included in the above operation range are only schematically illustrated. In actual applications, the user can set the target operation range through processing instructions according to actual needs, and the embodiments of this application do not make specific limitations.

[0062] In this embodiment, the operation range for performing operations on the contents of each screen can be determined by the large screen abstract structure. Through the operation range, the operations to be performed according to the intention recognition result are further screened, which is conducive to ensuring that the operations to be performed are all supported by the target screen, which is conducive to improving the effectiveness of performing operations on the target screen, thereby helping to improve processing efficiency.

[0063] In some embodiments, in the above step S103, the step of calling the task execution tool corresponding to the task type to execute the corresponding task and obtain the screen display content corresponding to the content processing requirement may specifically include: based on a preset screen construction process and a preset screen layout structure, generating dependencies between the tasks, the screen construction process includes the operations required to construct the screen components included in the target screen, the operations to be performed include the steps required to be performed in the process of generating the corresponding screen components, and the screen layout structure is used to indicate the layout method of the target screen; based on the dependency relationship, determining the calling order of each task; according to the calling order, calling the task execution tools corresponding to each task respectively to execute each task to obtain the screen display content corresponding to the content processing requirement.

[0064] Exemplarily, the screen construction process refers to the overall process of creating and deploying a screen display system based on a target screen, which includes but is not limited to the construction process of each screen composition content described in the above embodiment. The meaning of the screen layout structure can be found in the description of the above embodiment, and will not be repeated here.

[0065] Exemplarily, an operation can be understood as a specific step. Taking the screen component in the screen composition content as an example, when the screen component includes a chart component, the operations required to construct the screen composition content include, but are not limited to, the following steps: obtaining data from a specified data source (such as a database, data file, etc.); performing data cleaning on the obtained data (removing duplicates, processing missing values, etc.); rendering the chart component according to the processed data; adjusting the position and size of the chart component on the screen. By pre-defining the operations required to construct each screen composition content, it is helpful to ensure the accurate implementation of the corresponding screen construction content.

[0066] Exemplarily, for the screen construction process, the dependency between tasks may involve the logical order of operations required to construct the contents of each screen. For example, when it is necessary to add a countdown time component to the target screen, the component is used to display the countdown on the target screen, including at least the following three tasks. Task 1: Set the countdown end time, such as the time when the sales settlement starts. Task 2: After the countdown end time is set, start the countdown. Task 3: Update the countdown in real time. Among them, Task 1 defines the target time point of the countdown and needs to be completed first; Task 2 needs to know when to start the countdown, so this task depends on the completion of Task 1; in Task 3, the time needs to be updated only after the countdown starts, so Task 3 depends on the completion of Task 2. For another example, when it is necessary to display the relevant data of enterprise operation management on the target screen, managers at different levels have different data access rights. Grassroots managers can view daily operation data, while senior managers can access more comprehensive operation analysis data. Therefore, when you need to add a bar chart to the target screen to display operational analysis data, you need to first perform the user authority verification task. If the user authority verification passes, then perform the bar chart data display task.

[0067] For example, for the screen layout structure, the data dependency between screen components determines that the output data of one screen component may be the input data of another screen component. For example, the target screen contains a sales data component and a profit calculation component, such as a sales bar chart and a profit pie chart. When calculating the profit, the profit calculation component needs the sales data as input data, so the output of the sales data component (sales data) is one of the inputs of the profit calculation component (cost data is also required). Therefore, when tasks involve data dependencies between components, the data dependencies between the components can determine the dependencies between the corresponding tasks.

[0068] Exemplarily, in the process of processing the target screen, the dependency relationship between the tasks can be generated based on the screen construction process and the screen layout structure. The dependency relationship between the tasks is conducive to ensuring the correctness of the screen construction process and the screen layout structure.

[0069] Exemplarily, in the screen construction process, for example, operations such as data acquisition, data processing, and view rendering are involved. Based on the screen layout structure, the operations involved include, for example, determining the position and size of the chart component. When the user needs to replace the sales line chart in the upper left corner with a bar chart, the tasks involved include, for example: sales data acquisition task, data processing task (data processing to suit the bar chart format), rendering task, layout task (setting the position and size of the bar chart), and data display task. By analyzing the screen construction process and the screen layout structure, the following task dependencies can be determined: the bar chart rendering task depends on the data processing task and layout task of the sales data; the bar chart data display task depends on the bar chart rendering task.

[0070] In this example, based on the above task dependencies, the calling order of each task can be determined as follows: obtaining sales data; processing sales data to suit a bar chart format; defining the layout structure of the bar chart in the upper left corner area of ​​the target screen, for example: the area where the upper left line chart is located is used as the predetermined position of the bar chart; rendering the bar chart at the predetermined position of the bar chart.

[0071] In this embodiment, the dependency relationship of each task is generated based on the screen construction process and the screen layout structure, and the task calling order is determined based on the dependency relationship, and then each task is executed according to the task calling order. The generation of the dependency relationship ensures that the execution of each task is based on the completion of its predecessor task, thereby ensuring the correctness and processing efficiency of the entire screen construction process and screen layout.

[0072] Exemplarily, a task execution tool refers to a tool component used to execute a task of a corresponding type. Each tool component may be an agent used to execute a task of a corresponding type. In the field of artificial intelligence, an agent is defined as an entity that can perceive the environment, make decisions, and take actions, and has learning and decision-making capabilities to autonomously execute specific tasks.

[0073] Model agents are a type of agents that are built based on a preset recognition model and can provide more advanced functions such as natural language understanding, natural language reasoning, task generation, and task planning based on the powerful computing and learning capabilities of the preset recognition model, where task planning includes the generation of dependencies between tasks and the determination of the calling order. Exemplarily, when a model agent is an agent built based on a large language model, it can be referred to as a large model agent.

[0074] In the above-mentioned execution steps of the embodiment of the present application, a model agent or other agent application based on deep learning can be used to generate dependencies between tasks according to a preset screen construction process and a preset screen layout structure, and determine the calling order of each task based on the dependencies. The type of agent application can be customized according to actual needs, and the embodiment of the present application does not make specific limitations.

[0075] In this embodiment, by generating task dependencies and determining the task calling order, manual intervention can be reduced, which is beneficial to improving the efficiency of screen content processing, and each task execution tool is used to execute its corresponding task, so that the task execution tool has a high degree of matching with the specific task, which is beneficial to improving the professionalism of multi-tasking and improving the efficiency of task processing.

[0076] In some embodiments, the above-mentioned step of generating the dependency relationship between tasks based on a preset screen construction process and a preset screen layout structure may specifically include: determining the first dependency relationship between tasks based on the operations required to be performed for the screen composition content involved in each task; determining the second dependency relationship between tasks based on the dependency relationship between the screen components contained in the screen composition content involved in each task; and determining the dependency relationship between tasks by combining the first dependency relationship and the second dependency relationship.

[0077] Exemplarily, from the content of the above embodiments, it can be known that the first dependency relationship can be understood as a dependency relationship determined based on the logical order of operations required to construct the contents of each screen component, referred to as a logical dependency relationship, which can be used to indicate, for example, which tasks need to be performed before other tasks logically. The second dependency relationship can be understood as a dependency relationship determined based on the data flow between the screen components, referred to as a data dependency relationship, which can be used to indicate, for example, which tasks' outputs are inputs of other tasks.

[0078] In this example, the first dependency and the second dependency can be combined in the following manner to obtain the dependency between the tasks. For example: create a task dependency graph, and visualize the above logical dependency and data dependency in the task dependency graph. For each node in the task dependency graph, any node can represent a task. In the task dependency graph, if there is a directed edge between the nodes corresponding to any two tasks, for example: between the node corresponding to the first task (denoted as the first node) and the node corresponding to the second task (denoted as the second node), there is a directed edge from the first node to the second node, which means that the second task depends on the first task, and the second task can only start to execute after the first task is completed.

[0079] In some scenarios, if there is only a logical dependency between the first task and the second task; or, there is only a data dependency between the first task and the second task; or, there is both a logical dependency and a data dependency between the first task and the second task, and the requirements of the two tasks for the order of task execution are consistent, then the order of task execution between the first task and the second task is clear, and the dependency between the two tasks can be directly determined.

[0080] In other scenarios, if there is both a logical dependency and a data dependency between the first task and the second task, and if there is a conflict between the two, for example, the task execution order indicated by the logical dependency and the data dependency are inconsistent, it can be resolved in any of the following ways.

[0081] One way is to resolve conflicts through predetermined priority rules. For example, a preset logical dependency takes precedence over a data dependency. If the two conflict, the dependencies between tasks are determined based on the logical dependency. Alternatively, a preset data dependency takes precedence over a logical dependency. If the two conflict, the dependencies between tasks are determined based on the data dependency. The specific settings can be customized according to actual needs, and the embodiments of the present application do not make specific limitations.

[0082] Another way is to resolve the conflict through user instructions. For example, in the process of combining logical dependencies and data dependencies to determine the dependencies between tasks, if the above-mentioned conflict occurs, the conflict description information, at least one suggested solution, and the interactive elements corresponding to each solution can be displayed through a message pop-up window on the user terminal. In response to the interactive element for any suggested solution being triggered, the suggested solution selected by the user is obtained, so that the dependency selected by the user can be determined, and the dependency between the corresponding tasks can be determined.

[0083] Specifically, the conflict description information is used to explain: the conflicting tasks and the reasons for the conflict. For example, the conflict description information is "there is a dependency conflict between the first task and the second task. The logical dependency requires the first task to be executed before the second task, but the data dependency requires the second task to be executed before the first task."

[0084] Specifically, the suggested solution is used to provide a conflict resolution method so that the user can select according to the actual situation. For example, the suggested solution is "Suggested solution 1: give priority to satisfying the logical dependency relationship and ensure that the first task is executed before the second task. Suggested solution 2: give priority to satisfying the data dependency relationship and ensure that the second task is executed before the first one."

[0085] Accordingly, the message pop-up window includes a fifth interactive element corresponding to Suggestion Solution 1 and a sixth interactive element corresponding to Suggestion Solution 2. In response to the fifth interactive element being triggered, the suggested solution selected by the user is obtained as "Suggestion Solution 1", and the dependency relationship between the two tasks can be determined based on the logical dependency relationship. In response to the sixth interactive element being triggered, the suggested solution selected by the user is obtained as "Suggestion Solution 2", and the dependency relationship between the two tasks can be determined based on the data dependency relationship.

[0086] It should be noted that the fifth interactive element and the sixth interactive element may be options or buttons provided in the message pop-up window, and the embodiment of the present application does not make specific limitations.

[0087] In this embodiment, the dependencies between the tasks can be determined through the logical dependencies and data dependencies between the tasks shown in the task dependency graph, as well as the solutions to the conflicts between the logical dependencies and data dependencies described above.

[0088] As an example, there are many implementation methods for determining the execution order of each task according to the dependency relationship between the tasks. For example, the execution order of the tasks can be sorted through the following steps A1-A6.

[0089] Step A1, create an initially empty queue (which can be called a node queue) and an initially empty list (which can be a node sorting list). Step A2, add all nodes with zero in-degree to the node queue. Zero in-degree means that the task corresponding to the node is an independent task, and the number of tasks that the task depends on other tasks is zero. Step A3, obtain a node with zero in-degree from the node queue as the current node, take the current node out of the node queue, and add it to the node sorting list. In the node sorting list, each node is arranged in the order of joining, and the node sorting represents the order of task execution. Step A4, update the in-degree of all successor nodes of the current node. Among them, the successor node refers to the node that can be directly reached from the current node through a directed edge. The update of the in-degree means reducing the in-degree of the corresponding successor node. If the updated in-degree of a successor node is zero, the successor node can be added to the above node queue. Step A5, repeat the above steps A3 to A4 until the node queue is empty, indicating that all nodes have been processed, and the nodes arranged in sequence in the node sorting list are obtained. Step A6, obtaining the execution order of each task according to each node arranged in sequence in the node sorting list.

[0090] Through the above steps, it can be ensured that the task corresponding to each node can be executed independently if it is an independent task, and can be started after the dependent tasks are completed if it is a non-independent task, thereby obtaining a task execution order that conforms to the dependency relationship, which is conducive to ensuring that each task is executed in the correct order.

[0091] For another example, a task management tool can be called to process the dependencies between the tasks that have been determined above to obtain the execution order of each task. Task management tools include, but are not limited to, any of the following: task hierarchy management tool (Asana), project management tool (Zoho Projects), etc. These task management tools have task management and dependency processing functions, and can generate a task execution order based on the dependencies between tasks to ensure smooth execution of tasks. In actual applications, a suitable task management tool can be selected according to actual needs, and the embodiments of this application are not specifically limited.

[0092] In this embodiment, the screen content construction process and the screen content layout structure jointly determine the dependency relationship between the tasks. The calling order of the tasks is determined according to the dependency relationship between the tasks, which is conducive to ensuring the correct construction of the display content included in the target screen.

[0093] In some embodiments, the above-mentioned step of calling the task execution tools corresponding to each task in the calling order to execute each task and obtain the screen display content corresponding to the content processing requirement may specifically include: generating a task sequence containing each task in the calling order; for each independent task in the task sequence, using the task execution tools corresponding to each independent task to perform asynchronous calls on each independent task, and any independent task is a task that does not depend on other tasks among the tasks; for any non-independent task in the task sequence, obtaining the output result of the task on which the non-independent task depends, and calling the corresponding task execution tool to execute the non-independent task based on the output result; when all tasks in the task sequence are completed, generating the screen display content corresponding to the content processing requirement according to the output result of the task sequence.

[0094] For ease of understanding, the following Figure 2 Describe the order in which each task is called. Figure 2 A task dependency diagram of an exemplary embodiment of the present application is shown. Figure 2 In the figure, multiple tasks are schematically shown, including Task A, Task B, Task C, and Task D. The arrowed lines between the tasks indicate that there is a dependency relationship between the corresponding tasks. The task at the end of the line depends on the task at the start of the line. Figure 2 In the example, the execution of task C depends on the output of tasks A and B, and the execution of task D depends on the output of task C.

[0095] In some embodiments, if a task depends on other tasks, placeholder variables can be used in the task to represent the outputs of the other tasks on which it depends. One placeholder variable can correspond to one output. When the other tasks on which the task depends generates actual output data, the actual output data can be used to replace the corresponding placeholder variables until all placeholder variables in the task are replaced by the actual output data of the corresponding output.

[0096] Continue to refer Figure 2 , Task A and Task B do not depend on other tasks and can be called independent tasks. Each independent task can be called asynchronously. Asynchronous call means that tasks can be executed concurrently and are not limited by a specific execution order. It is suitable for scenarios where tasks can be processed in parallel and there is no dependency between tasks. Asynchronous call can ensure the independent execution of tasks. For example, when an independent task is obtained, an asynchronous call can be executed on it without waiting for the acquisition or execution of any other tasks, which is conducive to improving the efficiency of task execution.

[0097] Accordingly, Task C and Task D each depend on other tasks, and can be called dependent tasks. The output results of other tasks on which the dependent task depends can be used to execute the dependent task.

[0098] for Figure 2 For each task shown in, when tasks A to D are all executed, the output results of each task can be integrated. The integration of the output results, for example, involves operations such as data merging and data format conversion to ensure that the output of any task is not missed, and then the final screen display content is generated based on the integrated results.

[0099] In this embodiment, in the process of calling the task execution tools corresponding to each task in the calling order to execute each task, multiple independent tasks can be executed simultaneously by asynchronously calling independent tasks, which is beneficial to improving the overall processing efficiency of each task. Non-independent tasks are executed in a specific order based on task dependencies. The clear task sequence and dependency make it easier to achieve the processing requirements of the target screen, thereby improving the overall performance of task processing.

[0100] In some embodiments, after the above-mentioned step of asynchronously calling each independent task using the task execution tool corresponding to each independent task, the following steps may also be included: for any independent task, storing the output result of the independent task to a first storage address, the first storage address is a storage address pre-configured for the task execution tool corresponding to the independent task, and different task execution tools are configured with different storage addresses; obtaining the output result of the task on which the non-independent task depends, including: obtaining the output result of the task on which the non-independent task depends from a second storage address, the second storage address is a storage address pre-configured for the task execution tool corresponding to the task on which the non-independent task depends.

[0101] Exemplarily, different memory address spaces may be allocated in the memory for the first storage address and the second storage address, so that the task execution tool corresponding to each task can access its own memory address space, which is beneficial to ensuring the isolation and security of data processed by each task execution tool.

[0102] In this embodiment, each task execution tool is configured with a dedicated memory address space to store the intermediate results of task processing (different from the final task processing results obtained after all tasks are executed). After a task is completed, the intermediate result will be forwarded as input to other tasks that depend on the task, thereby facilitating the safe and efficient transmission of data between tasks.

[0103] In some embodiments, the content of each screen includes at least one of the following: global elements, data visualization components, screen layout structure and screen style information; wherein, global elements include elements that are always displayed in the target screen; data visualization components are used for data visualization display; and screen style information is used to indicate the visual display style of the target screen.

[0104] Exemplarily, the data visualization component includes but is not limited to at least one of components such as a chart, a table, a list, etc. For different types of components included in the data visualization component, different data display formats are used when displaying the same data content.

[0105] In this embodiment, each screen composition content includes the processed object in the target screen. It should be understood that in actual application scenarios, each screen composition content may also include more other content, which can be determined according to actual processing requirements. By flexibly processing these screen composition contents, the screen display system to which the target screen belongs can maintain its functionality to meet the ever-changing usage requirements.

[0106] In some embodiments, the above-mentioned step of obtaining the screen display content corresponding to the content processing requirements may specifically include: extracting predetermined characteristic indicators based on the generated task output results, the predetermined characteristic indicators including: various indicators used to measure whether the task is successfully executed; when at least one of the indicators meets predetermined conditions, determining that the task output result is a successful output result, the predetermined conditions are used to indicate a predetermined value range for each indicator; generating corresponding screen display content based on the successful output result.

[0107] Exemplarily, the extraction of the predetermined characteristic index is a characteristic index extracted based on the sample extraction logic. In some scenarios, if the task output result includes all relevant information of the task execution, such as output value, error rate, execution time, etc., then the task output result can be regarded as a sample. Accordingly, the characteristic index can include at least one of the following: output value, error rate and execution time.

[0108] Taking the processing instruction "replace the sales line chart in the upper left corner with a bar chart" as an example, the output value can refer to: the expected result after the task is executed, that is, the sales line chart in the upper left corner is successfully replaced with a bar chart. If the task is successfully executed, the output value can be expressed as "the sales bar chart has been displayed in the upper left corner". The error rate can refer to: the proportion of errors in the task output result. The value of the error rate can be calculated by the ratio of the number of task execution failures to the total number of task executions. Among them, the number of task execution failures refers to: the number of times the corresponding output value does not match the expected result, and the total number of task executions is equal to the total number of times the output value is obtained. Assuming that the total number of task executions is 10, and the output value obtained each time is consistent with the expected result, the error rate is 0%; if the output value obtained 3 times does not match the expected result, the error rate is 3%. It can be seen that the error rate is inversely proportional to the probability of successful task execution. The execution time can refer to the time required for all tasks in each task to complete the execution and obtain the task output result.

[0109] Exemplarily, the predetermined conditions corresponding to the output value include: the output value is consistent with the expected result after the task is executed, where the consistency means: the output value is equal to the expected result or the semantic similarity between the two is greater than or equal to the predetermined similarity threshold. The predetermined conditions corresponding to the error rate include: the error rate is less than or equal to the predetermined error rate threshold. The predetermined conditions corresponding to the execution time include: the execution time is less than or equal to the predetermined duration threshold.

[0110] It should be understood that the predetermined similarity threshold, the predetermined error rate threshold and the predetermined duration threshold can be customized according to actual needs, and the embodiments of the present application do not make specific limitations.

[0111] In this embodiment, feature indicators can be extracted based on sample extraction logic, and whether the task output results meet the expected goals can be judged based on the extracted feature indicators. The feature indicators can reflect the objective attributes of the task output results, which helps to accurately judge whether the output meets the expected goals, thereby improving the accuracy of the evaluation and making the evaluation results more objective and reliable.

[0112] In some embodiments, after extracting predetermined feature indicators based on the generated task output results, the following steps may also be included: using a language model to evaluate the task output results to obtain a model evaluation result; when at least one of the indicators meets a predetermined condition and the model evaluation result is a passed evaluation, determining that the task output result is a successful output result.

[0113] Exemplarily, using a language model to evaluate the task output result can be achieved in the following manner. For example, a rule set is pre-built, and the rule set defines the predetermined output result when the task is successfully executed. Taking the received processing instruction "change the sales line chart in the upper left corner into a bar chart" as an example, if the sales data in the task data result is in the bar chart format, it can be considered that the task has been completed.

[0114] For the bar chart format, for example, the difference between the data in the bar chart format and the data in the line chart format includes but is not limited to at least one of the following: the bar chart displays the amount of data through the height of the bar, while the line chart displays the changing trend of the data through the ups and downs of the line; the bar chart can be used to display classified data, and the line chart is suitable for displaying continuous data and time series data.

[0115] In this embodiment, the sample extraction logic and model evaluation method are combined to determine whether the task output result has achieved the expected goal. The advanced reasoning capabilities of the sample extraction logic and language model can be comprehensively utilized to more effectively determine whether the task has achieved the expected goal, thereby adapting to complex task evaluation requirements and providing more flexible and comprehensive evaluation results.

[0116] According to the screen content generation method of the embodiment of the present application, the content processing requirements for the target screen in the form of natural language are parsed using a preset recognition model, which helps to accurately parse the operator's content processing requirements for the target screen, so as to obtain tasks that better meet the operator's processing requirements and improve the accuracy of the generated screen display content. And, in this scheme, the advantage of configuring specific task execution tools for different task types is that: since each task is classified and can be executed by the task execution tool corresponding to the task type, the probability of each task waiting for the previous task to be completed before it can be started during the execution process is greatly reduced, which is conducive to reducing the waiting time before task execution and improving the response speed of task execution; each task execution tool has a high degree of matching with a specific type of task, which is conducive to improving the professionalism of task processing. In the processing process of this scheme, the operator does not need to have a deep understanding of the technical details. Without programming knowledge and a dedicated technical team, the processing instructions for the target screen can also be automatically parsed and the corresponding task execution tool can be called to realize the content processing of the target screen, which is conducive to efficiently and accurately realizing the content processing requirements for the target screen while reducing labor costs and time costs.

[0117] The embodiments of the present application can provide a faster and more controllable method for constructing a digital large screen through natural language interaction, assist users in creating and modifying the generated large screen, expand the application examples of the model in the construction of data visualization systems, provide the data visualization industry with more intelligent data large screen construction and local modification capabilities, and also provide a new solution for traditional graphical user interface application interaction methods. The overall processing flow does not need to repeatedly confirm the tasks to be executed through the model, effectively reducing the number of model calls, making the processing process more refined; and the method of the present application can automatically perform task planning and task execution results evaluation, enhancing the controllability of generating large screen content; and, by concurrently calling each independent task through the task execution tool, it can effectively reduce the task execution waiting time under the multi-task tool call and improve the task processing efficiency.

[0118] Figure 3 FIG. 1 shows a detailed flow chart of a method for generating screen content according to an exemplary embodiment of the present application. Figure 3 As shown, in some embodiments, the screen content generation method includes the following steps.

[0119] Step S301, receiving user instructions.

[0120] In this step, the user instruction may be in at least one of text or voice form.

[0121] Step S302, creating a task.

[0122] In this step, the natural language understanding capabilities and natural language reasoning technology of the preset recognition model are used to mine content processing requirements from historical conversation content and currently input user instructions, and the specific operations that need to be completed and the parameters required for the corresponding operations (also known as: actions and parameters required for actions) are extracted in combination with the preset operation scope to complete the creation of new tasks.

[0123] Step S303: task planning.

[0124] In this step, after the new task is created, all current tasks can be screened according to the preset screen building process and the pre-summarized screen layout structure to obtain independent tasks and dependent tasks.

[0125] Among them, the screen construction process can be a construction process summarized based on the experience of building digital large screens. The screen layout structure can also be called a large screen abstract structure. Independent tasks can be isolated from each other. Mutual isolation means that there is no direct connection between independent tasks, and the progress or result of one task will not affect other tasks. Non-independent tasks are also called associated tasks and tasks with dependencies. Associated tasks can be prioritized, for example, based on dependencies, determine the calling order of each task and the execution order of each associated task.

[0126] As an example, for each task that needs to be performed that is parsed from the content processing requirements, before task planning is performed, it can be understood as an unordered set of tasks. In order to ensure the accuracy of the completion of subsequent tasks, a task sequence and its dependencies can be generated according to the fixed order calling requirements caused by the dependencies between the tasks to form a directed acyclic graph. The interdependencies between the tasks are identified by using the pre-summarized ordered process (i.e., screen construction process) and abstract structure (i.e., screen layout structure) of the large screen construction. If a non-independent task depends on another task, a placeholder variable can be used to replace the output of the other task, and the actual output of the other task can be used to replace the placeholder variable later. After the above task planning is performed, the corresponding actions of each task that needs to be completed can be added to the action group in order to wait for being called. The corresponding action of each task is at least one operation required to perform each task.

[0127] Step S304, generating a task queue.

[0128] In this step, after determining the calling order of each task through the interdependence between the tasks, a corresponding task queue can be generated according to the calling order. Through the task queue, it can be ensured that the dependent tasks are executed first, and then the tasks that depend on other tasks are executed, ensuring that each task can be carried out smoothly and efficiently.

[0129] Step S305: concurrently calling the task execution tool to execute each task.

[0130] In this step, the tasks in the task queue are obtained in sequence, and the task execution tools corresponding to the obtained tasks are used to execute the tasks. After each task is completed, the screen display content corresponding to the content processing requirements is generated according to the task output results. Specifically, independent tasks can be called asynchronously and concurrently according to the interdependence of the tasks.

[0131] Exemplarily, the target screen comprises the following four types: global elements, data visualization components (such as charts and their display contents), screen layout structure (i.e., large-screen structure), and screen style information (i.e., large-screen style), and each is equipped with its own corresponding intelligent agent tool, i.e., task execution tool, for the execution of each task.

[0132] In some scenarios, each agent tool can be configured with a dedicated memory address space to store the output results of the current task. The output results of the task can be forwarded as intermediate results to other tasks that depend on it.

[0133] For example, in Figure 3 In the example, the task execution tool for modifying global elements can be named as a large-screen global element modification agent. The task execution tool for modifying charts and their displayed contents can be named as a large-screen chart recommendation agent; the task execution tool for modifying the screen layout structure can be named as a large-screen layout agent; and the task execution tool for modifying the screen style information can be named as a stylized agent.

[0134] In this example, the large screen global element modification agent can modify the global elements in the target screen. The large screen chart recommendation agent can be used to perform view display tasks. In the data visualization process of the target screen, the view can include but is not limited to charts, tables, lists and other data display forms. The style agent can be used to modify the style information in the target screen. It should be understood that in the description of the embodiments of the present application, the view node can be understood as a screen component.

[0135] In this example, when modifying the screen layout, the large screen layout agent can determine the layout of the screen components through the layout tree. Specifically, the layout tree includes multiple view nodes, one view node can represent the chart type of a chart view and the size information of the chart view in the target screen, and multiple view nodes are distributed in multiple levels; by mapping multiple view nodes in the layout tree to different positions in the target screen, the layout of the chart view in the target screen can be determined.

[0136] Step S306, result evaluation.

[0137] In this step, feature indicators can be extracted based on the sample extraction logic, and whether the task output results meet the expected goals can be determined based on the extracted feature indicators. Figure 3 In some embodiments, if it is determined that the task output result does not reach the expected goal, the process may return to step S303, i.e., re-execute the task planning.

[0138] Exemplarily, in the process of re-executing task planning, in response to a new user instruction to be received, the operation that needs to be modified and / or the parameters required to modify at least one operation can be obtained from the new user instruction, and at least one new task can be created based on the modified operation and / or the parameters required for each modified operation, and then task planning can be performed for each new task.

[0139] Step S306, constructing a large screen.

[0140] In this step, the target screen can be constructed using the task output results according to the abstract syntax. Abstract syntax is a way to describe the code structure in programming languages. Using abstract syntax, the layout of screen components can be determined through the above layout tree, thereby obtaining the corresponding large screen layout structure.

[0141] Through the above steps S301-S306, the natural language understanding and reasoning technology based on the preset recognition model can be used, combined with the calling ability of the task execution tool, so that the terminal user can directly use a spoken language to command the change of the configuration or content of the chart component on the digital large screen. According to this method, the terminal user does not need to have an in-depth understanding of the technical details, and can also automatically parse the processing instructions for the target screen and call the corresponding task execution tool to realize the content processing of the target screen without programming knowledge and a dedicated technical team. While reducing labor costs and time costs, it is conducive to efficiently and accurately realizing the content processing requirements of the target screen.

[0142] In an embodiment of the present application, when interacting with a preset recognition model, the model can be made to execute processing instructions or provide required information by calling specific functions or instructions. Utilize the capabilities of the preset recognition model to solve complex problems or obtain customized responses. According to the method of the embodiment of the present application, the natural language processing technology of the preset recognition model is utilized, and combined with the ability of the intelligent tool call, a new intelligent tool call framework is proposed and developed, and an overall process from intention recognition, task planning and sorting, to concurrent calls of task execution tools is built, which is conducive to reducing the user's learning cost for professional technology, and enables users to easily manage and quickly update the configuration content of the digital large screen. In this solution, through the preset recognition model intelligent agent can understand the user's needs in a complex context to propose task requirements, and then automatically plan the processing flow of each task through the task planning and sorting process, and coordinate multiple task execution tools to perform concurrent execution of each independent task to ensure efficient completion of the task.

[0143] According to the model agent and task execution tool agent contained in the agent tool calling framework of the embodiment of the present application, the digital large-screen content can be managed more intelligently. Non-technical users can easily update and customize the digital large-screen content without programming knowledge, which greatly reduces the operating cost and improves the timeliness and personalization level of content updates.

[0144] Figure 4 A flow chart of a method for displaying screen content according to an embodiment of the present application is shown. Figure 4 As shown, the method may include the following steps S401-S404.

[0145] S401, receiving a display instruction for a target screen.

[0146] S402: In response to the display instruction, display the current screen display content of the target screen.

[0147] S403, receiving a processing instruction for the target screen, where the processing instruction is used to indicate a content processing requirement for the target screen through natural language.

[0148] S404, in response to the processing instruction, display new screen display content, where the new screen display content is screen display content corresponding to the content processing requirement obtained by processing the processing instruction according to the screen content generation method of the embodiment of the present application.

[0149] According to the screen content display method of the embodiment of the present application, the screen display content can be displayed in response to the received display instruction for the target screen, and the new screen display content can be displayed in response to the received processing instruction for the target screen, and the new screen display content is the screen display content corresponding to the content processing requirement obtained by processing the processing instruction according to any screen content generation method of the above embodiment of the present application. Through this method, the content of the target screen can be dynamically displayed according to the received display instruction, and the new screen display content corresponding to the content processing requirement can be generated in response to the processing instruction, so that real-time processing can be flexibly performed according to the instruction.

[0150] The screen content generation method in the embodiment of the present application can refer to the above embodiment combined with Figure 1-Figure 3 The corresponding description in the screen content generation method described above has corresponding beneficial effects and will not be repeated here.

[0151] Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application also provides a screen content generating device.

[0152] Figure 5 A schematic diagram of the structure of a screen content generation device according to an embodiment of the present application is shown, and the device is used to execute the screen content generation method provided in any of the above embodiments, such as Figure 5 As shown, the screen content generating device includes: The receiving module 510 receives a processing instruction, where the processing instruction is used to indicate the content processing requirements of the target screen through natural language.

[0153] The parsing module 520, in response to the processing instruction, uses a preset recognition model to parse the content processing requirements to obtain the tasks that need to be executed.

[0154] The execution module 530, based on the task type to which each task belongs, calls a task execution tool corresponding to the task type to execute the corresponding task, and obtains the screen display content corresponding to the content processing requirement.

[0155] In some embodiments, when the parsing module 520 is used to parse the content processing requirements using a preset recognition model to obtain the tasks that need to be performed, it is specifically used to: perform intent recognition based on pre-stored historical session content and content processing requirements using a preset recognition model; determine the operations that need to be performed and the parameters required for each operation based on the intent recognition results; and create at least one task based on each operation and parameter, and the task is used to perform the corresponding operation based on at least one parameter.

[0156] In some embodiments, when the analysis module 520 is used to determine the operations to be performed and the parameters required for each operation based on the intention recognition result, it is specifically used to: obtain a preset operation range, the operation range is used to indicate at least one operation performed on the screen contents included in the target screen; based on the intention recognition result, extract the operations to be performed from the operation range, and determine the parameters required for each operation.

[0157] In some embodiments, when the execution module 530 is used to call the task execution tool corresponding to the task type to execute the corresponding task and obtain the screen display content corresponding to the content processing requirements, it is specifically used to: generate dependencies between tasks based on a preset screen construction process and a preset screen layout structure, the screen construction process includes the operations required to build the screen components included in the target screen, the operations to be performed include the steps required to be performed in the process of generating the corresponding screen components, and the screen layout structure is used to indicate the layout method of the target screen; based on the dependency relationship, determine the calling order of each task; according to the calling order, call the task execution tools corresponding to each task to execute each task to obtain the screen display content corresponding to the content processing requirements.

[0158] In some embodiments, when the execution module 530 is used to generate dependencies between tasks based on a preset screen construction process and a preset screen layout structure, it is specifically used to: determine a first dependency between tasks based on the operations required to be performed for the screen composition content involved in each task; determine a second dependency between tasks based on the dependencies between screen components contained in the screen composition content involved in each task; and determine the dependencies between tasks by combining the first dependency and the second dependency.

[0159] In some embodiments, when the execution module 530 is used to call the task execution tools corresponding to each task in accordance with the calling order to execute each task and obtain the screen display content corresponding to the content processing requirements, it is specifically used to: generate a task sequence containing each task in accordance with the calling order; for each independent task in the task sequence, use the task execution tools corresponding to each independent task to perform asynchronous calls on each independent task, and any independent task is a task that does not depend on other tasks in the task sequence; for any non-independent task in the task sequence, obtain the output result of the task on which the non-independent task depends, and call the corresponding task execution tool to execute the non-independent task based on the output result; when all tasks in the task sequence are completed, generate the screen display content corresponding to the content processing requirements according to the output result of the task sequence.

[0160] In some embodiments, the screen content generating device also includes a storage module, which is used to store the output result of the independent task to a first storage address for any independent task after asynchronously calling each independent task using the task execution tool corresponding to each independent task. The first storage address is a storage address pre-configured for the task execution tool corresponding to the independent task, and different task execution tools are configured with different storage addresses; when the execution module 530 is used to obtain the output result of the task on which the non-independent task depends, it is specifically used to: obtain the output result of the task on which the non-independent task depends from the second storage address, and the second storage address is a storage address pre-configured for the task execution tool corresponding to the task on which the non-independent task depends.

[0161] In some embodiments, the content of each screen includes at least one of the following: global elements, data visualization components, screen layout structure and screen style information; wherein, global elements include elements that are always displayed in the target screen; data visualization components are used for data visualization display; and screen style information is used to indicate the visual display style of the target screen.

[0162] In some embodiments, when the execution module 530 is used to obtain the screen display content corresponding to the content processing requirements, it is specifically used to: extract predetermined characteristic indicators based on the generated task output results, the predetermined characteristic indicators include: various indicators used to measure whether the task is successfully executed; when at least one of the indicators meets the predetermined conditions, determine that the task output result is a successful output result, the predetermined conditions are used to indicate the predetermined value range of each indicator; generate the corresponding screen display content based on the successful output result.

[0163] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

[0164] Figure 6 A schematic diagram of the structure of a screen content display device according to an embodiment of the present application is shown, and the device is used to execute the screen content display method provided in any of the above embodiments, such as Figure 6 As shown, the screen content display device includes: The receiving module 610 receives a display instruction for a target screen.

[0165] The display module 620 displays the current screen display content of the target screen in response to the display instruction.

[0166] The receiving module 610 is further used to receive a processing instruction for the target screen, where the processing instruction is used to indicate a content processing requirement for the target screen through natural language.

[0167] The display module 620 is also used to display new screen display content in response to the processing instruction. The new screen display content is the screen display content corresponding to the content processing requirement obtained by processing the processing instruction according to any of the above-mentioned screen content generation methods.

[0168] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

[0169] Figure 7 FIG. 1 shows a block diagram of an electronic device provided by an embodiment of the present application. Figure 7 As shown, the electronic device includes: a memory 701 and a processor 702, and the memory 701 stores a computer program that can be run on the processor 702. When the processor 702 executes the computer program, the method in the above embodiment is implemented. The number of the memory 701 and the processor 702 can be one or more. In a specific implementation, the electronic device may also include a communication interface 703 for communicating with external devices and performing data exchange transmission.

[0170] In specific implementation, if the memory 701, the processor 702 and the communication interface 703 are implemented independently, the memory 701, the processor 702 and the communication interface 703 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0171] Optionally, in a specific implementation, if the memory 701, the processor 702 and the communication interface 703 are integrated on a chip, the memory 701, the processor 702 and the communication interface 703 can communicate with each other through an internal interface.

[0172] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which implements any method provided in the embodiment of the present application when the program is executed by a processor.

[0173] An embodiment of the present application provides a computer program product, including a computer program, which implements any method provided in the embodiment of the present application when executed by a processor.

[0174] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes any method provided in the embodiment of the present application.

[0175] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute any one of the methods provided in the embodiment of the application.

[0176] It should be understood that the above processor can be a processor (Central Processing Unit, CPU), and can also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application-specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. It is worth noting that the processor can be a processor that supports the Advanced RISC Machines (ARM) architecture.

[0177] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of exemplary but not limiting description, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM).

[0178] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0179] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0180] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0181] Any process or method described in the flow chart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0182] The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.

[0183] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0184] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0185] The above is only an exemplary embodiment of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope recorded in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for generating screen content, characterized in that: include: receiving a processing instruction, wherein the processing instruction is used to indicate a content processing requirement of a target screen through natural language; In response to the processing instruction, the preset recognition model is used to parse the content processing requirements to obtain tasks that need to be performed; The task execution tool is called to execute the corresponding task, and the screen display content corresponding to the content processing requirement is obtained.

2. The method according to claim 1, characterized in that The using of a preset recognition model to parse the content processing requirements to obtain the tasks to be performed includes: Based on the pre-stored historical conversation content and the content processing requirements, the preset recognition model is used to perform intent recognition, wherein the historical conversation content is used to indicate the pre-collected conversation history related to the content processing of the target screen; Determine each operation to be performed and parameters required for each operation according to the intention recognition result; At least one task is created according to the operations and the parameters, and the task is used to perform the corresponding operation based on at least one parameter.

3. The method according to claim 2, characterized in that The determining of each operation to be performed and the parameters required for each operation according to the intention recognition result includes: Acquire a preset operation range, where the operation range is used to indicate at least one operation to be performed on each screen component content included in the target screen; Based on the intention recognition result, each operation to be performed is extracted from the operation range, and parameters required for each operation are determined.

4. The method according to claim 1, characterized in that: The calling of the task execution tool corresponding to the task type to execute the corresponding task and obtain the screen display content corresponding to the content processing requirement includes: Based on a preset screen construction process and a preset screen layout structure, the dependency relationship between the tasks is generated, the screen construction process includes operations required to be performed to construct each screen component content included in the target screen, the operations required to be performed include each step required to be performed in the process of generating the corresponding screen component content, and the screen layout structure is used to indicate the layout mode of the target screen; Based on the dependency relationship, determining the calling order of the tasks; According to the calling sequence, the task execution tools corresponding to the tasks are called to execute the tasks, so as to obtain the screen display content corresponding to the content processing requirement.

5. The method according to claim 4, characterized in that The generating of the dependency relationship between the tasks based on the preset screen building process and the preset screen layout structure includes: Determining a first dependency relationship between the tasks based on operations required to be performed for screen composition contents involved in the tasks; Determining a second dependency relationship between the tasks according to a dependency relationship between the screen components included in the screen composition content involved in the tasks; Dependencies between the tasks are determined by combining the first dependency and the second dependency.

6. The method according to claim 4, characterized in that The calling of the task execution tools corresponding to the tasks respectively according to the calling sequence to execute the tasks and obtain the screen display content corresponding to the content processing requirements includes: Generate a task sequence including the tasks according to the calling order; For each independent task in the task sequence, asynchronously call each independent task using the task execution tool corresponding to each independent task, and any independent task is a task that does not depend on other tasks among the tasks; For any dependent task in the task sequence, obtaining an output result of a task on which the dependent task depends, and calling a corresponding task execution tool to execute the dependent task based on the output result; When all the tasks in the task sequence are completed, screen display content corresponding to the content processing requirement is generated according to the output results of the task sequence.

7. The method according to claim 6, characterized in that After the task execution tools corresponding to the independent tasks are used to asynchronously call the independent tasks, the method further includes: For any independent task, the output result of the independent task is stored in a first storage address, where the first storage address is a storage address pre-configured for a task execution tool corresponding to the independent task, and different task execution tools are configured with different storage addresses; The obtaining of the output result of the task on which the non-independent task depends includes: obtaining the output result of the task on which the non-independent task depends from a second storage address, wherein the second storage address is a storage address pre-configured for a task execution tool corresponding to the task on which the non-independent task depends.

8. The method according to claim 4, characterized in that The contents of each screen include at least one of the following: global elements, data visualization components, the screen layout structure and screen style information; wherein the global elements include elements that are always displayed in the target screen; the data visualization components are used for data visualization display; and the screen style information is used to indicate the visual display style of the target screen.

9. The method according to claim 1, characterized in that: The obtaining of the screen display content corresponding to the content processing requirement includes: Extracting predetermined characteristic indicators based on the generated task output result, wherein the predetermined characteristic indicators include: indicators for measuring whether the task is successfully executed; When at least one of the indicators satisfies a predetermined condition, determining that the task output result is a successful output result, wherein the predetermined condition is used to indicate a predetermined value range of the indicators; Generate corresponding screen display content based on the successful output result.

10. A method for displaying screen content, characterized in that: Applied to the target screen, the method comprises: receiving a display instruction for the target screen; In response to the display instruction, display the current screen display content of the target screen; receiving a processing instruction for the target screen, wherein the processing instruction is used to indicate a content processing requirement for the target screen through natural language; In response to the processing instruction, new screen display content is displayed, where the new screen display content is the screen display content corresponding to the content processing requirement obtained by processing the processing instruction according to the method described in any one of claims 1-9.

11. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the method of any one of claims 1 to 9 or claim 10 when executing the computer program.

12. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 or claim 10 is implemented.

13. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9 or claim 10.

Images