Method and device for executing system functions of software system, equipment and storage medium

Abstract

The invention provides a method and a device for executing system functions of a software system, electronic equipment and a storage medium. The method comprises the following steps: acquiring demand information of a user; based on an analysis result of the demand information by the first artificial intelligence model, determining an operation path corresponding to the demand information, and providing a system function corresponding to the operation path, a tree structure is adopted to represent system functions in a software system and an execution sequence among the system functions, an artificial intelligence model is combined with the tree structure to analyze demand information, nodes related to the demand information in the tree structure are identified, and a corresponding operation path can be generated based on an association relationship among the nodes in the tree structure. According to the method, the operation parameters of the system functions are stored by using the nodes of the tree structure, code packaging for the system functions is not needed, and the applicability of automatically executing the system functions is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a method and apparatus for executing system functions of a software system, an electronic device, and a non-transitory computer-readable storage medium. Background Art

[0002] With the development of artificial intelligence (AI), more and more software systems are integrating digital assistants. These digital assistants interact with AI models (e.g., large models) and, with the help of these models, automatically execute the software system's functions, making it more convenient for users to use the software system.

[0003] Typically, each system function involved in a software system is encapsulated in code. An AI model analyzes user requirements, determines the system functions involved, and the execution order of each function. It then constructs an operation path based on this execution order and provides feedback to the digital assistant. The digital assistant can then execute the encapsulated code for each function based on the execution order indicated by the operation path.

[0004] However, in the above method, code is encapsulated separately for each system function, and each system function is relatively isolated, making it difficult to accurately and comprehensively describe the overall operation process of the software system, and thus difficult to ensure the reliability of the operation path fed back by the artificial intelligence model. Inaccuracies are very likely to occur when meeting user needs. Summary of the Invention

[0005] This summary is provided to briefly introduce concepts that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0006] At least one embodiment of the present disclosure provides a method for executing system functions of a software system, the method comprising: obtaining user demand information, wherein the demand information comprises: indicating the user's demand for function execution of at least one of the software systems; determining an operation path corresponding to the demand information based on an analysis result of the demand information by a first artificial intelligence model, wherein the operation path is determined based on a tree structure corresponding to at least one of the software systems, the tree structure comprising a plurality of nodes, each of the plurality of nodes corresponding to a system function of at least one of the software systems and comprising operation parameters related to the corresponding system function, the association relationship between the plurality of nodes of the tree structure representing the execution order between the plurality of system functions of at least one of the software systems, the operation path comprising at least one node having an association relationship in the tree structure; and providing the system function corresponding to the operation path.

[0007] At least one embodiment of the present disclosure provides a device for executing system functions of a software system, the device comprising: an acquisition module configured to acquire user demand information, wherein the demand information comprises: an indication of the user's demand for function execution of at least one of the software systems; a determination module configured to determine an operation path corresponding to the demand information based on an analysis result of the demand information by a first artificial intelligence model, wherein the operation path is determined based on a tree structure corresponding to at least one of the software systems, the tree structure comprising a plurality of nodes, each of the plurality of nodes corresponding to a system function of at least one of the software systems and comprising operation parameters related to the corresponding system function, the association relationship between the plurality of nodes of the tree structure representing the execution order between the plurality of system functions of at least one of the software systems, and the operation path comprising at least one node having an association relationship in the tree structure; and a providing module configured to provide the system function corresponding to the operation path.

[0008] At least one embodiment of the present disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for executing system functions of a software system provided by at least one embodiment of the present disclosure.

[0009] At least one embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method for executing system functions of a software system provided by at least one embodiment of the present disclosure.

[0010] At least one embodiment of the present disclosure provides a computer program product, including a computer program. When the computer program is executed by a processor, the computer program implements the method for executing system functions of a software system provided in at least one embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, rather than limiting the present disclosure.

[0012] Figure 1 A diagram schematically illustrates an application scenario of a method for executing system functions of a software system provided by at least one embodiment of the present disclosure;

[0013] Figure 2A flowchart schematically illustrates a method for executing system functions of a software system provided by at least one embodiment of the present disclosure;

[0014] Figure 3 A schematic diagram of a process for constructing a tree structure provided by at least one embodiment of the present disclosure is shown;

[0015] Figure 4 A schematic diagram of a tree structure provided by at least one embodiment of the present disclosure is schematically shown;

[0016] Figure 5 A schematic diagram of a process for updating a tree structure provided by at least one embodiment of the present disclosure is shown;

[0017] Figure 6 A schematic diagram illustrating the structure of an apparatus for executing system functions of a software system provided by at least one embodiment of the present disclosure is shown;

[0018] Figure 7 A schematic diagram of the structure of an electronic device for implementing the system function of executing a software system provided by at least one embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0020] Unless otherwise defined, the technical or scientific terms used in this disclosure should have the usual meanings understood by persons of ordinary skill in the field to which this disclosure belongs. The words "first", "second" and similar terms used in this disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Words such as "include" or "comprise" mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0021] In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits detailed descriptions of some known functions and components.

[0022] With the development of computer technology, artificial intelligence (AI) has been widely used. For example, AI models with a large number of parameters, built by AI networks, have been widely applied in search engines, intelligent agents, related vertical industries, and basic disciplines, driving the intelligent development of various industries.

[0023] Specifically, more and more software systems are connected to digital assistants, which can improve the convenience of users using software systems by interacting with artificial intelligence models. For example, when using a software system, users can interact with the digital assistant to inform the digital assistant of their functional execution requirements for the system functions of the software system. The digital assistant will call the artificial intelligence model, identify the user's intention, and automatically execute various system functions. In this way, there is no need for manual operation by the user, making it easier for users to use the software system more flexibly.

[0024] Usually, each system function involved in the software system is encapsulated in code in advance. For example, each system function is encapsulated in code based on technologies such as function call, model context protocol (MCP), or agent-user interaction protocol (AG-UI), and the parameters and functions of each system function after code encapsulation are described.

[0025] After the digital assistant receives the user's request information, it calls the artificial intelligence model and sends the user's request information to the artificial intelligence model. The artificial intelligence model analyzes the user's request information based on the parameters and function descriptions of each system function after code encapsulation, determines the system functions involved in the user's request information, and the execution order of each system function. It then forms an operation path based on the execution order and feeds this operation path back to the digital assistant. The digital assistant can execute the encapsulated code corresponding to each system function according to the execution order of each system function indicated by the operation path.

[0026] For example, the software system can be an email system. User A interacts with the digital assistant and informs the digital assistant of the need to "send an email to user B". The digital assistant calls the artificial intelligence model, which analyzes the need information based on the parameters and function descriptions of each system function after the email system code is encapsulated. It determines the system functions involved in the need information, such as logging into the email system, filling in email information, and sending emails, as well as the execution order between the various system functions, such as first executing the system function of logging into the email system, then executing the system function of filling in email information, and finally executing the system function of sending emails, to generate an operation path. The digital assistant receives the operation path fed back by the artificial intelligence model and executes the encapsulated code corresponding to logging into the email system, filling in email information, and sending email in the order of execution, thereby completing the user's function execution requirements for the email system.

[0027] However, the above method has the following problems: First, the above method performs code encapsulation for each system function separately, and each system function is relatively isolated, making it difficult to accurately and comprehensively describe the overall operation process of the software system. Secondly, the above method relies too much on the artificial intelligence model for planning the operation path. When the artificial intelligence model is replaced or the artificial intelligence model has hallucination problems, it is very easy to cause the operation path planned by the artificial intelligence model to be incorrect, making it difficult to form a reliable operation path. In addition, the above method relies on the interface service or page service provided by the software system for code encapsulation of system functions. For software systems with strong closedness, it is difficult to achieve code encapsulation of each system function, which in turn leads to the fact that the capabilities that the digital assistant can provide are not comprehensive enough.

[0028] In view of this, at least one embodiment of the present disclosure provides a method for executing system functions of a software system, the method comprising: obtaining user demand information, wherein the demand information comprises: indicating the user's demand for function execution of at least one software system; determining an operation path corresponding to the demand information based on an analysis result of the demand information by a first artificial intelligence model, wherein the operation path is determined based on a tree structure corresponding to at least one software system, the tree structure comprising multiple nodes, each of the multiple nodes corresponding to a system function of at least one software system, and comprising operation parameters related to the corresponding system function, the association relationship between the multiple nodes of the tree structure representing the execution order between multiple system functions of the at least one software system, the operation path comprising at least one node having an association relationship in the tree structure; and providing a system function corresponding to the operation path.

[0029] In the above-mentioned embodiments of the present disclosure, a tree structure is used to model the system functions of the software system. For example, the tree structure can represent the execution order of the system functions themselves and the system functions in the software system, and accurately model the overall operation process of the software system. Therefore, the artificial intelligence model analyzes the demand information in combination with the tree structure, identifies the nodes involved in the demand information in the tree structure, and generates the corresponding operation path based on the association relationship between the nodes in the tree structure. There is no need for the artificial intelligence model to plan the execution order of each system function; and, by using the nodes of the tree structure to store the operation parameters of the system functions, there is no need to encapsulate the code for the system functions, thereby improving the applicability of the automatic execution of the system functions.

[0030] At least one embodiment of the present disclosure further provides an apparatus for executing system functions of a software system, an electronic device, a computer-readable storage medium, and a computer program product. The above-described method for executing system functions of a software system can be applied to the apparatus for executing system functions of a software system provided in an embodiment of the present disclosure, and the apparatus for executing system functions of a software system can be configured on an electronic device.

[0031] For example, the electronic device of the embodiment of the present disclosure may be a personal computer, a mobile terminal, etc., and the mobile terminal may be a mobile phone, a laptop computer, a tablet computer, a wearable device, a vehicle-mounted device, etc.

[0032] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings, but the present disclosure is not limited to these specific embodiments.

[0033] Figure 1 An application scenario diagram of a method for executing system functions of a software system provided by at least one embodiment of the present disclosure is schematically shown.

[0034] like Figure 1 As shown, the application scenario of this embodiment includes user 101. For example, user 101 may be a user of software system 103. In the embodiment of the present disclosure, software system 103 may be deployed and run on a terminal device. For example, the terminal device may be a mobile phone, tablet computer, portable computer (such as a laptop or tablet computer), desktop computer, smart wearable device, smart home appliance, or smart car terminal. In other words, user 101 may use software system 103 through the terminal device.

[0035] Specifically, user 101 can send demand information to software system 103. For example, when software system 103 is connected to a digital assistant, user 101 can send demand information to software system 103 by interacting with the digital assistant. The demand information can indicate user 101's demand for the function execution of software system 103.

[0036] In the disclosed embodiments, software system 103 can be understood as a system that provides software services. For example, software system 103 can be a desktop or mobile application (APP). Software system 103 provides system functions, and a system function can be understood as a sub-service within software system 103. Different types of software systems 103 can provide different system functions. For example, when software system 103 is a collaborative office system, it can provide system functions related to schedule management, meeting management, and document processing. For another example, when software system 103 is an instant messaging system, it can provide system functions such as group chat, one-on-one chat, and message management.

[0037] In the embodiment of the present disclosure, software system 103 has a corresponding tree structure 104. Each node included in tree structure 104 corresponds to each system function of software system 103. That is, each node in tree structure 104 represents a system function of software system 103, and the associations between multiple nodes in tree structure 104 represent the execution order of multiple system functions of software system 103. In other words, the tree structure's characteristic of including associations between nodes can be used to depict the functions themselves and the execution order of each system function in software system 103.

[0038] The embodiment of the present disclosure does not limit the form of the tree structure 104. For example, the tree structure 104 can be stored in association with the software system 103 in the form of a directed graph. For another example, the tree structure 104 can also be stored in association with the software system 103 in the form of a code file, that is, the relationship between each node in the tree structure 104 and the relationship between the nodes is described using code and stored as a code file.

[0039] In response to user 101's demand information, software system 103 (e.g., a digital assistant connected to software system 103) can call artificial intelligence model 105. For example, artificial intelligence model 105 can be deployed on a server, e.g., as an internal component of software system 103 and called by software system 103, which is also deployed on the same server. Alternatively, artificial intelligence model 105 can be deployed externally to software system 103, as an external component of software system 103 and called by software system 103, which is deployed on a client or on a server different from software system 103.

[0040] Artificial intelligence model 105 can analyze user 101's requirement information based on tree structure 104 corresponding to software system 103, generate an operation path corresponding to the requirement information, and return the operation path to software system 103. The operation path can include at least one node with an associated relationship in tree structure 104, that is, the operation path can be used to indicate the system functions involved in the requirement information and the execution order of the involved system functions.

[0041] In this way, the software system 103 can provide the user 101 with corresponding system functions according to the operation path, thereby realizing the automatic execution of the system functions corresponding to the user's 101 demand information.

[0042] Figure 2 The flowchart of a method for executing system functions of a software system provided by at least one embodiment of the present disclosure is schematically shown. Figure 2 As shown, the method specifically includes:

[0043] S201: Obtain user demand information.

[0044] In embodiments of the present disclosure, requirement information may include information indicating a user's requirements for executing at least one software system function. Specifically, the requirement information may be understood as the user's requirements for executing system functions of the software system. For example, if the software system provides system functions A, B, and C, the requirement information may indicate the user's requirements for executing system functions A and C.

[0045] In some embodiments, the requirement information can be expressed in natural language. That is, the user can express their requirement for the software system's function execution by inputting natural language. For example, if the software system is an email system, the requirement information can be "Please send an email to user B to ask for the meeting time."

[0046] The user may input the demand information described in natural language in different ways. For example, the user may input the natural language content representing the demand information through text input, or through voice input, or by triggering a preset control to input the natural language content representing the demand information, wherein the preset control may correspond to the preset demand information.

[0047] The embodiments of the present disclosure do not limit the method for obtaining user demand information. In some possible implementations, the software system can be connected to a digital assistant, and a control for invoking the digital assistant is presented on the software system's usage page. The user's demand information is obtained in response to the user clicking the digital assistant control and inputting information on the digital assistant's interactive interface.

[0048] S202: Determine an operation path corresponding to the demand information based on the analysis result of the demand information by the first artificial intelligence model.

[0049] In an embodiment of the present disclosure, a first artificial intelligence model is used to analyze the demand information and return an operation path that matches the demand information and can be used to execute the system function involved in the demand information. The first artificial intelligence model (i.e., the artificial intelligence model 105 mentioned above) can be understood as a model with artificial intelligence capabilities (such as natural language processing capabilities, image processing capabilities, etc.). For example, the first artificial intelligence model can specifically include any one of a large language model, a large visual model, a large audio model, and a large multimodal model, or a combination of multiple thereof. For example, the first artificial intelligence model can be a model built based on a transformer architecture, a model built based on a recurrent neural network, a model built based on an attention mechanism, etc. Alternatively, the first artificial intelligence model can also be a model improved on the basis of the transformer architecture, such as a mixture of experts (MoE).

[0050] The operation path returned by the first artificial intelligence model can be determined based on a tree structure corresponding to the at least one software system. The tree structure can include multiple nodes, each of the multiple nodes corresponding to a system function of the at least one software system, and each node includes an operation parameter related to the corresponding system function. The association relationship between the multiple nodes of the tree structure can represent the execution order between multiple system functions of the at least one software system.

[0051] That is, a software system corresponds to a tree structure, and the tree structure can be used to describe the overall operation process of each system function in the software system. In the embodiment of the present disclosure, a single system function is described in the form of a node in the tree structure, for example, the node name of the tree structure is used to describe the system function.

[0052] The nodes of the tree structure store the operating parameters required to execute the system function. That is, according to the operating parameters stored in the node, the system function corresponding to the node can be executed. In some embodiments, the operating parameters can be represented in the form of a data structure S:

[0053] S={F{M, P}, R, XP},

[0054] Among them, F is the functional element set of the system function, each F includes M and P, M represents the operation corresponding to the functional element, such as "clicking the login control", P represents the parameters associated with the operation corresponding to the functional element, such as "user name information" and "password information", R represents the execution events involved in each functional element in the system function, such as click events, input events, sliding events, selection events, etc., XP represents the functional element set of other system functions associated with the system function, for example, XP={F1, F2,..., Fn}, F1 to Fn are the functional element sets of other system functions associated with the system function.

[0055] It can be understood that the tree structure is a directed graph, and multiple nodes of the tree structure can have an association relationship. In at least one embodiment of the present disclosure, the association relationship between multiple nodes in the tree structure is used to represent the execution order between multiple system functions of the software system. For example, the tree structure includes node A and node B, and there is an association relationship between node A and node B, and node A points to node B. In this case, the system function corresponding to node A is executed first, and then the system function corresponding to node B is executed.

[0056] In at least one embodiment of the present disclosure, a first artificial intelligence model analyzes demand information using a tree structure of a software system. For example, the first artificial intelligence model can perform intent recognition on the demand information to obtain an analysis result of the demand information, which can be used to characterize the user's intention to use the software system. Then, the first artificial intelligence model can match the analysis result of the demand information with the tree structure of the software system, and determine the nodes that match the analysis result of the demand information among the nodes included in the tree structure of the software system. Since the nodes in the tree structure are associated with each other, the first artificial intelligence model can generate an operation path that includes the nodes that match the analysis result of the demand information, thereby realizing automatic planning of the operation path.

[0057] For example, the first artificial intelligence model can identify the intention of the demand information and obtain the analysis results of the demand information. The analysis results of the demand information indicate that the intention to use the software system includes executing system function A, system function B and system function C. The first artificial intelligence model matches the analysis results of the demand information with the tree structure of the software system, and determines the node A1 corresponding to system function A, the node B1 corresponding to system function B and the node C1 corresponding to system function C in the tree structure of the software system. In the tree structure of the software system, the association relationship between node A1, node B1 and node C1 is "node A1 points to node C1, node C1 points to node B1". Therefore, the operation path is node A1-node C1-node B1, that is, first execute the system function A corresponding to node A1, then execute the system function C corresponding to node C1, and finally execute the system function B corresponding to node B1.

[0058] S203: Provide system functions corresponding to the operation path.

[0059] By providing system functions corresponding to the operation path, the function execution requirements indicated by the requirement information are automatically executed, and the relevant operations of the software system are completed without manual operation by the user.

[0060] In some possible implementations, since each node in the tree structure of the software system includes operation parameters related to the corresponding system function, it is possible to execute the system function corresponding to the requirement information of at least one software system based on the operation path according to the operation parameters of each node in the operation path.

[0061] That is, the system functions corresponding to the nodes of the operation path are executed in the execution order of the nodes included in the operation path. When each node of the operation path is executed, the corresponding system function is executed according to the operation parameters of the node.

[0062] In this way, since the operation parameters are stored in the nodes of the tree structure in advance, after the operation path is determined through the tree structure of the software system, the operation parameters of each node included in the operation path are used to execute the corresponding system functions without the need for other operations such as code encapsulation.

[0063] For example, an execution function may be pre-configured, and for a target node in an operation path, the operation parameters of the target node are used as input parameters, and the corresponding execution function is run to execute the system function corresponding to the target node.

[0064] Here, the target node can be understood as the node currently being executed in the operation path. Since the operation parameters can include the functional element set of the system function, the operation corresponding to each functional element and the parameters associated with the operation, the execution event involved in each functional element, and the functional element sets of other system functions associated with the system function, the system function corresponding to the target node can be executed by calling the execution function with the target node's operation parameters as input parameters.

[0065] For example, the execution function may include the execution codes of multiple candidate execution events. Through the execution event of each functional element in the operation parameters, according to the operations corresponding to each functional element and the parameters associated with the operations, the execution code corresponding to the execution event of each functional element is executed to complete the system function corresponding to the target node.

[0066] In some embodiments, the execution function can be a generalized system execution method function, i.e., a preconfigured execution function that matches the system functionality of each node. This means that the execution function can be applied to the operating parameters of each node, making it universally applicable. Regardless of the node included in the operation path, the operating parameters of that node can be used as input to execute the corresponding system functionality by calling the generalized system execution method function.

[0067] In the method for executing system functions of a software system provided in at least some embodiments of the present disclosure, a tree structure can be used to represent the execution order of the system functions themselves and between the system functions in the software system. The artificial intelligence model analyzes the demand information in combination with the tree structure to generate corresponding operation paths. There is no need for the artificial intelligence model to plan the execution order of each system function; and there is no need to perform code encapsulation for the system functions, thereby improving the applicability of automatically executing system functions.

[0068] In some embodiments, the user's demand information may also be related to multiple software systems, that is, the demand information may include instructions for the user's functional execution requirements of multiple software systems. For example, the user's demand information is "Help me organize a meeting arrangement document and then email it to user B." In this case, the user's demand information may be related to the document system and the email system.

[0069] In these embodiments, because users have functional execution requirements for multiple software systems, the first artificial intelligence model can analyze the tree structures of the multiple software systems. Specifically, the first artificial intelligence model analyzes the requirement information to determine the multiple software systems associated with the requirement information. Based on the tree structures corresponding to the multiple software systems, the first artificial intelligence model determines the multiple sub-operation paths corresponding to the multiple software systems. The sub-operation paths corresponding to the multiple software systems are sorted to determine the operation path corresponding to the requirement information.

[0070] That is to say, in the case where the demand information involves multiple software systems, the process of the first artificial intelligence model generating an operation path can be divided into three steps: first, the first artificial intelligence model determines the software system associated with the demand information; then, for each software system, the first artificial intelligence model determines the nodes involved in the demand information in the tree structure of the software system, and then generates the sub-operation path of the software system; finally, the first artificial intelligence model determines the execution order of each software system, sorts the sub-operation path of each software system, and generates the final overall operation path.

[0071] For example, the first artificial intelligence model determines that the software systems associated with the requirement information are software systems A and B. From the tree structure of software system A, the first artificial intelligence model determines that the nodes involved in the requirement information include nodes A1, A2, and A3, and that the sub-operation path corresponding to software system A is "node A1 - node A2 - node A3." From the tree structure of software system B, the first artificial intelligence model determines that the nodes involved in the requirement information include nodes B1, B2, B3, and B4, and that the sub-operation path corresponding to software system B is "node B1 - node B3 - node B4 - node B2." The first artificial intelligence model determines that the execution order of software systems A and B is "software system B first, then software system A." Consequently, the first artificial intelligence model generates the operation path "node B1 - node B3 - node B4 - node B2 - node A1 - node A2 - node A3."

[0072] In this way, even in the complex situation where the demand information involves multiple software systems, since each software system has a corresponding tree structure, the first artificial intelligence model only needs to determine the software systems involved in the demand information and the execution order between the software systems, that is, to execute the sorting of the software system dimension. There is no need to sort the software function dimension for multiple different software functions of multiple different software systems, which reduces the analysis complexity of the first artificial intelligence model and improves the analysis accuracy of the first artificial intelligence model.

[0073] In at least one embodiment of the present disclosure, the first artificial intelligence model can generate an operation path based on a prompt learning technique. In prompt learning, prompts are used to guide the artificial intelligence model's thinking. In other words, prompts are sent to the artificial intelligence model to tell it what content to generate next, thereby guiding the artificial intelligence model's generation direction.

[0074] For example, a target prompt word is generated, the prompt information is sent to the first artificial intelligence model, and the operation path corresponding to the demand information returned by the first artificial intelligence model is received.

[0075] For example, the target prompt words include: requirement information, a tree structure corresponding to the candidate software system, and prompt information including instructing the first artificial intelligence model to determine the software system associated with the requirement information, determine the operation paths corresponding to the software system associated with the requirement information, and determine the operation path corresponding to the requirement information.

[0076] That is to say, through the prompt ability of the target prompt word, the first artificial intelligence model can think according to the guidance of the target prompt word, first determine the software system associated with the demand information, then combine the tree structure of the software system associated with the demand information, and determine the operation paths corresponding to the software systems associated with the demand information. Finally, determine the operation path corresponding to the demand information, for example, sort the sub-operation paths corresponding to multiple software systems according to the execution order of multiple software systems, and then output the operation path corresponding to the demand information.

[0077] In this way, through prompt learning technology, the target prompt words are used to guide the first artificial intelligence model to generate the desired content. If the system function of the software system changes, only the tree structure in the target prompt words needs to be changed, and there is no need to adjust the structure of the first artificial intelligence model, thereby improving the application scenario of the first artificial intelligence model.

[0078] As described above, in the embodiment of the present disclosure, a tree structure is used to model the overall operation process of each system function in the software system. Therefore, a tree structure corresponding to at least one software system can also be obtained.

[0079] The following is an exemplary description of the tree structure and its construction process. Figure 3 A schematic diagram of a process for constructing a tree structure provided by at least one embodiment of the present disclosure is shown; Figure 4 A schematic diagram of a tree structure provided by at least one embodiment of the present disclosure is schematically shown.

[0080] like Figure 3 As shown, obtaining a tree structure corresponding to a target software system may include the following steps:

[0081] S301: Acquire multiple functional elements of a target software system in a software system.

[0082] S302: Generate multiple object nodes based on multiple functional elements.

[0083] S303: Constructing association relationships among the multiple object nodes according to the execution order among the multiple object nodes.

[0084] Here, the target software system can be understood as any software system in the at least one software system to be processed, and each of the multiple functional elements can represent an operation. In other words, the overall operation process of the target software system is disassembled according to the dimension of operation to obtain multiple functional elements. For example, the operation of "entering a username" in the target software system can correspond to a functional element, and the operation of "entering a password" can correspond to a functional element, and the operation of "clicking a login control" can correspond to a functional element.

[0085] Considering that one functional element corresponds to one operation, and a system function often involves multiple operations (i.e., multiple functional elements), multiple object nodes (i.e., nodes in the tree structure) are generated based on multiple functional elements, and the operations are integrated into system functions.

[0086] In some embodiments, a plurality of functional elements having an associated relationship among the functional elements are determined as an object node, thereby obtaining a plurality of object nodes. In other words, the plurality of functional elements are divided into object nodes based on the associated relationships between the functional elements, so that the functional elements included in an object node are closely associated with each other, thereby achieving reasonable and effective integration of system functional dimensions.

[0087] The functional elements with an associated relationship may include at least one of the following: functional elements corresponding to operations executed continuously, functional elements executed on the same operation page, and functional elements with a parameter transfer relationship.

[0088] Among them, the functional elements corresponding to operations performed continuously can be understood as functional elements corresponding to operations that are related in time sequence. For example, under normal circumstances, users will continuously perform the two operations of "entering user name" and "entering password". Therefore, the functional elements corresponding to the two operations of "entering user name" and "entering password" can be functional elements with an associated relationship.

[0089] The functional elements executed on the same operation page can be understood as the functional elements corresponding to the operations triggered on the same operation page. For example, the operation page is an email editing page, and the user can perform multiple operations such as "select recipients", "fill in email subjects", "add attachments", and "write email body" on the email editing page. Therefore, the functional elements corresponding to the multiple operations of "select recipients", "fill in email subjects", "add attachments", and "write email body" can be functional elements with an associated relationship.

[0090] Functional elements with a parameter passing relationship can be understood as functional elements related to the same parameters. For example, executing the operation of "clicking the login control" requires the use of the user name information entered in the operation of "entering the user name" and the password information entered in the operation of "entering the password". Therefore, the functional elements corresponding to the three operations of "clicking the login control", "entering the user name" and "entering the password" can be functional elements with an associated relationship.

[0091] After integrating multiple functional elements into multiple object nodes, the execution order of these object nodes is used to establish relationships between them. For example, if the execution order of object nodes A and B is first A, then B, then a relationship is established from A to B. This creates a tree representation of the overall operation of the target software system.

[0092] like Figure 4 As shown, the tree structure of the exemplary target software system includes multiple nodes, such as the object nodes "New Folder", "Rename New Folder", "Save New Folder", "Select Folder", "Rename Selected Folder", "Cancel Rename" and "Save Rename" shown in the figure, among which "New Folder" is associated with "Rename New Folder" and "Save New Folder" (that is, they are connected by arrows, the node corresponding to the starting position of the arrow is the node executed first, and the node pointed by the arrow is the node executed later), "Select Folder" is associated with "Rename Selected Folder", "Rename New Folder", "Rename Selected Folder" and "Cancel Rename", and "Rename New Folder", "Rename Selected Folder" and "Save Rename" are associated with each other.

[0093] In some possible implementations, multiple functional elements of the target software system can be obtained through video. For example, an operation video of the target software system is obtained, and element recognition is performed on the operation video to obtain multiple functional elements.

[0094] For example, an operation video includes the operation process of at least one system function in the target software system. In other words, the operation video can be understood as a video of operating the target software system. The embodiments of the present disclosure do not limit the method and form of obtaining the operation video. For example, the target software system can be manually operated and the process of manual operation of the target software system can be recorded to obtain the operation video of the target software system; for example, the process of operating the target software system can be simulated by a simulation tool to obtain the operation video of the target software system; for example, the operation video can also be a video of the display screen interface captured or recorded during the user's operation.

[0095] By performing element recognition on the operation video, each operation in the operation video is identified, and then multiple functional elements are obtained. In some possible implementations, image recognition is performed on the operation video to determine multiple operating elements associated with the multiple operations in the operation video, and text recognition is performed on the multiple operating elements to obtain text recognition results corresponding to the multiple operating elements. Based on the text recognition results, multiple functional elements corresponding to the multiple operations in the operation video are determined.

[0096] For example, an operating element can be understood as an operation-related component on an operation page. For example, an operating element can include one or more of a control, an input box, a drop-down box, a scroll bar, etc. Since the operation video shows the operation process on different operation pages of the target software system, the operating elements involved in the operation process of executing the system function of the target software system can be found by performing image recognition on each frame of the operation video.

[0097] Next, for each operating element, text recognition is performed to identify the text contained within the operating element, generating a text recognition result. For example, if the operating element is a control, the text recognition result may be the control name; if the operating element is an input box, the text recognition result may be an input item (such as a username or password); and if the operating element is a drop-down box, the text recognition result may be a selectable item.

[0098] In this way, based on the text recognition results, the operations performed by each operating element are judged, and then multiple functional elements corresponding to the multiple operations in the operation video are determined, thereby realizing the recognition of the functional elements.

[0099] In other possible implementations, the functional elements of the target software system can also be identified using an AI model. For example, an operation video is sent to a second AI model, which then returns the functional elements of the target software system. In this way, the video processing capabilities of the AI model are utilized to obtain multiple functional elements corresponding to multiple operations involved in the operation of the target software system. For example, the second AI model can be implemented in the same or similar manner as the first AI model.

[0100] Furthermore, considering that the system functions provided by the target software system can be relatively complex, it is difficult to fully and completely cover all system functions or operations of the target software system through a single operation video. Therefore, the operation video includes a first video and a second video, and the first video and the second video can include the operation process of at least one different system function in the target software system. In other words, different system functions of the target software system are presented through multiple operation videos.

[0101] In this case, a tree structure is constructed for each operation video (e.g., the first video and the second video described above), resulting in a tree structure for the first video and a tree structure for the second video. In response to the presence of nodes representing the same system function in the tree structure of the first video and the tree structure of the second video, the tree structure corresponding to the software system is updated based on the tree structure of the first video and the tree structure of the second video.

[0102] That is to say, considering that the operation processes of different system functions may have the same operations, the tree structures of nodes with the same system functions are merged to form a more complete tree structure, and a tree structure is used to comprehensively represent the overall operation status of the target software system.

[0103] Figure 5 A schematic diagram of a process for updating a tree structure provided by at least one embodiment of the present disclosure is schematically shown.

[0104] like Figure 5As shown, tree structure 501 may be the tree structure of the first video, and tree structure 501 includes node 1, node 2, node 3, node 4, and node 5. Tree structure 502 may be the tree structure of the second video, and tree structure 502 includes node 1, node 3, node 6, node 7, node 8, and node 9. Tree structures 501 and 502 include the same node 1 and node 3. Therefore, tree structures 501 and 502 are merged to obtain tree structure 503, and tree structure 503 includes node 1, node 2, node 3, node 4, node 5, node 6, node 7, node 8, and node 9. Moreover, the association relationship between each node in tree structure 503 is the same as the association relationship between each node in tree structure 501 and tree structure 502. In this way, multiple tree structures of the target software system are merged into one tree structure.

[0105] It is understood that before using the technical solutions disclosed in each embodiment of the present disclosure, the user should be informed of the type, scope of use, and usage scenarios of the personal information involved in the present disclosure (e.g., user profile features, historical conversation texts, etc.) and their authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations. For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. In this way, the user can independently choose whether to provide personal information to the software or hardware such as the electronic device, application, server, or storage medium that implements the technical solution of the present disclosure based on the prompt message.

[0106] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0107] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0108] It is understandable that the data involved in the technical solutions of the present disclosure (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0109] Combined with the above Figures 1 to 5 The method for executing the system function of the software system provided by the embodiment of the present disclosure is introduced in detail. The device and electronic device provided by the embodiment of the present disclosure will be introduced below with reference to the accompanying drawings.

[0110] Figure 6A schematic structural diagram of a device for executing system functions of a software system provided by at least one embodiment of the present disclosure.

[0111] like Figure 6 As shown, the apparatus 600 for executing the system functions of the software system in this embodiment includes an acquisition module 601, a determination module 602, and a provision module 603. For example, these units or modules can be implemented by hardware (e.g., circuit) modules or software modules, etc. The following embodiments are the same and will not be repeated here. For example, these units or modules can be implemented by a central processing unit (CPU), a general-purpose graphics processing unit (GPGPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a field programmable gate array (FPGA), or other forms of processing units with data processing capabilities and / or instruction execution capabilities, as well as corresponding computer instructions.

[0112] The acquisition module 601 is configured to: acquire user demand information, wherein the demand information includes: indicating the user's demand for function execution of at least one software system.

[0113] Determination module 602 is configured to: determine an operation path corresponding to the requirement information based on an analysis result of the requirement information by the first artificial intelligence model, wherein the operation path is determined based on a tree structure corresponding to at least one software system, the tree structure including a plurality of nodes, each of the plurality of nodes corresponding to a system function of the at least one software system and including an operation parameter related to the corresponding system function, the association relationship between the plurality of nodes of the tree structure representing the execution order between the plurality of system functions of the at least one software system, and the operation path including at least one node having an association relationship in the tree structure;

[0114] The providing module 603 is configured to provide the system function corresponding to the operation path.

[0115] Exemplarily, the acquisition module 601 can be configured to execute step S201 described above, and its specific implementation principle can refer to the relevant description of step S201. The determination module 602 can be configured to execute step S202 described above, and its specific implementation principle can refer to the relevant description of step S202. The provision module 603 can be configured to execute step S203 described above, and its specific implementation principle can refer to the relevant description of step S203, which will not be repeated here.

[0116] In at least one embodiment of the present disclosure, the providing module 603 is further configured to: execute system functions corresponding to the requirement information of at least one software system based on the operation path according to the operation parameters of each node in the operation path.

[0117] In at least one embodiment of the present disclosure, the providing module 603 is further configured to: for a target node in the operation path, use the operation parameters of the target node as input parameters, run the corresponding execution function to execute the system function corresponding to the target node.

[0118] In at least one embodiment of the present disclosure, the demand information includes: indicating the user's demand for the function execution of multiple software systems; the determination module 602 is further configured to: use the first artificial intelligence model to analyze the demand information, determine the multiple software systems associated with the demand information, based on the tree structure corresponding to the multiple software systems, determine the multiple sub-operation paths corresponding to the multiple software systems, sort the sub-operation paths corresponding to the multiple software systems, and determine the operation path corresponding to the demand information.

[0119] In at least one embodiment of the present disclosure, the determination module 602 is further configured to: generate a target prompt word, wherein the target prompt word includes: requirement information, a tree structure corresponding to the candidate software system, and prompt information including instructing the first artificial intelligence model to determine the software system associated with the requirement information, determine the operation paths corresponding to the software systems associated with the requirement information, and determine the operation path corresponding to the requirement information; send the prompt information to the first artificial intelligence model, and receive the operation path corresponding to the requirement information returned by the first artificial intelligence model.

[0120] In at least one embodiment of the present disclosure, the apparatus 600 for executing the system function of the software system may further include a tree structure acquisition module, where the tree structure acquisition module is configured to acquire a tree structure corresponding to at least one software system.

[0121] In at least one embodiment of the present disclosure, the tree structure acquisition module is further configured to: acquire multiple functional elements of a target software system in at least one software system, wherein each of the multiple functional elements represents an operation; generate multiple object nodes based on the multiple functional elements; and construct an association relationship between the multiple object nodes based on the execution order between the multiple object nodes.

[0122] In at least one embodiment of the present disclosure, the tree structure acquisition module is further configured to: determine a functional element having an association relationship among the multiple functional elements as an object node, to obtain multiple object nodes.

[0123] In at least one embodiment of the present disclosure, the functional elements having an associated relationship include at least one of the following: functional elements corresponding to operations executed continuously; functional elements executed on the same operation page; and functional elements having a parameter transfer relationship.

[0124] In at least one embodiment of the present disclosure, the tree structure acquisition module is further configured to: obtain an operation video of the target software system, wherein the operation video includes the operation process of at least one system function in the target software system; perform element recognition on the operation video to obtain multiple functional elements.

[0125] In at least one embodiment of the present disclosure, the tree structure acquisition module is further configured to: perform image recognition on the operation video to determine multiple operating parts associated with the multiple operations in the operation video; perform text recognition on the multiple operating parts to obtain text recognition results corresponding to the multiple operating parts; and determine multiple functional elements corresponding to the multiple operations in the operation video based on the text recognition results.

[0126] In at least one embodiment of the present disclosure, the operation video includes a first video and a second video, and the first video and the second video include the operation process of at least one different system function in the target software system; the tree structure acquisition module is also configured to: in response to the existence of nodes representing the same system function in the tree structure of the first video and the tree structure of the second video, update the tree structure corresponding to the software system according to the tree structure of the first video and the tree structure of the second video.

[0127] It should be noted that for the sake of clarity and brevity, the embodiments of the present disclosure do not illustrate all components of the apparatus 600 for executing the system functions of the software system. To implement the necessary functions of the apparatus for executing the system functions of the software system, those skilled in the art may provide and configure other components not shown as needed, and the embodiments of the present disclosure are not limited thereto.

[0128] The present disclosure also provides an electronic device. Figure 6 The functions of the device 600 for executing system functions of the software system in the illustrated embodiment.

[0129] Figure 7 An exemplary structural diagram of an electronic device 700 is provided. Figure 7 As shown, electronic device 700 includes bus 701, processor 702, communication interface 703 and memory 704. Processor 702, memory 704 and communication interface 703 communicate with each other via bus 701.

[0130] The bus 701 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7Only one line segment is used to represent the bus, but it does not mean that there is only one bus or one type of bus.

[0131] The processor 702 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0132] The communication interface 703 is used for communicating with the outside, for example, the communication interface 703 can be used for communicating with a terminal.

[0133] The memory 704 may include volatile memory, such as random access memory (RAM), or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0134] The memory 704 stores executable codes, and the processor 702 executes the executable codes to perform the aforementioned method for executing the system functions of the software system.

[0135] Specifically, in the implementation Figure 6 In the case of the embodiment shown, and Figure 6 When each module or unit of the apparatus 600 for executing the system function of the software system described in the embodiment is implemented by software, Figure 6 The software or program code required for the functions of each module / unit in the system may be partially or completely stored in the memory 704. The processor 702 executes the program code corresponding to each unit stored in the memory 704 and performs the method for executing the system functions of the software system.

[0136] The present disclosure also provides a non-transitory computer-readable storage medium. This computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, hard disk, or magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). This computer-readable storage medium includes instructions that instruct the computing device to execute the method for executing a system function of a software system described in the apparatus 600 for executing a system function of a software system.

[0137] The present disclosure also provides a computer program product containing instructions. This computer program product can be software or a program product containing instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to execute a method for executing system functions of a software system.

[0138] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the present disclosure is not limited to technical solutions formed by specific combinations of the aforementioned technical features. It also encompasses other technical solutions formed by any combination of the aforementioned technical features or their equivalents, without departing from the scope of the above disclosure. For example, a technical solution formed by replacing the aforementioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0139] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0140] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A method for executing a system function of a software system, the method comprising: Acquiring user demand information, wherein the demand information includes: indicating the user's demand for at least one function execution of the software system; Determining an operation path corresponding to the requirement information based on an analysis result of the requirement information by the first artificial intelligence model, wherein the operation path is determined based on a tree structure corresponding to at least one of the software systems, the tree structure including a plurality of nodes, each of the plurality of nodes corresponding to a system function of at least one of the software systems and including an operation parameter related to the corresponding system function, the association relationship between the plurality of nodes of the tree structure representing an execution order between the plurality of system functions of the at least one software system, and the operation path including at least one node having an association relationship in the tree structure; Provides system functions corresponding to the operation path.

2. The method according to claim 1, wherein The system function corresponding to the operation path is provided, including: According to the operation parameters of each node in the operation path, based on the operation path, at least one system function of the software system corresponding to the requirement information is executed.

3. The method according to claim 2, wherein: The step of executing, based on the operation path and according to the operation parameters of each node in the operation path, a system function of at least one software system corresponding to the requirement information includes: For the target node in the operation path, the operation parameter of the target node is used as an input parameter, and the corresponding execution function is run to execute the system function corresponding to the target node.

4. The method according to claim 1, wherein The demand information includes: indicating the user's demand for function execution of the plurality of software systems; The determining the operation path corresponding to the demand information based on the analysis result of the demand information by the first artificial intelligence model includes: Utilize the first artificial intelligence model to analyze the demand information, determine the multiple software systems associated with the demand information, determine the multiple sub-operation paths corresponding to the multiple software systems based on the tree structure corresponding to the multiple software systems, sort the sub-operation paths corresponding to the multiple software systems, and determine the operation path corresponding to the demand information.

5. The method according to claim 1, wherein The determining the operation path corresponding to the demand information based on the analysis result of the demand information by the first artificial intelligence model includes: generating a target prompt word, wherein the target prompt word includes: the requirement information, a tree structure corresponding to candidate software systems, and prompt information instructing the first artificial intelligence model to determine software systems associated with the requirement information, determine operation paths corresponding to the software systems associated with the requirement information, and determine the operation path corresponding to the requirement information; The prompt information is sent to the first artificial intelligence model, and the operation path corresponding to the requirement information returned by the first artificial intelligence model is received.

6. The method according to any one of claims 1 to 5, further comprising: A tree structure corresponding to at least one of the software systems is obtained.

7. The method according to claim 6, wherein: The acquiring of a tree structure corresponding to at least one of the software systems includes: Acquire multiple functional elements of a target software system in at least one of the software systems, wherein each of the multiple functional elements represents an operation; generating a plurality of object nodes according to the plurality of functional elements; and An association relationship between the multiple object nodes is constructed according to the execution order between the multiple object nodes.

8. The method according to claim 7, wherein: Generating a plurality of object nodes according to the plurality of functional elements includes: The functional elements having an associated relationship among the multiple functional elements are determined as an object node to obtain a plurality of the object nodes.

9. The method according to claim 8, wherein The functional elements having an associated relationship include at least one of the following: Functional elements corresponding to operations performed continuously; Functional elements executed on the same operation page; Functional elements with parameter passing relationships.

10. The method according to claim 7, wherein: The acquiring of multiple functional elements of a target software system in at least one of the software systems comprises: Acquiring an operation video of the target software system, wherein the operation video includes an operation process of at least one system function in the target software system; Element recognition is performed on the operation video to obtain multiple functional elements.

11. The method according to claim 10, wherein: The operation video is subjected to element recognition to obtain multiple functional elements, including: performing image recognition on the operation video to determine a plurality of operating elements associated with a plurality of operations in the operation video; Performing text recognition on the multiple operating elements to obtain text recognition results corresponding to the multiple operating elements; Based on the text recognition result, multiple functional elements corresponding to the multiple operations in the operation video are determined.

12. The method according to claim 10, wherein: The operation video includes a first video and a second video, wherein the first video and the second video include the operation process of at least one different system function in the target software system; The method further comprises: In response to the presence of nodes representing the same system function in the tree structure of the first video and the tree structure of the second video, the tree structure corresponding to the software system is updated according to the tree structure of the first video and the tree structure of the second video.

13. A device for executing a system function of a software system, the device comprising: An acquisition module is configured to: acquire user demand information, wherein the demand information includes: indicating the user's demand for executing at least one function of the software system; a determination module configured to: determine an operation path corresponding to the requirement information based on an analysis result of the requirement information by a first artificial intelligence model, wherein the operation path is determined based on a tree structure corresponding to at least one of the software systems, the tree structure comprising a plurality of nodes, each of the plurality of nodes corresponding to a system function of at least one of the software systems and comprising an operation parameter related to the corresponding system function, the association relationship between the plurality of nodes of the tree structure representing an execution order between the plurality of system functions of the at least one software system, and the operation path comprising at least one node having an association relationship in the tree structure; The providing module is configured to provide the system function corresponding to the operation path.

14. An electronic device comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 12.

15. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 12.

Images