Information processing method and device, equipment, storage medium and program product

The method and device facilitate efficient configuration and testing of machine learning model call nodes by presenting and testing prompt words, enhancing application development efficiency and quality.

CN119088435AInactive Publication Date: 2024-12-06BEIJING ZITIAO NETWORK TECH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411132229.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-18
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Developers face challenges in efficiently configuring and testing configuration information, such as prompt words, for machine learning model call nodes in application development, which affects the editing and testing efficiency and quality of applications.

Method used

An information processing method and device that present configuration information for model call nodes, allowing developers to intuitively test and adjust prompt words based on model outputs, thereby improving the editing and testing efficiency of machine learning model call nodes.

Benefits of technology

Enhances the configuration and testing efficiency of prompt words for machine learning models, leading to improved application development quality and performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088435A_ABST
    Figure CN119088435A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides an information processing method and device, equipment, a storage medium and a program product. The method comprises: in response to a configuration request for a model calling node in a workflow, presenting first configuration information of the model calling node, the model calling node being configured to execute a task by using a machine learning model, the first configuration information comprising at least one first prompt for providing to the machine learning model; in response to an operation instruction for the model calling node, obtaining a first model output by utilizing a first machine learning model based on the at least one first prompt; and presenting a first operation result of the model calling node determined based on the first model output. According to the embodiment of the invention, the configuration efficiency and the test efficiency of the model calling node in the workflow can be improved, so that the operation effect of the application can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and more particularly, to information processing methods, apparatuses, devices, computer-readable storage media, and computer program products. Background Art

[0002] With the development of information technology, various terminal devices can provide various services to people in work and life. Applications that provide services can be deployed in terminal devices. Terminal devices present corresponding content and interact with users through the user interface of the application to meet various needs of users. In some cases, developers may want to develop applications simply and quickly. Therefore, how to reduce the difficulty of application development has become a concern. Summary of the invention

[0003] In a first aspect of the present disclosure, an information processing method is provided. The method includes: in response to a configuration request for a model call node in a workflow, presenting first configuration information of the model call node, the workflow includes one or more model call nodes, the model call node is configured to perform a task using a machine learning model, and the first configuration information includes at least one first prompt for providing to the machine learning model; in response to an operation instruction for the model call node, using the first machine learning model based on the at least one first prompt to obtain a first model output; and presenting a first operation result of the model call node determined based on the first model output.

[0004] In a second aspect of the present disclosure, a device for information processing is provided. The device includes: a configuration module configured to present first configuration information of a model call node in response to a configuration request for a model call node in a workflow, wherein the workflow includes one or more model call nodes, the model call node is configured to perform a task using a machine learning model, and the first configuration information includes at least one first prompt for providing to the machine learning model; an acquisition module configured to obtain a first model output using a first machine learning model based on at least one first prompt in response to an operation instruction for the model call node; and a presentation module configured to present a first operation result of the model call node determined based on the first model output.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the electronic device executes the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the medium, and when the computer program is executed by a processor, the method of the first aspect is implemented.

[0007] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method according to the first aspect of the present disclosure is implemented.

[0008] It should be understood that the content described in this section is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented;

[0011] Figure 2 A flowchart showing a process of information processing according to some embodiments of the present disclosure;

[0012] Figure 3 A schematic diagram showing an example of a workflow orchestration interface according to some embodiments of the present disclosure;

[0013] Figure 4 A schematic diagram showing an example of a configuration interface of a model call node according to some embodiments of the present disclosure;

[0014] Figure 5A and Figure 5B Schematic diagrams of examples of integrated development environments according to some embodiments of the present disclosure are respectively shown;

[0015] Figure 6 A schematic structural block diagram of an apparatus for information processing according to some embodiments of the present disclosure is shown; and

[0016] Figure 7 A block diagram of an electronic device in which one or more embodiments of the present disclosure may be implemented is shown. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0018] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0019] Herein, unless explicitly stated, executing a step “in response to A” does not mean executing the step immediately after “A” but may include one or more intermediate steps.

[0020] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition, use, storage or deletion of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0021] It is understandable that before using the technical solutions disclosed in the various embodiments of the present disclosure, the types, scopes of use, usage scenarios, etc. of the information involved in the present disclosure should be informed to relevant users and their authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations. The relevant users may include any type of right holders, such as individuals, enterprises, and groups.

[0022] For example, in response to receiving an active request from a user, a prompt message is sent to the relevant user to clearly prompt the relevant user that the operation requested to be performed will require obtaining and using the information of the relevant user, so that the relevant user can independently choose whether to provide information to software or hardware such as an electronic device, application, server or storage medium that executes the operation of the technical solution of the present disclosure based on the prompt message.

[0023] As an optional but non-limiting implementation, in response to receiving an active request from a relevant user, a prompt message is sent to the relevant user, for example, in the form of a pop-up window, in which the prompt message may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to select "agree" or "disagree" to provide information to the electronic device.

[0024] It is understandable that the above notification and the process of obtaining user authorization are only illustrative and do not limit the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure. The activation of the digital assistant related functions, the acquired data, the processing and storage of the data, etc. in the embodiment of the present disclosure shall obtain the prior authorization of the user and other rights holders associated with the user, and shall comply with the relevant laws and regulations and the agreement rules between the rights holders.

[0025] As used herein, the term "model" can learn the association between the corresponding input and output from the training data, so that after the training is completed, the corresponding output can be generated for a given input. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multi-layer processing units. A neural network model is an example of a model based on deep learning. In this article, "model" may also be referred to as "machine learning model", "learning model", "machine learning network" or "learning network", and these terms are used interchangeably in this article.

[0026] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. Environment 100 involves an application management platform 110, which can support the creation of applications and / or the running of applications. In some embodiments, the portion of the application management platform 110 that supports the creation of applications can also be referred to as an application creation portion. In some embodiments, the portion of the application management platform 110 that supports the running of applications can also be referred to as an application running portion.

[0027] As shown in the figure, the application creation part can provide a user 105 with an environment for creating and publishing applications. User 105 can be referred to as an application creation user or creator. In some embodiments, the application creation part can be a low-code platform that provides a collection of tools for application creation. The application creation part can support visual development of various types of applications, so that developers can skip the process of manual coding and speed up the development cycle and cost of applications. The application creation part can support any appropriate platform for users to develop one or more types of applications, for example, it can include a platform based on application platform as a service (aPaaS). Such a platform can support users to efficiently develop applications and realize operations such as application creation and application function adjustment.

[0028] The application creation part can be deployed locally on the terminal device of user 105, and / or can be supported by a server-side device. For example, the terminal device of user 105 can run a client of the application creation part, which can support the interaction between the user and the application creation part provided by the server. In the case where the application creation part runs locally on the user's terminal device, the user 105 can directly interact with the local application creation part using the terminal device. In the case where the application creation part runs on the server-side device, the server-side device can, based on the communication connection with the terminal device, implement service provision for the client running in the terminal device. The application creation part can present a corresponding page 130 to the user 105 based on the operation of the user 105, so as to output to the user 105 and / or receive from the user 105 information related to application creation.

[0029] In some embodiments, the application creation part can be associated with a corresponding database, which stores the data or information required for the application creation process supported by the application creation part. For example, the database can store the code and description information corresponding to each functional module that constitutes the application. The application creation part can also perform operations such as calling, adding, deleting, and updating the functional modules in the database. The database can also store operations that can be performed on different functional blocks. Exemplarily, in a scenario where an application is to be created, the application creation part can call the corresponding functional blocks from the database to build the application.

[0030] In an embodiment of the present disclosure, a user 105 can create a target application 120 as needed on the application creation part and publish the target application 120. The target application 120 can be published to any appropriate application running part, as long as the application running part can support the operation of the target application 120. After publishing, the target application 120 can be used to be operated by one or more terminal users 145. The terminal user 145 can operate the target application 120 through an associated terminal device 146, and then interact with the application management platform 110. The terminal user 145 can be referred to as the terminal user of the target application 120. In some embodiments, the target application 120 may include or be implemented as a digital assistant 122.

[0031] The digital assistant 122 can be configured to have the ability of intelligent dialogue. In the example shown in the figure, the digital assistant 122 can be integrated into the target application 120, as part of the target application 120 to assist in the task processing within the target application 120. In other examples, the digital assistant 122 can be configured as an independently running application, such as a web application or other types of applications. In such an example, the digital assistant 122 and the target application 120 can be regarded as the same application. The digital assistant 122 is provided to assist users in various task processing requirements in different applications and scenarios. In the interaction process with the digital assistant 122, the user inputs an interactive message, and the digital assistant 122 provides a reply message in response to the user input. Typically, the digital assistant 122 can support users to input questions in a natural language, and perform tasks and provide replies based on the understanding of natural language input and logical reasoning ability.

[0032] In some embodiments, digital assistant 122 can interact with end user 145 as a contact of end user 145. For example, digital assistant 122 can be implemented in an instant messaging (IM) application. Digital assistant 122 can interact with end user 145 in a single chat session with end user 145. In some embodiments, digital assistant 122 can interact with multiple users in a group chat session including multiple users.

[0033] For each terminal user 145, the client of the application running part can present an interactive window 142 of the target application 120 or the digital assistant 122 in the client interface, such as a conversation window with the digital assistant 122. The terminal user 145 can enter a conversation message in the conversation window, and the target application 120 can determine the reply message of the digital assistant 122 based on the created configuration information and present it to the user in the interactive window 142. In some embodiments, depending on the configuration of the target application 120, the interactive message with the target application 120 may include a message in a multimodal form, such as a text message (e.g., a natural language text), a voice message, an image message, a video message, and the like.

[0034] Similar to the application creation part, the application running part can be deployed locally on the terminal device of each terminal user 145, and / or can be supported by the server device. For example, the terminal device of the terminal user 145 can run a client with the application running part, and the client can support the interaction between the user and the application running part provided by the server. In the case where the application running part runs locally on the user's terminal device, the terminal user 145 can directly use the terminal device to interact with the local application running part. In the case where the application running part runs on the server device, the server device can realize the service supply to the client running in the terminal device based on the communication connection with the terminal device. The application running part can present the corresponding application page to the terminal user 145 based on the operation of the terminal user 145, so as to output and / or receive information related to the use of the application to the terminal user 145.

[0035] In some embodiments, the implementation of at least part of the functions of the target application 120 and / or the implementation of at least part of the functions of the digital assistant 122 in the target application 120 can be implemented based on the model. During the creation or operation of the target application 120, one or more models 155, such as the capabilities of the model 155, can be called. In the target application 120, the digital assistant 122 can use the model 155 to understand the user input and provide a reply to the user based on the output of the model 155.

[0036] During the creation process, the application management platform 110 needs to use the model 155 to determine whether the operation result of the target application 120 meets expectations when testing the target application 120. During the operation process, in response to different operation requests of the user of the target application 120, the application operation part may need to use the model 155 to determine the response result to the user.

[0037] Although shown as being independent of the application management platform 110, one or more models 155 may run on the application management platform 110, or other remote servers. In some embodiments, the model 155 may be a machine learning model, a deep learning model, a learning model, a neural network, etc. In some embodiments, the model may be based on a language model (LM). The language model can have question-answering capabilities by learning from a large amount of corpus. The model 155 may also be based on other appropriate models.

[0038] The application management platform 110 can run on appropriate electronic devices. The electronic devices here can be any type of device with computing capabilities, including terminal devices or server devices. The terminal device can be any type of mobile terminal, fixed terminal or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication systems (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio broadcast receivers, e-book devices, gaming devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. The server device can include, for example, a computing system / server, such as a mainframe, an edge computing node, a computing device in a cloud environment, and the like. In some embodiments, the management platform 110 can be implemented based on cloud services.

[0039] It should be understood that the structure and functionality of the environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure. For example, although a single user interacting with the application creation portion and a single user interacting with the application execution portion are shown, in reality multiple users can access the application management platform 110 to each create a digital assistant, and each digital assistant can be used to interact with multiple users.

[0040] As mentioned above, applications that provide services can be deployed in terminal devices. Traditionally, developers need to have high software development capabilities to develop an application with high stability and high availability. Usually, technicians with professional software development skills are required to complete the development of the application.

[0041] With the development of machine learning technology, its application in terminal device scenarios is becoming more and more extensive. During the operation of some applications, one or more machine learning models are called to provide services to users. This application method reduces the difficulty of application development to a certain extent, allowing more developers to participate in the application development process.

[0042] For nodes that execute machine learning model calls, the configuration information of the nodes needs to be configured, debugged, and evaluated during the development phase. The editing quality of the configuration information is directly related to the performance of the node. In particular, the prompt words provided to the machine learning model will directly affect the quality of the response content of the machine learning model. Traditionally, the editing quality of configuration information such as prompt words is usually judged based on the test results at the application level. This is not conducive to improving the editing efficiency, testing efficiency, and editing quality of configuration information such as prompt words.

[0043] In view of this, an embodiment of the present disclosure provides an improved scheme for information processing. In this scheme, in response to a configuration request for a model call node in a workflow, first configuration information of the model call node is presented, the workflow includes one or more model call nodes, the model call node is configured to perform tasks using a machine learning model, and the first configuration information includes at least one first prompt for providing to the machine learning model. In response to an operation instruction for the model call node, a first machine learning model is used based on at least one first prompt to obtain a first model output. A first operation result of the model call node determined based on the first model output is presented.

[0044] In the embodiments of the present disclosure, the configuration information such as prompt words of the model call node in the workflow can be edited, and the model call node can be run. The machine learning model is called based on the configuration information such as prompt words to obtain the running result of the model call node. In this way, the quality of the configuration information such as prompt words can be intuitively determined based on the running result, which is conducive to improving the configuration efficiency and testing efficiency of the model call node, thereby improving the efficiency of application development. This in turn is conducive to improving the running effect after the application is deployed.

[0045] Some example embodiments of the present disclosure will be described in detail below with reference to examples of the accompanying drawings.

[0046] The information processing process described in the embodiments of the present disclosure may be implemented in an application management platform, a terminal device installed with the application management platform, and / or a server corresponding to the application management platform. In the following examples, for the purpose of discussion, the description is from the perspective of the application management platform, for example Figure 1 The application management platform 110 shown. The user interface presented by the application management platform 110 can be presented via the terminal device of the user 145, and the application management platform 110 can receive user input via the terminal device of the user 145. In some embodiments of the present disclosure, the user 145 is the terminal user of the target application 120. It should be understood that the user interface presented by the application management platform 110 can also be presented via the terminal device of the user 105, and the application management platform 110 can also receive user input via the terminal device of the user 105. In some embodiments of the present disclosure, the user 105 is the creator, manager or maintainer of the target application 120.

[0047] Figure 2 1 is a flowchart of a process 200 of information processing according to some embodiments of the present disclosure. The process 200 may be implemented in the application management platform 110, for example, by the application creation part of the application management platform 110. Figure 1 To describe Figure 2 Process 200 is shown.

[0048] In block 210 of process 200, the application management platform 110 presents first configuration information of the model call node in response to a configuration request for the model call node in the workflow. The workflow here may include one or more model call nodes. Of course, the multiple working nodes in the workflow are not limited to including model call nodes, but may also include other nodes. For example, input nodes, nodes for specific data processing, function call nodes, interface call nodes, record nodes, logic branch nodes, output nodes, etc. In the actual arrangement of the workflow, different types of working nodes can be selected according to actual needs.

[0049] The model calling node is configured to perform tasks using a machine learning model. Alternatively or additionally, the model calling node can be configured to generate a predetermined type of content by calling a machine learning model. The predetermined type of content here may include, but is not limited to, text, images, audio, video, program code, structured data with a specific structure, and the like. Taking program code as an example, a predetermined type of program code may include, for example, data query instructions (such as database query instructions written in structured query language), data processing instructions (such as data processing instructions written in a programming language), and the like. Of course, the above content is only exemplary, and those skilled in the art may choose to call the content generated by the model according to actual needs.

[0050] The first configuration information includes at least one first prompt for providing to the machine learning model. The at least one first prompt may include one first prompt, or may include multiple first prompts. Exemplarily, the at least one first prompt may include at least one of the following: a system prompt, one or more user prompts, and one or more assistant prompts. The system prompt may be selected from a plurality of predetermined system prompts provided by the application management platform 110. A user (e.g., an application creator) may set one or more user prompts according to actual needs. A user may set one or more assistant prompts according to actual needs. Only one of the user prompt and the assistant prompt may be set, or both the user prompt and the assistant prompt may be set. For example, the user prompt and the assistant prompt may be set in a one-to-one or one-to-many relationship. Of course, the specific content of the above-mentioned first prompt and the setting method of the first prompt are only exemplary. In actual applications, an appropriate prompt may be selected as the first prompt according to implementation needs.

[0051] Alternatively or additionally, the first configuration information may also include at least one of the following: information for specifying a machine learning model, one or more input parameters of a model call node, and one or more output parameters of a model call node. The information for specifying a machine learning model may include, for example, the name, model, interface, address, and other information that can indicate a machine learning model. The one or more input parameters of a model call node include one or more parameters (also called variables) that need to be input into the model call node when running the model call node. For example, the one or more input parameters may include "Target", "Input", and "Context", etc. The one or more output parameters of a model call node may be used to indicate the output content of the model call node, the type of the output content, the data structure of the output content, and the like. For example, the one or more output parameters may include a parameter for indicating the data structure of the output content, and the parameter value of the parameter may include "String", indicating that the output data structure of the model call node is an output content of the "String" type. It is understood that the above-mentioned input parameters and output parameters are only exemplary. The input parameters and output parameters of a model call node are directly related to the tasks performed by the model call node. When the tasks performed by the model call node are different, the input parameters and output parameters of the model call node may also be different.

[0052] In some embodiments, the application management platform 110 may present corresponding first configuration information of each model calling node in response to a configuration request for each model calling node in two or more model calling nodes in the workflow.

[0053] To illustrate the configuration process of the model call node, the following will combine Figures 3 to 5B Explain the configuration process of the model call node. Figures 3 to 5B Schematic diagrams of example interfaces 300 to 500B (also referred to as examples 300 to 500B) according to some embodiments of the present disclosure are shown. It should be understood that the interfaces shown in the drawings are merely examples, and various interface designs may actually exist. The various graphical elements in the interface may have different arrangements and different visual representations, one or more of which may be omitted or replaced, and one or more other elements may also exist. The embodiments of the present disclosure are not limited in this respect.

[0054] For example, Figure 3 As shown, example 300 shows an example of a workflow orchestration interface. The application management platform 110 can respond to a workflow orchestration request by presenting a workflow orchestration interface such as Figure 3The workflow arrangement interface shown in the figure. The workflow arrangement interface may include a control for adding a work node. The application management platform 110 may present a candidate work node interface in response to a trigger operation on the add control. The candidate work node interface may present multiple candidate work nodes. The user may add one or more candidate work nodes to the workflow arrangement interface by, for example, clicking, dragging, selecting, etc., to form work nodes 301, work nodes 302, work nodes 303, etc. as shown in example 300.

[0055] The user can also add connecting lines between work nodes to indicate the relationship between work nodes, such as time sequence relationship, logical relationship, data connection relationship, etc. The connecting lines between work nodes can form a flowchart, which can indicate a workflow composed of multiple work nodes. As shown in example 300, connecting lines 304, 305, 306, 307, etc. can be added between work nodes, and work nodes 301, 302, 303 and connecting lines 304, 305, 306, 307 together form a flowchart that can identify a workflow 310.

[0056] One or more of the working nodes 301, 302, and 303 may be model call nodes. For example, the working nodes 301, 302, and 303 may all be model call nodes, and the application management platform 110 may present first configuration information of the corresponding model call node in response to a configuration operation on any of the model call nodes.

[0057] In some embodiments, the application management platform 110 may present a configuration interface of the model calling node in response to a configuration request for the model calling node. At least a portion of the first configuration information for the model calling node is received using the configuration interface. Afterwards, the application management platform 110 may present the first configuration information and the operation result presentation area in response to the debugging request for the model calling node.

[0058] For example, the working node 302 may be a model calling node. In response to receiving a configuration request for the working node 302, the application management platform 110 may present the following: Figure 4 The configuration interface is shown in the figure. Figure 4As shown, example 400 shows an example of a configuration interface of a model call node. Example 400 includes prompt word input entries 401 and 402, and the application management platform 110 can receive system prompt words input by the user through the prompt word input entry 401. For example, the user can select one of multiple predetermined system prompt words and input it into the prompt word input entry 401. The application management platform 110 can accept user prompt words input by the user through the prompt word input entry 402. Example 400 also includes a prompt word adding control 403, and the application management platform 110 can add a prompt word input entry to the configuration interface in response to a trigger operation on the prompt word adding control 403. The user can input, for example, user prompt words or other prompt words to the configuration interface through the added prompt word input entry. For example, when the prompt word input entries 401 and 402 have been presented, the application management platform 110 can present a prompt word input entry for inputting assistant prompt words in response to a trigger operation of the prompt word adding control 403 to receive assistant prompt words input by the user.

[0059] Example 400 may also include an input parameter configuration entry 404. The input parameter configuration entry 404 may include input parameters (in Figure 4 The application management platform 110 may receive the parameter name and parameter value of the input parameter through the input box. For example, the user may configure three input parameters with parameter names of "Target", "Input" and "Context" through the input parameter configuration entry 404. The parameter value may be preset or default, or may come from a previous node in the workflow. Figure 4 In the example above, the parameter values ​​come from the Start Node.

[0060] Example 400 may also include a model configuration entry 405. The application management platform 110 may receive information specifying a machine learning model through the model configuration entry 405. For example, a user may select a machine learning model from a plurality of candidate machine learning models provided by the model configuration entry 405 as a model (e.g., a first machine learning model) to be called when the model calling node is running.

[0061] Example 400 may also include an output parameter configuration entry 406. The application management platform may receive information indicating the type of output content through the output parameter configuration entry 406. For example, a user may input "String" into the output parameter configuration entry 406 to instruct the model call node to output content whose output data structure is "String". In this way, the application management platform 110 may receive at least a portion of the first configuration information through at least one of the prompt word input entries 401, 402, the input parameter configuration entry 404, the model configuration entry 405, and the output parameter configuration entry 406. Of course, the above-mentioned entry for inputting the first configuration information is only exemplary. In a specific implementation, any appropriate entry or control may be selected to receive the configuration information according to actual needs or different configuration information.

[0062] Example 400 may also include an integrated development environment portal 407. In response to receiving a trigger operation on the integrated development environment portal 407, the application management platform 110 may determine that a debugging request for a model call node (eg, the working node 302) is received and enter an integrated development environment (IDE).

[0063] like Figure 5A As shown, example 500A shows an example of an integrated development environment. The integrated development environment of example 500A can present a debugging interface 510 of a model call node. The debugging interface 510 can include a configuration area 511 and a test area 512. The configuration area 511 can be arranged with, for example, prompt word input entrances 401, 402, model configuration entrance 405, output parameter configuration entrance 406 and other entrances for inputting configuration information. In this way, the application management platform can not only use the configuration area 511 to present at least part of the first configuration information, but also use the configuration area to receive at least part of the first configuration information. The test area 512 may include an operation result presentation area 514, which is used to present the operation result of the model call node.

[0064] In block 220 of process 200, the application management platform 110 utilizes the first machine learning model based on at least one first prompt in response to the run instruction for the model call node to obtain a first model output. Alternatively or additionally, the application management platform 110 may determine the first model input based on at least one first prompt in response to the run instruction for the model call node. The first model input may be provided to the first machine learning model specified by the first configuration information to obtain a first model output of the first machine learning model output.

[0065] For example, Figure 5AAs shown, the integrated development environment shown in example 500A may be provided with a running control 501. The application management platform 110 may determine that a running instruction for a model call node is received in response to receiving a trigger operation on the running control 501. For example, a target prompt word for providing to the first machine learning model may be determined based on multiple prompt words such as a system prompt word and a user prompt word.

[0066] In some embodiments, the application management platform 110 may determine one or more input parameters for the model call node based on the received user input. The application management platform 110 may determine a first model input based on the one or more input parameters and at least one first prompt. Thereafter, the first model input is provided to the first machine learning model to obtain a first model output.

[0067] Exemplarily, the test area 512 may include a debugging parameter input entry 513. The debugging parameter input entry 513 may include an input box for the input parameters of the model call node. For example, the user configures three input parameters with parameter names of "Target", "Input" and "Context" through the input parameter configuration entry 404. The debugging parameter input entry 513 may include input boxes for the "Target", "Input" and "Context" input parameters. The application management platform 110 may use the input box to receive user input. Afterwards, the parameter values ​​of the input parameters such as "Target", "Input" and "Context" are determined based on the received user input. Afterwards, the first model input is determined based on the parameter values ​​of the input parameters such as "Target", "Input" and "Context", as well as system prompt words, user prompts, etc.

[0068] In block 230 of process 200, the application management platform 110 presents a first running result of the model call node determined based on the first model output. Alternatively or additionally, after receiving the first model output, the application management platform 110 may determine the first running result of the model call node based on the first model output. The first running result may include all or part of the fields of the first model output, and may also include an operation result obtained by performing one or more operations based on the first model output.

[0069] For example, Figure 5AAs shown, the model call node can be used to call the machine learning model to perform translation operations on the user input. The application management platform 110 can extract, for example, the field value of the "literal translation" field and the field value of the "free translation" field from the first model output to form the first running result of the model call node. The application management platform 110 can use the running result presentation area 514 of the debugging interface 510 to present the first running result, such as the field value of the "literal translation" field and the field value of the "free translation" field. It can be understood that the specific content of the above-mentioned first running result is only exemplary. The specific content of the first running result mainly depends on the task performed by the model call node. In the case where the tasks performed by the model call node are different, the specific content of the first running result may also be different.

[0070] In the integrated development environment, not only can the prompt words of the model call node be edited, but also the running results of the model call node based on the edited prompt words can be tested, and then the quality of the prompt words can be determined. Furthermore, at least one parameter in the first configuration information can be adjusted through the configuration area 511 to test the running results of the edited prompt words under different machine learning models or other parameters, so as to accurately determine the quality of the edited prompt words and the running effect of the model call node, which is conducive to improving the editing efficiency, testing efficiency and editing quality of the configuration information such as the prompt words of the model call node.

[0071] In some embodiments, the application management platform 110 may present the second configuration information of the model calling node in response to a comparison debugging request for the model calling node. In response to the running instruction for the model calling node, a second machine learning model is used based on at least one second prompt to obtain a second model output. Afterwards, a second running result of the model calling node determined based on the second model output is presented. The second configuration information here may be the same or different from the first configuration information, for example, the second configuration information and the first configuration information may have the same parameters, and the parameter values ​​of each parameter may be set to be the same or different. The second configuration information includes at least one second prompt for providing to the machine learning model. The at least one second prompt may include one or more second prompts. The second prompt may be the same or different from the first prompt. For example, the at least one second prompt may also include a system prompt word, and one or more user prompt words. In this way, the running results of the model calling node based on the two sets of configuration information can be compared to judge the editing quality of the two sets of configuration information.

[0072] In some embodiments, the first operation result is presented in a first interface area, and the second operation result is presented in a second interface area that is distinct from and parallel to the first interface. Figure 5A and Figure 5BAs shown, the adjustment interface shown in example 500A may also include a comparison group creation control 515. The application management platform 110 may determine that a comparison debugging request for a model call node is received in response to a trigger operation for the comparison group creation control 515. Figure 5B As shown, the application management platform 110 can present a debugging interface 510 and a comparison interface 520 in the integrated development environment in response to a comparison debugging request for a model call node. For example, the debugging interface 510 and the comparison interface 520 can be arranged in parallel horizontally in the integrated development environment. The debugging interface 510 here can present the first configuration information. The specific parameters and receiving method of the first configuration information have been described above, and will not be repeated here. Please refer to the above explanation.

[0073] The comparison interface 520 may include an entry for inputting configuration information, such as a prompt word input entry 401, a model configuration entry 405, and an output parameter configuration entry 406. The user may input the second configuration information to the comparison interface 520 through these entries for inputting configuration information. For example, the user may expand the prompt word input entry 401 to input a system prompt word and at least one user prompt word to the comparison interface 520. For another example, the model configuration entry 405 may be used to specify a model call node to call a second machine learning model to perform a task.

[0074] The comparison interface 520 may also include a debugging parameter input entry 523 and an operation result presentation area 524. The application management platform 110 may receive another set of user inputs through the debugging parameter input entry 523. The application management platform 110 may determine that an operation instruction for a model call node is received in response to a trigger operation for the operation control 501. Based on the first configuration information and user input of the debugging interface 510, the first machine learning model is called to obtain a first operation result, and the first operation result is presented through the operation result presentation area 514 of the debugging interface 510.

[0075] The application management platform 110 can also determine the second model input in response to the run instruction based on the other set of user inputs input to the comparison interface 520 and the at least one second prompt. The second model input is provided to the second machine learning model to obtain a second model output. The application management platform 110 can determine the second run result of the model call node based on the second model output. Thereafter, the second run result is presented, for example, in the run result presentation area 524. In this way, the user (for example, the developer of the workflow) can intuitively determine the editing quality of the two sets of configuration information based on the two sets of run results, which is conducive to improving the development efficiency and quality of the model call node.

[0076] In some embodiments, the application management platform 110 may set the parameter value of at least one parameter in the first configuration information to be the same as the parameter value of at least one parameter in the second configuration information in response to a synchronization indication for at least one parameter. Figure 5B As shown, the comparison interface may also include synchronization controls 525, 526, 527, and 528. In response to the selection operation of the synchronization control 525, the application management platform 110 may set at least one first prompt in the debugging interface 510 and at least one second prompt in the comparison interface 520 to be the same. In actual application, the user may select the synchronization control 525 to set the prompt words of the two sets of configuration information to be the same. The user may also set the parameter value of at least one other parameter in the two sets of configuration information to be different. In this way, the performance of the prompt words in different application scenarios may be tested, such as the quality of the operating results formed when calling different machine learning models, and so on.

[0077] The application management platform 110 can set the information in the model configuration entry 405 in the debugging interface 510 and the comparison interface to be the same in response to the selection operation of the synchronization control 526. For example, the user can enter different prompt words in the debugging interface 510 and the comparison interface 520, respectively, and can select the synchronization control 527, so as to test the effect of different prompt words when calling the same machine learning model.

[0078] The application management platform 110 can set the parameter values ​​in the output parameter configuration entry 406 in the debugging interface 510 and the comparison interface to be the same in response to the selection operation of the synchronization control 527. The application management platform 110 can also set the user input in the debugging parameter input entry 513 and the debugging parameter input entry 523 to be the same in response to the selection operation of the synchronization control 528. It can be understood that the user can also not select any one or more of the above synchronization controls so that the parameter value of at least one parameter in the two sets of configuration information can be set to be different, thereby comparing the running results of the model call node based on the two sets of configuration information to determine the quality of the two sets of configuration information.

[0079] In some embodiments, the application management platform 110 may determine the selected configuration information as the target configuration information of the model call node in response to the selection of the first configuration information or the second configuration information. The target configuration information here should be understood as a set of configuration information selected for creating the model call node. For example, in the case where the model call node is one of multiple working nodes of a workflow, the model call node containing the target configuration information may be used to create a workflow.

[0080] For example, in an integrated development environment, the first configuration information in the debugging interface can be set as the target configuration information of the model call node by default. Figure 5B As shown, a configuration selection control 511 may be set in the comparison interface 520. In response to a trigger operation on the configuration selection control 511, the application management platform 110 may swap the configuration information in the debugging interface 510 with the configuration information in the comparison interface 520, or may switch the debugging interface 510 and the comparison interface 520, thereby setting the second configuration information as the target configuration information of the model call node.

[0081] It is understandable that the default configuration information may not be set in the integrated development environment. On this basis, the configuration selection control may be set in the debugging interface 510 and the comparison interface 520 respectively. In response to the triggering operation on the configuration selection control, the application management platform 110 may set a set of configuration information corresponding to the triggered configuration selection control as the target configuration information of the model call node.

[0082] In summary, according to the embodiments of the present disclosure, in the process of editing the prompt words and other configuration information of the model call node, the model call node can be run, and the machine learning model can be called based on the prompt words and other configuration information to obtain the running result of the model call node. In this way, the editing quality of the prompt words and other configuration information can be intuitively determined based on the running result, which is conducive to improving the configuration efficiency, test efficiency and running effect of the model call node.

[0083] The embodiments of the present disclosure also provide corresponding devices for implementing the above methods or processes. Figure 6 A schematic structural block diagram of an apparatus 600 for information processing according to some embodiments of the present disclosure is shown. The apparatus 600 may be implemented in or included in the application management platform 110, for example. Each module / component in the apparatus 600 may be implemented by hardware, software, firmware, or any combination thereof.

[0084] As shown in the figure, the device 600 may include a configuration module 610, an acquisition module 620 and a presentation module 630. The configuration module 610 is configured to present first configuration information of the model call node in response to a configuration request for a model call node in a workflow, wherein the workflow includes one or more model call nodes, and the model call node is configured to perform a task using a machine learning model, and the first configuration information includes at least one first prompt for providing to the machine learning model. The acquisition module 620 is configured to obtain a first model output using a first machine learning model based on at least one first prompt in response to an operation instruction for the model call node. The presentation module 630 is configured to present a first operation result of the model call node determined based on the first model output.

[0085] In some embodiments, the configuration module 610 is further configured to: in response to a configuration request for each model calling node in two or more model calling nodes in the workflow, present corresponding first configuration information of each model calling node.

[0086] In some embodiments, the configuration module 610 is further configured to: present a configuration interface of the model calling node in response to a configuration request for the model calling node; receive at least a portion of the first configuration information for the model calling node using the configuration interface; and present the first configuration information and an operation result presentation area in response to a debugging request for the model calling node.

[0087] In some embodiments, the device 600 may also include: a determination module configured to determine one or more input parameters for a model call node based on the received user input; and the acquisition module is further configured to: determine a first model input based on one or more input parameters and at least one first prompt; and provide the first model input to a first machine learning model to obtain a first model output.

[0088] In some embodiments, the first configuration information also includes at least one of the following: information for specifying a machine learning model, one or more input parameters of a model calling node, or one or more output parameters of a model calling node.

[0089] In some embodiments, the configuration module 610 is further configured to: in response to a comparison debugging request for the model call node, present second configuration information of the model call node, the second configuration information including at least one second prompt for providing to the machine learning model; the acquisition module 620 is further configured to: in response to an operation instruction for the model call node, obtain a second model output by utilizing the second machine learning model based on at least one second prompt; and the presentation module 630 is further configured to: present a second operation result of the model call node determined based on the second model output.

[0090] In some embodiments, the first running result is presented in a first interface area, and the second running result is presented in a second interface area separate from and parallel to the first interface.

[0091] In some embodiments, the apparatus 600 further includes: a setting module configured to set a parameter value of at least one parameter in the first configuration information and a parameter value of at least one parameter in the second configuration information to be the same in response to a synchronization indication for at least one parameter.

[0092] In some embodiments, the apparatus 600 further includes: a selection module configured to, in response to a selection of the first configuration information or the second configuration information, determine the selected configuration information as the target configuration information of the model calling node.

[0093] The units and / or modules included in the device 600 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units and / or modules can be implemented using software and / or firmware, such as machine executable instructions stored on a storage medium. In addition to or as an alternative to machine executable instructions, some or all of the units and / or modules in the device 600 can be implemented at least in part by one or more hardware logic components. As an example and not limitation, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0094] Figure 7 1 shows a block diagram of an electronic device 700 in which one or more embodiments of the present disclosure may be implemented. It should be understood that Figure 7 The electronic device 700 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. Figure 7 The electronic device 700 shown may include or be implemented as Figure 1 application management platform 110, or Figure 6 Device 600.

[0095] like Figure 7 As shown, the electronic device 700 is in the form of a general electronic device. The components of the electronic device 700 may include, but are not limited to, one or more processors or processing units 710, a memory 720, a storage device 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760. The processing unit 710 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 720. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 700.

[0096] The electronic device 700 typically includes a plurality of computer storage media. Such media may be any accessible media that is accessible to the electronic device 700, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 720 may be a volatile memory (e.g., registers, caches, random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 730 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data and may be accessed within the electronic device 700.

[0097] The electronic device 700 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 7 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 720 may include a computer program product 725 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0098] The communication unit 740 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 700 can be implemented with a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 700 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0099] The input device 750 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 760 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 700 may also communicate with one or more external devices (not shown) through the communication unit 740 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 700, or communicate with any device that allows the electronic device 700 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0100] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0101] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0102] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0103] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0104] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0105] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. An information processing method, comprising: In response to a configuration request for a model call node in a workflow, presenting first configuration information of the model call node, the workflow comprising one or more model call nodes, the model call node being configured to perform a task using a machine learning model, the first configuration information comprising at least one first prompt for providing to the machine learning model; In response to the run instruction for the model call node, using the first machine learning model based on the at least one first prompt to obtain a first model output; as well as A first running result of the model calling node determined based on the first model output is presented.

2. The method of claim 1, wherein presenting the first configuration information comprises: In response to a configuration request for each of the two or more model calling nodes in the workflow, corresponding first configuration information of each of the model calling nodes is presented.

3. The method of claim 1 , wherein presenting the first configuration information comprises: In response to a configuration request for the model calling node, presenting a configuration interface of the model calling node; Receiving at least a portion of the first configuration information for the model calling node using the configuration interface; as well as In response to a debugging request for the model calling node, the first configuration information and the running result presentation area are presented.

4. The method according to claim 1, further comprising: Determining one or more input parameters for the model call node based on the received user input; and Wherein obtaining the first model output comprises: determining a first model input based on the one or more input parameters and the at least one first prompt; as well as The first model input is provided to the first machine learning model to obtain the first model output.

5. The method according to claim 1, wherein the first configuration information further includes at least one of the following: Information used to specify the machine learning model, One or more input parameters of the model call node, or The model calls one or more output parameters of the node.

6. The method according to claim 1, further comprising: In response to a comparison debugging request for the model calling node, presenting second configuration information of the model calling node, wherein the second configuration information includes at least one second prompt for providing to the machine learning model; In response to the run instruction for the model call node, using a second machine learning model based on the at least one second prompt to obtain a second model output; as well as A second running result of the model calling node determined based on the second model output is presented. 7 . The method according to claim 6 , wherein the first operation result is presented in a first interface area, and the second operation result is presented in a second interface area that is distinct from and parallel to the first interface.

8. The method according to claim 6, further comprising: In response to a synchronization indication for at least one parameter, a parameter value of the at least one parameter in the first configuration information and a parameter value of the at least one parameter in the second configuration information are set to be the same.

9. The method according to claim 6, further comprising: In response to selection of the first configuration information or the second configuration information, the selected configuration information is determined as target configuration information of the model calling node.

10. An apparatus for information processing, comprising: a configuration module configured to present first configuration information of a model call node in a workflow in response to a configuration request for the model call node, wherein the workflow includes one or more model call nodes, and the model call node is configured to perform a task using a machine learning model, and the first configuration information includes at least one first prompt for providing to the machine learning model; an obtaining module configured to obtain a first model output using a first machine learning model based on the at least one first prompt in response to an execution instruction for the model calling node; as well as A presentation module is configured to present a first running result of the model calling node determined based on the first model output.

11. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 9 when executed by the at least one processing unit.

12. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, wherein the computer program implements the method according to any one of claims 1 to 9 when executed by a processor.

Citation Information

Patent Citations

  • Model evaluation method and device, electronic equipment and storage medium

    CN114661571A

  • Application creation method and device, electronic equipment and storage medium

    CN117406966A

  • Method and device for managing workflow, equipment and medium

    CN118409681A