Interaction method and device, equipment and storage medium

Abstract

According to the embodiment of the invention, a method and device for interaction, equipment and a storage medium are provided. The method comprises the following steps: presenting an interactive interface, wherein the interactive interface comprises at least one first interactive object; in response to detection of selection for a first interaction object in the at least one first interaction object, presenting the selected first interaction object, prompt word information associated with the selected first interaction object and model parameters in the interaction interface; in response to detection of a sending instruction for the selected first interaction object, multimedia content, prompt word information and model parameters associated with the selected first interaction object are sent to a target machine learning model; and in response to an execution result for the target task received from the target machine learning model, presenting the execution result in the interactive interface. Therefore, the user can intuitively understand the capability of the machine learning model at low operation cost.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and more particularly, to methods, apparatuses, devices, and computer-readable storage media for interaction. Background Art

[0002] To evaluate a machine learning model, such as evaluating the machine learning model's ability to understand multimedia content, users are required to upload multimedia content themselves, and they are also required to specify prompt word information and various model parameters. Since prompt word information and model parameters play an important role in the model's understanding ability, unreasonable settings of prompt word information and / or model parameters will directly affect the parsing effect of the machine learning model. This will make it difficult for users to evaluate the machine learning model's understanding ability, and setting prompt word information and model parameters will also increase the complexity and cost of user operations. Summary of the invention

[0003] In the first aspect of the present disclosure, a method for interaction is provided. The method includes: presenting an interactive interface, the interactive interface includes at least one first interactive object, each of the at least one first interactive object is associated with multimedia content, prompt word information and model parameters, the prompt word information is used to describe the target task that the target machine learning model needs to perform on the multimedia content, and the model parameters indicate the parameters used by the target machine learning model in performing the target task; in response to detecting the selection of a first interactive object among the at least one first interactive object, presenting the selected first interactive object, the prompt word information and the model parameters associated with the selected first interactive object in the interactive interface; in response to detecting a sending instruction for the selected first interactive object, sending the multimedia content, the prompt word information and the model parameters associated with the selected first interactive object to the target machine learning model; and in response to receiving an execution result for the target task from the target machine learning model, presenting the execution result in the interactive interface.

[0004] In the second aspect of the present disclosure, a device for interaction is provided. The device includes: a first presentation module, configured to present an interactive interface, the interactive interface includes at least one first interactive object, at least one first interactive object is each associated with multimedia content, prompt word information and model parameters, the prompt word information is used to describe the target task that the target machine learning model needs to perform on the multimedia content, and the model parameters indicate the parameters used by the target machine learning model in performing the target task; a second presentation module, configured to present the selected first interactive object, the prompt word information and the model parameters associated with the selected first interactive object in the interactive interface in response to detecting the selection of the first interactive object in the at least one first interactive object; an information sending module, configured to send the multimedia content, the prompt word information and the model parameters associated with the selected first interactive object to the target machine learning model in response to detecting the sending instruction for the selected first interactive object; and a third presentation module, configured to present the execution result in the interactive interface in response to receiving the execution result for the target task from the target machine learning model.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect.

[0007] In a fifth aspect of the present disclosure, a computer program product is provided, which includes computer executable instructions, wherein the computer executable instructions implement the method of the first aspect when executed by a processor.

[0008] It should be understood that the contents described in this content section are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented;

[0011] Figure 2 A flowchart showing a process of user interaction according to some embodiments of the present disclosure;

[0012] FIG. 3A to FIG. 3C A schematic diagram showing an example interface according to some embodiments of the present disclosure;

[0013] Figure 4 A block diagram showing an apparatus for user interaction according to some embodiments of the present disclosure; and

[0014] Figure 5 A block diagram of a device capable of implementing various embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0015] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0016] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0017] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0018] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0019] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0020] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0021] It should be noted that the titles of any sections / subsections provided herein are not restrictive. Various embodiments are described throughout this article, and any type of embodiment may be included under any section / subsection. In addition, the embodiments described in any section / subsection may be combined in any manner with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0022] Herein, unless explicitly stated, executing a step “in response to A” does not mean executing the step immediately after “A” but may include one or more intermediate steps.

[0023] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below. The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may be included below.

[0024] As used herein, the term "model" can learn the association between the corresponding input and output from the training data, so that after the training is completed, the corresponding output can be generated for a given input. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multiple layers of processing units. In this article, "model" may also be referred to as "machine learning model", "machine learning network" or "network", which are used interchangeably in this article. A model may also include different types of processing units or networks.

[0025] As briefly mentioned above, evaluating a machine learning model, such as evaluating the multimedia content understanding ability of a machine learning model, requires users to upload multimedia content by themselves, and also requires users to specify prompt word information and various model parameters. When uploading multimedia content, users need to perform operations such as editing the multimedia content, which leads to the complexity and cost of user operations. In addition, since prompt word information and model parameters play a key role in the model's understanding ability, unreasonable settings of prompt word information and / or model parameters will directly affect the parsing effect of the machine learning model. If the user does not have certain professional knowledge, the specified prompt word information or model parameters will be unreasonable, which will directly affect the machine learning model's understanding of multimedia content, making it difficult for users to evaluate the machine learning model's understanding ability, and setting prompt word information and model parameters will also increase the complexity and cost of user operations.

[0026] The embodiments of the present disclosure propose a scheme for interaction. According to various embodiments of the present disclosure, an interactive interface is presented, the interactive interface includes at least one first interactive object, each of which is associated with multimedia content, prompt word information and model parameters; in response to detecting the selection of a first interactive object in at least one first interactive object, the selected first interactive object, the prompt word information and model parameters associated with the selected first interactive object are presented in the interactive interface; in response to detecting a sending instruction for the selected first interactive object, the multimedia content, prompt word information and model parameters associated with the selected first interactive object are sent to the target machine learning model; and in response to receiving the execution result of the target task from the target machine learning model, the execution result is presented in the interactive interface. In this way, when evaluating the understanding ability of a machine learning model for multimedia content, the user only needs to select the corresponding multimedia content and confirm the sending, and the multimedia content, the prompt word information and model parameters associated with the multimedia content can be sent to the target machine learning model, and the execution result of the target machine learning model can be returned to the user, so that the user can intuitively understand the ability of the machine learning model with a relatively low operating cost.

[0027] Figure 1A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. In environment 100, an application 130 for evaluating a machine learning model, such as a web application or other type of application, is deployed in a terminal device 110. In some embodiments, application 130 is provided to assist users in various task processing requirements in different applications and scenarios (for example, evaluating the ability of a machine learning model to understand multimedia content). During the interaction with application 130, the user inputs an interactive message, and application 130 provides a reply message in response to the user input. Typically, application 130 can support users to input questions in natural language, and perform tasks and provide replies based on the understanding of natural language input and logical reasoning ability.

[0028] In some embodiments, application 130 can interact with end user 150 as a contact. For example, application 130 can be implemented in an instant messaging (IM) application. Application 130 can interact with end user 150 in a single chat session with end user 150. In some embodiments, application 130 can interact with multiple users in a group chat session including multiple users.

[0029] In some embodiments, interactive messages with application 130 may include messages in multimodal forms, such as text messages (eg, natural language text), voice messages, image messages, video messages, and the like.

[0030] In environment 100, machine learning models 140-1, 140-2, ..., 140-N are deployed in server 120. For ease of discussion, machine learning models 140-1, 140-2, ..., 140-N may be collectively or individually referred to as machine learning model 140.

[0031] In some embodiments, the machine learning model 140 may be, for example, a model for processing image data or a model for processing video data.

[0032] In some embodiments, the terminal device 110 can be any type of mobile terminal, fixed terminal or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, handheld computers, portable game terminals, VR / AR devices, personal communication systems (PCS) devices, personal navigation devices, personal digital assistants (PDA), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio broadcast receivers, e-book devices, game devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the client device can also support any type of interface for the user (such as "wearable" circuits, etc.).

[0033] In some embodiments, the server 120 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms. The server 120 may include, for example, a computing system / server, such as a mainframe, an edge computing node, a computing device in a cloud environment, and the like.

[0034] A communication connection may be established between the server 120 and the terminal device 110. The communication connection may be established in a wired manner or a wireless manner. The communication connection may include, but is not limited to, a Bluetooth connection, a mobile network connection, a Universal Serial Bus (USB) connection, a Wireless Fidelity (WiFi) connection, etc., and the embodiments of the present disclosure are not limited in this respect. In the embodiments of the present disclosure, the server 120 and the terminal device 110 may implement signaling interaction through the communication connection between the two.

[0035] It should be understood that the structure and functionality of environment 100 are described for exemplary purposes only and does not imply any limitation on the scope of the present disclosure.

[0036] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0037] Figure 2 FIG. 2 is a flowchart of a process 200 for interaction according to some embodiments of the present disclosure. The following description will be given by taking the terminal device executing the interaction process 200 as an example. Figure 1The scheme for interaction provided by the present disclosure is introduced in detail.

[0038] In block 210 , the application 130 in the terminal device 110 presents an interactive interface.

[0039] The interactive interface includes at least one first interactive object, each of which is associated with multimedia content, prompt word information, and model parameters. The prompt word information is used to describe the target task that the target machine learning model needs to perform on the multimedia content. The model parameters indicate the parameters used by the target machine learning model in performing the target task.

[0040] In some embodiments, the multimedia content, prompt word information and model parameters associated with the first interactive object may be stored locally in the terminal device 110. In some embodiments, the multimedia content, prompt word information and model parameters associated with the first interactive object may be stored in the server 120.

[0041] In this article, "interactive object" refers to an element, component, etc. that can be interacted with by a user in an interactive interface. In some embodiments, the first interactive object may include, for example, a picture, icon, card, logo, etc. for displaying multimedia content corresponding to the first interactive object. The embodiments of the present disclosure do not limit the specific presentation method of the first interactive object.

[0042] In some embodiments, the multimedia content may include but is not limited to audio, video or images. For example, if the multimedia content is a video, the prompt word information corresponding to the video may be, for example, "describe the details of the video". Accordingly, the target task is to use the corresponding machine learning model (such as a video understanding model) to analyze the video to "describe the details of the video". For another example, if the multimedia content is an image, the prompt word information corresponding to the image may be, for example, "describe the content in the image". Accordingly, the target task is to use the corresponding machine learning model (such as an image recognition model) to analyze the image to "describe the content in the picture".

[0043] It should be understood that the examples of machine learning models and their tasks given here are for illustrative purposes only. The embodiments of the present disclosure can be applied to the capability assessment of various machine learning models related to multimedia content processing.

[0044] Take the first interactive object as an example, which is a picture showing multimedia content. Figure 3A As shown, pictures 310 - 1 , 310 - 2 , . . . , 310 -N are presented in the interface 300A presented by the terminal device 110. For the convenience of discussion, pictures 310 - 1 , 310 - 2 , . . . , 310 -N may be collectively or individually referred to as pictures 310.

[0045] In the interface 300A, the picture 310 is configured to be selectable. As an example, a selection button may be presented on the picture 310. After the picture 310 is selected, as shown in the figure, for example, the border of the picture 310 may be displayed in bold.

[0046] In box 220, in response to detecting the selection of a first interactive object among the at least one first interactive object, the terminal device 110 presents the selected first interactive object, prompt word information associated with the selected first interactive object, and model parameters in the interactive interface.

[0047] In some embodiments, in response to detecting the selection of a first interactive object among at least one first interactive object, the terminal device 110 can obtain multimedia content, prompt word information, and model parameters associated with the selected first interactive object from the terminal device 110 locally or from the server 120.

[0048] After acquiring the multimedia content, prompt word information, and model parameters associated with the selected first interactive object, the terminal device 110 may present the selected first interactive object, the prompt word information, and model parameters associated with the selected first interactive object in the interactive interface.

[0049] like Figure 3B As shown, an input area 320 and a parameter configuration area 330 are presented in the interface 300B. After the terminal device 110 detects that the first interactive object is selected, the selected first interactive object and prompt word information 340 corresponding to the selected first interactive object can be presented in the input area 320, and the model parameters 350 corresponding to the selected first interactive object can be presented in the parameter configuration area 330.

[0050] It should be understood that the selected first interactive object presented in the input area 320 and the prompt word information 340 corresponding to the selected first interactive object can be presented in any suitable manner, and the model parameters 350 corresponding to the selected first interactive object presented in the parameter configuration area 330 can also be presented in any suitable manner, and the present disclosed embodiment does not impose any limitations.

[0051] In some embodiments, when the first interactive object is presented in the input area 320, the selected picture 310 may be presented directly, or other icons, texts, etc. that can indicate the selected picture 310 may be presented. The embodiments of the present disclosure are not limited thereto.

[0052] In some embodiments, the prompt word information presented in the interactive interface can be edited. Figure 3BThe prompt word information 340 presented in the input area 320 may be, for example, default prompt word information, that is, prompt word information obtained by the terminal device 110 . The terminal user 150 may edit the prompt word information in the input area 320 according to actual needs.

[0053] In some embodiments, the terminal device 110 may present the edited prompt word information in the interface 300B in response to detecting that the prompt word information 340 in the input area 320 is edited. As an example, the terminal device 110 may present the editing process of the user while the terminal user 150 is editing the prompt word information; or may present the final result of the editing after the user completes the editing process.

[0054] In some embodiments, the model parameters in the interactive interface can be edited. Figure 3B The model parameters 350 presented in the parameter configuration area 330 may be, for example, default model parameters, that is, model parameters obtained by the terminal device 110 . The terminal user 150 may edit the model parameters in the parameter configuration area 330 according to actual needs.

[0055] In some embodiments, in response to detecting that the model parameter 350 in the parameter configuration area 330 is edited, the terminal device 110 may present the edited model parameter in the interface 300B.

[0056] In some embodiments, the model parameters may include, for example, at least any one of the following: sampling parameters, execution result text length, the number of frames in the multimedia content, the resolution used to collect the multimedia content, the frame rate used to collect the multimedia content, the method of collecting the multimedia content, or the area of ​​​​collecting the multimedia content.

[0057] The sampling parameters may indicate the randomness of the picture frames in the multimedia content collected by the machine learning model. For example, the higher the sampling parameters, the greater the randomness of the picture frames in the multimedia content collected by the machine learning model; conversely, the smaller the randomness of the picture frames in the multimedia content collected by the machine learning model. The execution result text length may indicate the maximum length of the text output by the machine learning model. The number of frames of the picture frames collected in the multimedia content may indicate the number of frames of the picture frames collected from the multimedia content. The method of collecting the picture frames in the multimedia content may, for example, indicate the method of collecting the picture frames, such as uniformly collecting the picture frames, or collecting one or more key frames. The area for collecting multimedia content may, for example, include collecting an area of ​​a predetermined shape from the picture frames of the multimedia content, generating a predetermined shape from the collected picture frames of the multimedia content, or filling the black edges of the picture frames to form a predetermined shape.

[0058] In box 230, in response to detecting a sending instruction for the selected first interactive object, the terminal device 110 sends the multimedia content, prompt word information and model parameters associated with the selected first interactive object to the target machine learning model.

[0059] Before sending the multimedia content, prompt word information and model parameters associated with the selected first interactive object to the target machine learning model, it is necessary to determine the target machine learning model.

[0060] In some embodiments, the interactive interface presented by the terminal device 110 also includes a second interactive object, and the second interactive object indicates at least one machine learning model. In response to detecting the selection of a machine learning model in at least one machine learning model, the terminal device 110 determines the selected machine learning model as the target machine learning model. In some embodiments, the second interactive object can be the name of the machine learning model.

[0061] In some embodiments, the terminal device 110 may send the identification or name of the determined target machine learning model to the server 120 so that the server 120 determines the target machine learning model from multiple machine learning models.

[0062] Continue to see Figure 3B , a model selection area 360 is presented in the interface 300B, and the name 370 of the selected machine learning model is presented in the model selection area 360. As an example, the name of the machine learning model 370 can be presented in the form of a drop-down menu, that is, when the terminal user 150 clicks the model selection area 360, a drop-down menu will pop up, and multiple machine learning models are presented in the drop-down menu. When the terminal user 150 selects any machine learning model, the name 370 of the corresponding machine learning model can be presented in the model selection area 360.

[0063] After determining the target machine learning model, the terminal user 150 can click the send button 325 to cause the terminal device 110 to send the multimedia content, prompt word information, and model parameters associated with the selected first interactive object to the server 120, so that the server 120 can send the multimedia content, prompt word information, and model parameters associated with the selected first interactive object to the target machine learning model to perform the target task. The server 120 can send the execution result to the terminal device 110 after the target machine learning model completes the execution of the target task.

[0064] In box 240, the terminal device 110 presents the execution result in the interactive interface in response to receiving the execution result for the target task from the target machine learning model.

[0065] In some embodiments, the interactive interface includes an interactive window with the target machine learning model, and the terminal device 110 can present the selected first interactive object, prompt word information associated with the selected first interactive object, and the execution result in the interactive window as the context of the interactive window.

[0066] like Figure 3C As shown, the interface 300C presents an input area 320 and an interaction window 390 . After the terminal user 150 clicks the send button 325 , the first interaction object and prompt word information may stop being presented in the input area 320 , and the first interaction object and prompt word information may be presented in the interaction window 390 .

[0067] In response to receiving the execution result for the target task from the target machine learning model, the terminal device 110 can present the execution result 380 in the interactive window 390 as the context of the interactive window 390 to facilitate the user to observe the execution result output by the target machine learning model for the target task.

[0068] In the interaction window 390, the prompt word information 340 is configured to be editable. After the terminal user 150 edits the prompt word information 340, the send button 325 can be clicked again to make the terminal device 110 send the multimedia content associated with the selected first interactive object, the edited prompt word information and the model parameters to the target machine learning model, or make the terminal device only send the edited prompt word information to the target machine learning model, so that the machine learning model can perform the target task again based on the edited prompt word information.

[0069] Similarly, after the target machine learning model returns the execution result, the model parameters 350 can also be edited. After the terminal user 150 edits the model parameters 350, the send button 325 can be clicked again to allow the terminal device 110 to send the multimedia content, prompt word information and edited model parameters associated with the selected first interactive object to the target machine learning model, or to allow the terminal device to only send the edited model parameters to the target machine learning model so that the machine learning model can perform the target task again based on the edited model parameters.

[0070] It should be understood that after the target machine learning model returns the execution result, the prompt word information 340 and the model parameters 350 can be edited separately or simultaneously.

[0071] Continue to see Figure 3CAfter the terminal device 110 sends the multimedia content, prompt word information and model parameters associated with the selected first interactive object to the target machine learning model again, the re-sent first interactive object and the re-sent prompt word information 340-1 can be presented in the interactive window 390. After the target machine learning model returns the execution result, the returned execution result 380-1 is presented in the interactive window as a new context in the interactive window 390.

[0072] In the disclosed embodiment, when evaluating a machine learning model's ability to understand multimedia content, the user only needs to select the corresponding multimedia content and confirm sending, and the multimedia content, the prompt word information associated with the multimedia content, and the model parameters can be sent to the target machine learning model, and the execution result of the target machine learning model can be returned to the user, thereby allowing the user to intuitively understand the capabilities of the machine learning model at a lower operating cost.

[0073] Figure 4 A schematic structural block diagram of an apparatus 400 for interaction according to some embodiments of the present disclosure is shown. The apparatus 400 may be implemented as or included in a terminal device 110. Each module / component in the apparatus 400 may be implemented by hardware, software, firmware, or any combination thereof.

[0074] As shown in the figure, the device 400 includes a first presentation module 410, which is configured to present an interactive interface, wherein the interactive interface includes at least one first interactive object, each of which is associated with multimedia content, prompt word information, and model parameters, wherein the prompt word information is used to describe the target task that the target machine learning model needs to perform on the multimedia content, and the model parameters indicate the parameters used by the target machine learning model in performing the target task. The device 400 also includes a second presentation module 420, which is configured to present the selected first interactive object, the prompt word information and the model parameters associated with the selected first interactive object in the interactive interface in response to detecting the selection of the first interactive object in the at least one first interactive object. The device 400 also includes an information sending module 430, which is configured to send the multimedia content, the prompt word information and the model parameters associated with the selected first interactive object to the target machine learning model in response to detecting the sending instruction for the selected first interactive object. The device 400 also includes a third presentation module 440, which is configured to present the execution result in the interactive interface in response to receiving the execution result for the target task from the target machine learning model.

[0075] In some embodiments, the interaction interface includes an interaction window with the target machine learning model, and presenting the execution result in the interaction interface includes: presenting the selected first interaction object, the prompt word information associated with the selected first interaction object, and the execution result as the context of the interaction window in the interaction window.

[0076] In some embodiments, the apparatus 400 further includes an acquisition module configured to acquire the multimedia content, the prompt word information, and the model parameters associated with the selected first interactive object.

[0077] In some embodiments, the prompt word information presented in the interactive interface can be edited, and the device 400 also includes a fourth presentation module, which is configured to present the edited prompt word information in the interactive interface in response to detecting that the prompt word information is edited.

[0078] In some embodiments, the model parameters presented in the interactive interface may be edited, and the apparatus 400 further includes a fifth presentation module configured to present the edited model parameters in the interactive interface in response to detecting that the model parameters are edited.

[0079] In some embodiments, the interaction interface also includes a second interaction object, which indicates at least one machine learning model.

[0080] In some embodiments, the device 400 also includes a model determination module, which is configured to determine the selected machine learning model as the target machine learning model in response to detecting a selection of a machine learning model among the at least one machine learning model.

[0081] In some embodiments, the model parameters include at least any one of the following: sampling parameters, execution result text length, the number of frames in the captured multimedia content, the resolution used to capture the multimedia content, the frame rate used to capture the multimedia content, the method of capturing the frames in the multimedia content, or the area of ​​the captured multimedia content.

[0082] Figure 5 A block diagram of an electronic device 500 is shown in which one or more embodiments of the present disclosure may be implemented. It should be understood that Figure 5 The electronic device 500 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. Figure 5 The electronic device 500 shown can be used to implement Figure 1 terminal device 110.

[0083] like Figure 5As shown, the electronic device 500 is in the form of a general electronic device. The components of the electronic device 500 may include, but are not limited to, one or more processors or processing units 510, a memory 520, a storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. The processing unit 510 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 520. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 500.

[0084] The electronic device 500 typically includes a plurality of computer storage media. Such media may be any accessible media that is accessible to the electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 520 may be a volatile memory (e.g., registers, caches, random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 530 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data and may be accessed within the electronic device 500.

[0085] The electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 5 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 520 may include a computer program product 525 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0086] The communication unit 540 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 500 can be implemented in a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 500 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0087] The input device 550 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 560 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 500 may also communicate with one or more external devices (not shown) through the communication unit 540 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 500, or communicate with any device that allows the electronic device 500 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0088] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0089] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0090] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0091] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0092] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0093] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. A method for interaction, comprising: Presenting an interactive interface, the interactive interface comprising at least one first interactive object, each of the at least one first interactive object being associated with multimedia content, prompt word information, and model parameters, the prompt word information being used to describe a target task that a target machine learning model needs to perform on the multimedia content, and the model parameters indicating parameters used by the target machine learning model in performing the target task; In response to detecting a selection of a first interactive object among the at least one first interactive object, presenting the selected first interactive object, the prompt word information associated with the selected first interactive object, and the model parameters in the interactive interface; In response to detecting a sending instruction for the selected first interactive object, sending the multimedia content, the prompt word information, and the model parameters associated with the selected first interactive object to the target machine learning model; as well as In response to receiving an execution result for the target task from the target machine learning model, presenting the execution result in the interactive interface.

2. The method according to claim 1, wherein the interactive interface includes an interactive window with the target machine learning model, and presenting the execution result in the interactive interface includes: The selected first interactive object, the prompt word information associated with the selected first interactive object, and the execution result are presented in the interactive window as the context of the interactive window.

3. The method according to claim 1, wherein before presenting the selected first interactive object, the prompt word information associated with the selected first interactive object and the model parameters, the method further comprises: The multimedia content, the prompt word information and the model parameters associated with the selected first interactive object are obtained.

4. The method according to claim 1, wherein the prompt word information presented in the interactive interface can be edited, and the method further comprises: In response to detecting that the prompt word information is edited, presenting the edited prompt word information in the interactive interface.

5. The method according to claim 1, wherein the model parameters presented in the interactive interface are editable, the method further comprising: In response to detecting that the model parameters are edited, presenting the edited model parameters in the interactive interface.

6. The method according to claim 1, wherein the interactive interface further comprises a second interactive object, wherein the second interactive object indicates at least one machine learning model.

7. The method according to claim 6, further comprising: In response to detecting a selection of a machine learning model from the at least one machine learning model, the selected machine learning model is determined as the target machine learning model.

8. The method according to claim 1, wherein the model parameters include at least any one of the following: Sampling parameters, Length of the execution result text. The number of frames in the multimedia content is collected. The resolution at which the multimedia content is captured, The frame rate at which multimedia content is captured, A method of capturing frames of multimedia content, or The area where multimedia content is collected.

9. A device for interaction, comprising: A first presentation module is configured to present an interactive interface, wherein the interactive interface includes at least one first interactive object, each of which is associated with multimedia content, prompt word information, and model parameters, wherein the prompt word information is used to describe a target task that a target machine learning model needs to perform on the multimedia content, and the model parameters indicate parameters used by the target machine learning model in performing the target task; A second presentation module is configured to present, in response to detecting a selection of a first interactive object among the at least one first interactive object, the selected first interactive object, the prompt word information associated with the selected first interactive object and the model parameters in the interactive interface; an information sending module, configured to send the multimedia content, the prompt word information and the model parameters associated with the selected first interactive object to the target machine learning model in response to detecting a sending instruction for the selected first interactive object; as well as The third presentation module is configured to present the execution result in the interactive interface in response to receiving the execution result for the target task from the target machine learning model.

10. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 1 to 8.

12. A computer program product comprising computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 8.

Images