Clue information acquisition method and device, storage medium and program product

Abstract

The embodiment of the invention provides a clue information acquisition method and device, a storage medium and a program product. In the embodiment of the invention, the clue information acquisition method is provided, different workflow plug-ins are set for different web pages, and the clue acquisition process including loading and displaying the web pages, simulating user interaction, AI automatic reply and clue identification is automatically executed, so that manual intervention is reduced, the efficiency and accuracy of clue acquisition are improved, and the user experience is improved. In addition, the user experience is enhanced, the web page is not invaded, and the risk of being sealed and controlled is reduced. Furthermore, the clue information identified from different web pages can be directly acquired, mutual information does not need to be replied one by one for the different web pages, the clue information does not need to be identified one by one for the different web pages, and the acquisition efficiency of the clue information is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a clue information acquisition method, device, storage medium and program product. Background Art

[0002] In today's digital marketing world, companies are increasingly relying on social media platforms and other online channels to interact with potential users. In order to effectively acquire new users (customer acquisition), companies need to respond to users' private messages or comments in a timely manner and identify clue information from them. Clue information refers to relevant information about potential users, which can assist companies in identifying, evaluating and following up on potential users to ultimately achieve marketing conversions. However, in actual applications, especially for off-site channels (such as third-party social platforms or third-party forums, etc.), companies need to respond to a large number of users' private messages or comments on multiple off-site channels and identify clue information from them. In order to improve the efficiency of responses, script development can be used to embed scripts in the pages corresponding to the off-site channels to obtain private messages and comment content and provide response information. However, this method is invasive to the pages of off-site channels and is prone to triggering risk control. Summary of the invention

[0003] Multiple aspects of the present application provide a clue information acquisition method, device, storage medium and program product to improve the efficiency and accuracy of clue acquisition and reduce the risk of being blocked.

[0004] The embodiment of the present application provides a method for acquiring clue information, which is applied to a target application that has multiple workflow plug-ins deployed, and different workflow plug-ins are associated with different web pages in the target application, and the workflow plug-ins include a first functional plug-in for executing a clue acquisition workflow for a first user terminal; the method includes: during the operation of the target application, in response to a first workflow start event, determining the first functional plug-in pointed to by the first workflow start event; according to the clue acquisition workflow corresponding to the first functional plug-in, at least performing the following operations: loading and displaying a first web page associated with the workflow plug-in to which the first functional plug-in belongs, the first web page displaying a first interactive view; simulating the first user terminal in the first interactive view A first user interaction operation is performed on a web page, so that the first web page switches from the first interaction view to the second interaction view in response to the first user interaction operation; first interaction information sent by a second user terminal is obtained from the second interaction view; second interaction information corresponding to the first interaction information is generated based on an AI language model, and sent to the second user terminal, so as to replace the first user terminal and the second user terminal to perform at least one round of interaction and obtain at least one round of interaction information; when clue information is identified based on the at least one round of interaction information, the identified clue information is sent to a clue server, so that the first user terminal can subsequently obtain the required clue information from the clue server.

[0005] An embodiment of the present application also provides an electronic device, including: a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement each step in the clue information acquisition method provided in the embodiment of the present application.

[0006] The embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor implements each step of the clue information acquisition method provided in the embodiment of the present application.

[0007] An embodiment of the present application also provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the processor is caused to implement each step of the clue information acquisition method provided in the embodiment of the present application.

[0008] In an embodiment of the present application, a method for obtaining clue information is provided, in which different workflow plug-ins are set for different web pages, and the clue obtaining process including loading and displaying web pages, simulating user interaction, AI automatic reply and clue identification is automatically executed, reducing manual intervention, not only improving the efficiency and accuracy of clue acquisition, but also enhancing user experience, and not being invasive to web pages, reducing the risk of being blocked. Furthermore, the clue information identified from different web pages can be directly obtained, without the need to reply to interactive information one by one for different web pages, nor is there any need to identify clue information one by one for different web pages, thereby improving the efficiency of obtaining clue information. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0010] Figure 1 A flowchart of a clue information acquisition method provided for an exemplary embodiment of the present application;

[0011] Figure 2a A schematic diagram of a first web page displaying a first interactive view provided by an exemplary embodiment of the present application;

[0012] Figure 2b A schematic diagram of a first web page displaying a second interactive view provided by an exemplary embodiment of the present application;

[0013] Figure 3a A time sequence interaction flow chart of obtaining clue information provided by an exemplary embodiment of the present application;

[0014] Figure 3b A schematic diagram of an interactive control provided for an exemplary embodiment of the present application;

[0015] Figure 3c A schematic diagram of a workflow construction interface provided by an exemplary embodiment of the present application;

[0016] Figure 3d to Figure 3e A schematic diagram of target work icon connection provided for an exemplary embodiment of the present application;

[0017] Figure 3f to Figure 3g A schematic diagram of a working icon provided for an exemplary embodiment of the present application;

[0018] Figure 3h A schematic diagram of a startup control provided by an exemplary embodiment of the present application;

[0019] Figure 4A schematic structural diagram of an electronic device provided for an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0021] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0022] The various models involved in this application (including but not limited to language models or large models) are in compliance with relevant laws and standards.

[0023] In response to the above technical problems, in an embodiment of the present application, a method for obtaining clue information is provided, in which different workflow plug-ins are set for different web pages, and the clue obtaining process including loading and displaying web pages, simulating user interaction, AI automatic reply and clue identification is automatically executed, reducing manual intervention, not only improving the efficiency and accuracy of clue acquisition, but also enhancing the user experience, and having no intrusion to web pages, reducing the risk of being blocked. Furthermore, the clue information identified from different web pages can be directly obtained, without the need to reply to the interactive information one by one for different web pages, nor the need to identify the clue information one by one for different web pages, thereby improving the efficiency of obtaining clue information.

[0024] A solution provided by an embodiment of the present application is described in detail below in conjunction with the accompanying drawings.

[0025] Figure 1 A flowchart of a clue information acquisition method provided for an exemplary embodiment of the present application, the method is applied to a target application deployed with multiple workflow plug-ins, different workflow plug-ins are associated with different web pages in the target application, the workflow plug-ins include a first functional plug-in for executing a clue acquisition workflow for a first user terminal; Figure 1 As shown, the method includes:

[0026] 101. During the operation of the target application, in response to a first workflow start event, determine a first functional plug-in pointed to by the first workflow start event; obtain a workflow according to a clue corresponding to the first functional plug-in, and at least perform the following operations:

[0027] 102. Load and display a first web page associated with a workflow plug-in to which the first function plug-in belongs, wherein the first web page displays a first interactive view;

[0028] 103. Simulate the first user terminal to perform a first user interaction operation on the first web page, so that the first web page switches from the first interaction view to the second interaction view in response to the first user interaction operation;

[0029] 104. Acquire first interaction information sent by the second user terminal from the second interaction view; generate second interaction information corresponding to the first interaction information based on the AI ​​language model, and send the second interaction information to the second user terminal, so as to replace the first user terminal in performing at least one round of interaction with the second user terminal, and obtain at least one round of interaction information;

[0030] 105. When clue information is identified based on at least one round of interaction information, the identified clue information is sent to the clue server, so that the first user terminal can subsequently obtain required clue information from the clue server.

[0031] In this embodiment, the target application may be any application that can display a web (World Wide Web) page. For example, the target application may include, but is not limited to, a browser application, a search engine application, a desktop application, a social application, an e-book application, a news or magazine application, a game application, an online education application, and a smart TV and set-top box application. Optionally, the target application may be implemented as a target browser.

[0032] In this embodiment, multiple workflow plug-ins can be deployed on the target application, and different workflow plug-ins are associated with different web pages in the target application. The web page can be an in-application web page of the target application, or a third-party web page, without limitation. For example, workflow plug-in B1 is deployed for the web page of social application A1, workflow plug-in B2 is deployed for the web page of social application A2, and workflow plug-in B3 is deployed for the web page of social application A3. For another example, workflow plug-in D1 is deployed for the web page of social application C1, workflow plug-in D2 is deployed for the in-application web page C2 of the target application, and workflow plug-in D3 is deployed for the web page of online education application C3.

[0033] The workflow plug-in may be implemented as an automation plug-in. Optionally, the workflow plug-in may be implemented as a browser plug-in.

[0034] In this embodiment, no matter which of the above-mentioned web pages corresponds to the workflow plug-in, the workflow plug-in can include at least one functional plug-in, and different functional plug-ins are used to execute different workflows. For the sake of distinction and description, the functional plug-in that executes the clue acquisition workflow for the first user terminal is called the first functional plug-in. Among them, the first user terminal corresponds to the first user, and the first user has an application resource account registered on the target application, and the terminal device corresponding to the first resource account is called the first user terminal. Among them, the first user may include but is not limited to: corporate users, marketing personnel, and individual users. In addition, the first user can also register page resource accounts on different web pages, and the server corresponding to different web pages can save the page resource account of the first user.

[0035] The first functional plug-in corresponds to a first workflow start event, and the first workflow start event is used to trigger the execution of the clue acquisition workflow corresponding to the first functional plug-in. That is, when the first workflow start event is generated, the clue acquisition workflow corresponding to the first functional plug-in can be triggered to be automatically executed.

[0036] Among them, the first workflow start event can be implemented as a user operation, an application event, or a system event, without limitation. Among them, user operations include but are not limited to various interaction methods such as touch operation, gesture operation, voice operation, head movement operation, eye movement operation, etc. Touch operation includes but is not limited to single operation, double-click operation, long press operation, and sliding operation, etc. Application events may include but are not limited to: jump events or clock events from other applications to target applications, etc. System events may include but are not limited to: power-on events, events of recovering from sleep mode, and hardware connection events, etc. Optionally, the first workflow start event is implemented as a timed event, and the length of the timer is: 5 minutes, 10 minutes, 1 hour or 12 hours, etc., without limitation.

[0037] In an optional embodiment, during the operation of the target application, it is possible to monitor whether a workflow start event is generated. If the first workflow start event is monitored, the first functional plug-in pointed to by the first workflow start event is determined in response to the first workflow start event; according to the clue acquisition workflow corresponding to the first functional plug-in, at least the following operations are performed: loading operation, simulation operation, interaction operation and identification operation. Among them, in the clue acquisition workflow, the execution order of the following loading operation, simulation operation, interaction operation and identification operation is not limited. For example, a clue acquisition workflow can be implemented as: loading operation->simulation operation->acquisition operation->interaction operation->acquisition operation->interaction operation->acquisition operation. For another example, a clue acquisition workflow can be implemented as: loading operation->simulation operation->loading operation->simulation operation->acquisition operation->interaction operation->interaction operation->interaction operation->acquisition operation. For another example, a clue acquisition workflow can be implemented as: loading operation->simulation operation->loading operation->simulation operation->acquisition operation->interaction operation->interaction operation->acquisition operation. For another example, a clue acquisition workflow can be implemented as: loading operation->simulation operation->interaction operation->acquisition operation.

[0038] In this embodiment, the loading operation is to load and display the first web page associated with the workflow plug-in to which the first functional plug-in belongs, and the first web page displays the first interactive view. The loading operation maintains the page identification information of the first web page, and the page identification information refers to the information used to uniquely identify and describe the first web page. For example, the page identification information may include but is not limited to: URL (Uniform Resource Locator), title tag of the first web page (such as, <title> Tags), page ID, version number or revision date, etc. The first web page can be loaded and displayed according to the page identification information maintained by the loading operation. The first interactive view is not limited, for example, it can be the interactive view displayed by default on the first web page.< / title> Figure 2a As shown, a first web page is provided to display a first interactive view example.

[0039] In this embodiment, the simulation operation is: simulating the first user terminal to perform the first user interaction operation on the first web page, so that the first web page switches from the first interaction view to the second interaction view in response to the first user interaction operation. For example, the implementation method of performing the first user interaction operation on the first web page is not limited. For example, the first web page includes a navigation bar, and the navigation bar includes a plurality of controls, and the first user terminal can be simulated to perform the first interaction operation on any one of the controls in the navigation bar. Alternatively, the first user terminal can be simulated to perform the first user interaction operation on the first interaction view. Among them, the first user interaction operation may include but is not limited to: a single-click operation, a double-click operation, a drag operation, a long press operation, etc.

[0040] The second interactive view may be an interactive view between the first user terminal and other user terminals. For the sake of distinction, the other user terminal is described as the second user terminal, but it is not limited to this. The second user terminal corresponds to a second user, and the second user may be a potential user or fan user of the first user. The second user may register a page resource account on the first web page, and accordingly, the server corresponding to the first web page may also maintain a page resource account of the second user.

[0041] The second interactive view may be a private message chat page between the first user terminal and the second user terminal, or may be an interface for the second user terminal to comment on a video posted by the first user terminal, etc.

[0042] In this embodiment, the interaction operation is: obtaining the first interaction information sent by the second user terminal from the second interaction view; generating the second interaction information corresponding to the first interaction information based on the AI ​​language model, and sending it to the second user terminal, so as to replace the first user terminal and the second user terminal to perform at least one round of interaction, and obtain at least one round of interaction information. Among them, based on the AI ​​language model, the interaction can be quickly established between the first user terminal and the second user terminal, and a reply can be generated in time, ensuring efficient communication.

[0043] Among them, for the convenience of distinction and description, the interactive information sent by the second user end to the first user end is called the first interactive information, and the interactive information sent by the first user end to the second user end is called the second interactive information. The information form of the first interactive information and the second interactive information can be graphic information carried by the card, question and answer information, animated pictures, link information, or voice information or voice interactive controls. Further, the first interactive information and the second interactive information can also be displayed on the second interactive view. Among them, the AI ​​language model is not limited. From the perspective of the technology used by the AI ​​language model, the types of AI language models include but are not limited to: supervised learning models, semi-supervised learning models, unsupervised learning models, reinforcement learning models, convolutional neural networks (CNN) models, recurrent neural networks (RNN) models, generative models or anomaly detection models, etc. From the perspective of the functions implemented by the AI ​​language model, the AI ​​language model includes but is not limited to: natural language understanding models, image or video understanding models, chat models, intelligent customer service models, AI search engines, and multimodal image generation models, etc. Furthermore, the AI ​​language model is implemented as a large language model (LLM). LLM is a deep learning model trained with a large amount of text data that can generate natural language text or understand the meaning of language text. Large language models can handle a variety of natural language tasks, such as text classification, question answering, and dialogue, and are an important path to artificial intelligence.

[0044] In this embodiment, a clue service is maintained, and the clue service user maintains clue information corresponding to the first user, wherein the clue information refers to the contact information and interest information of the potential user, and the clue information can help the first user to better understand the needs of the second user and provide personalized services for the second user. For example, the email address, phone number, social account, and user browsing behavior collected through the website form. The clue service can maintain clue information obtained by multiple workflow plug-ins from multiple web pages.

[0045] Based on this, the acquisition operation is as follows: identifying clue information according to at least one round of interaction information, and if clue information is identified, sending the identified clue information to the clue server, so that the first user terminal can subsequently obtain the required clue information from the clue server. For example, the clue information can be identified from the first interaction information in at least one round of interaction information, or based on the first interaction information and the second interaction information in at least one round of interaction information, and this is not limited.

[0046] Among them, different workflow plug-ins can identify clue information from different web pages associated with them, and upload the identified clue information to the clue server. The first user does not need to reply to the second interaction information from the second user on different web pages one by one, and identify clue information from the existing interaction information. It only needs to maintain the resource account of the first user in the target application to obtain the clue information identified from different web pages from the clue server, thereby improving the efficiency of obtaining clue information.

[0047] In an embodiment of the present application, a method for obtaining clue information is provided, in which different workflow plug-ins are set for different web pages, and the clue obtaining process including loading and displaying web pages, simulating user interaction, AI automatic reply and clue identification is automatically executed, reducing manual intervention, not only improving the efficiency and accuracy of clue acquisition, but also enhancing user experience, and not being invasive to web pages, reducing the risk of being blocked. Furthermore, the clue information identified from different web pages can be directly obtained, without the need to reply to interactive information one by one for different web pages, nor is there any need to identify clue information one by one for different web pages, thereby improving the efficiency of obtaining clue information.

[0048] In an optional embodiment, the first interaction view includes at least one message item, and the clue acquisition workflow indicates that the first user interaction operation is performed on the at least one message item. For example, the first interaction view can be implemented as a message list, which includes at least one message item, and each message item corresponds to a second user. Figure 2a As shown, the message list includes message item E1, message item E2, ..., and message item En (n is a positive integer, and n≥1) as an example for illustration, but is not limited thereto.

[0049] Among them, the implementation method of simulating the first user terminal to perform the first user interaction operation on the first web page so that the first web page switches from the first interaction view to the second interaction view in response to the first user interaction operation is not limited. The following is an exemplary implementation method. For any message item in the first interaction view, according to the identification information of the any message item, the first node corresponding to the any message item is located in the page node tree corresponding to the first web page; a first event object representing the first user interaction operation is constructed, and the first event object is passed to the first node through the event object dispatcher corresponding to the page node tree, so as to simulate the first user terminal performing the first user interaction operation on the first web page, so that the first web page switches from the first interaction view to the second interaction view in response to the first user interaction operation.

[0050] The identification information of the message item may include, but is not limited to: nickname, avatar, name, personal signature, timestamp, and text content information of the second user corresponding to the message item.

[0051] Wherein, the first web page corresponds to a page node tree, and the page node tree includes a plurality of nodes. For example, the page node tree can be implemented as a document object model (Document Object Model, DOM). The page node tree structures the first web page into a tree structure including a plurality of nodes. Each node can correspond to one or more page elements on the first web page, and the message item belongs to a type of page element. Based on this, the first node corresponding to any message item can be located in the page node tree corresponding to the first web page according to the identification information of any message item.

[0052] Wherein, a first event object representing a first user interaction operation can be constructed, and the first event object includes attributes and parameters for accurately simulating the intention of the user interaction operation. The first event object includes, but is not limited to, type, target, timestamp, etc. Wherein, the type of the first event object may include, but is not limited to, mouse events, keyboard events, touch events, and voice events, etc. The target of the first event object may be the first node corresponding to any message item. For example, if the first user interaction operation is a click operation, a mouse click (MouseEvent) object may be created. Wherein, the event object dispatcher corresponding to the page node tree is mainly used to submit the event object to the process of the corresponding node, thereby triggering the operation associated with the event object to be performed on the node. For example, the event object dispatcher can be implemented by an event dispatch (dispatchEvent) method. Based on this, the first event object can be passed to the first node by the event object dispatcher corresponding to the page node tree to simulate the first user end performing the first user interaction operation on the first web page. The first web page can switch from the first interaction view to the second interaction view in response to the first user interaction operation to display the second interaction view.

[0053] Among them, all user interaction operations are implemented through predefined workflow plug-ins to avoid risk control problems that may be caused by direct capture.

[0054] The first user terminal and the second user terminal may interact on the second interaction view. Figure 2b A schematic diagram of displaying a second interactive view on a first web page. Figure 2aThe example in which the second user sends the first interaction information and the first user replies to the second interaction information is used for display, wherein the second interaction information sent by the first user is generated based on the AI ​​language model, and the second interaction information generated by the AI ​​language model is added to the input box of the second interaction view to simulate the input operation of the first user end, and the trigger operation for the "send control" is simulated on the second interaction view, so that the first web page responds to the trigger operation, sends the second interaction information to the second user end, and displays the second interaction information on the second interaction view.

[0055] In an optional embodiment, the first web page includes a third interactive view, the third interactive view includes at least one interactive control, and the clue acquisition workflow indicates that a second user interactive operation is performed on at least one interactive control. The second user interactive operation may include, but is not limited to, a single click operation, a double click operation, a long press operation, a drag operation, a scroll operation, a selection operation, a zoom operation, a voice command, and a gesture operation. The implementation form of the interactive control is not limited. For example, the implementation form of the interactive control may include, but is not limited to, a button, an input box, a drop-down menu, etc. Among them, different interactive controls correspond to different first interactive views. For example, the interactive control may be implemented as at least one of a private message control and a comment control. For example, if the interactive control is implemented as a private message control, the first interactive view is implemented as a private message list, and the private message list includes multiple private message entries. For another example, if the interactive control is implemented as a comment control, the first interactive view is implemented as a comment list, and the comment list includes multiple comment entries. Based on this, the method also includes: for any interactive control, simulating the first user end to perform a second user interactive operation on the any interactive control, so that the first web page responds to the second user interactive operation and switches from the third interactive view to the first interactive view corresponding to any interactive control.

[0056] Among them, the introduction of the third interactive view and its interactive controls enables the clue acquisition workflow to cover more user interaction scenarios. By simulating the first user end to perform the second user interaction operation on any interactive control, the interaction level with the user is further deepened and the interaction depth is enhanced. According to the operation of the first user end in the third interactive view, the interaction path is flexibly adjusted, and the third interactive view is switched to the first interactive view corresponding to different interactive controls to ensure the coherence and integrity of the interaction logic.

[0057] Optionally, the implementation method of simulating the first user end to perform a second user interaction operation on any interactive control so that the first web page switches from the third interactive view to the first interactive view corresponding to any interactive control in response to the second user interaction operation is not limited. The following exemplary implementation method provides a specific implementation method, according to the identification information of any interactive control, locate the second node corresponding to any interactive control in the page node tree corresponding to the first web page; construct a second event object representing the second user interaction operation, and pass the second event object to the second node through the event object dispatcher corresponding to the page node tree, so as to simulate the first user end to perform a second user interaction operation on any interactive control, so that the first web page switches from the third interactive view to the first interactive view corresponding to any interactive control in response to the second user interaction operation. Among them, the introduction of the page node tree and the event object dispatcher can be referred to the aforementioned embodiment, which will not be repeated here. For the introduction of the second event object, please refer to the aforementioned introduction to the first event object, which will not be repeated here. Among them, through the identification information of any interactive control (such as identity identifier (ID), class name, tag name, etc.), the second node corresponding to the interactive control can be accurately located in the page node tree corresponding to the first web page, ensuring that each simulation operation is targeted at the correct node, avoiding unnecessary misoperations, and improving the accuracy and reliability of the interaction.

[0058] In an optional embodiment, when clue information is identified from at least one round of interaction information, the first user terminal may stop performing at least one round of interaction with the second user terminal, and send the clue information to the clue service terminal so that the first user terminal can subsequently obtain the required clue information from the clue service terminal. The first user terminal may establish an interaction channel with the second user terminal corresponding to the clue service for subsequent interaction. The interaction channel may be implemented as a voice call, a video call, a phone call, an email, etc.

[0059] Optionally, if clue information is not identified from at least one round of interaction information, second interaction information corresponding to the first interaction information is generated based on the AI ​​language model to replace the first user terminal and the second user terminal for the next round of interaction until clue information is identified from the existing interaction information.

[0060] Among them, the method introduces a loop mechanism. When clue information is not identified from at least one round of interaction information, new interaction information continues to be generated based on the AI ​​language model for the next round of interaction. And set a clear termination condition, that is, once the clue information is successfully identified, the interaction is immediately stopped and the clue information is sent to the clue server to ensure the efficiency of the process. By accumulating more information through multiple rounds of interaction, the AI ​​language model can analyze the content of the conversation more comprehensively, laying the foundation for more accurate identification of clue information in the future. Furthermore, according to the results of each round of interaction, the dialogue strategy can be adjusted in real time to increase the possibility of identifying clue information.

[0061] In an optional embodiment, the implementation method of identifying clue information from the at least one round of interaction information is not limited. The following exemplarily provides an implementation method, providing a clue information extraction model, which is used to identify and extract clue information of potential users from at least one round of unstructured or semi-structured interaction information. Specifically, the at least one round of interaction information is input into the clue information extraction model to extract clue information to obtain at least one candidate clue information; determine whether there is target clue information that meets the set clue format conditions in the at least one candidate clue information, and if so, use the target clue information as clue information identified from the at least one round of interaction information.

[0062] The candidate clue information may include but is not limited to: contact information (telephone number or email address, etc.), service requirement description, questions or feedback from the second user, expression of the second user's intention (such as seeking help or consulting services), etc.

[0063] Among them, the clue format conditions refer to the format conditions that the clue information should meet according to the service requirements. The clue format conditions may include, but are not limited to: contact information format conditions, service requirement description format conditions, and intent expression format conditions. For example: the contact information format condition is implemented as a valid email address or phone number format. The service requirement description format condition may include specific keywords or phrases, such as "need help" or "project support". The intent expression format condition may be in accordance with predefined intent classifications, such as "seeking solutions" or "requesting quotes". Among them, each candidate clue information can be judged one by one to verify whether it meets the set clue format conditions. For example, use regular expressions to match phone numbers and email addresses, or use natural language processing (NLP) technology to identify keywords in service requirement descriptions.

[0064] Wherein, when the clue format conditions include multiple types, if the candidate clue information satisfies any one of the clue format conditions, it can be determined that the candidate clue information meets the set clue format conditions. Alternatively, if the candidate clue information satisfies all of the clue format conditions, it can be determined that the candidate clue information meets the set clue format conditions. When the clue format conditions are implemented as one type, if the candidate clue information satisfies the clue format condition, it can be determined that the candidate clue information meets the set clue format conditions.

[0065] Among them, by inputting at least one round of interactive information into the clue information extraction model to extract clue information, and judging whether there is target clue information that meets the set clue format conditions, this method not only improves the accuracy and efficiency of clue identification, but also ensures the quality and consistency of clue information.

[0066] In an optional embodiment, no matter which web page is associated with the workflow plug-in, a second functional plug-in may be included. The second functional plug-in is used to execute the video publishing workflow for the first user terminal. The second functional plug-in corresponds to a second workflow start event, and the second workflow start event can be implemented as at least one of a user operation, an application event, and a system event. For a detailed introduction to user operations, application events, and system events, please refer to the introduction to the first workflow start event, which will not be repeated here.

[0067] Based on this, before responding to the first workflow start event and determining the first functional plug-in pointed to by the first workflow start event, the method also includes: during the operation of the target application, it is possible to monitor whether a workflow start event is generated. If a second workflow start event is detected to occur, then in response to the second workflow start event, the second functional plug-in pointed to by the second workflow start event is determined, and according to the video publishing workflow corresponding to the second functional plug-in, at least the following operations are performed: loading operation and simulation operation. Among them, in the video publishing workflow, the execution order of the following loading operation and simulation operation is not limited. For example, a video publishing workflow can be implemented as: loading operation->simulation operation->loading operation->simulation operation->simulation operation; for another example, a video publishing workflow can be implemented as: loading operation->simulation operation->simulation operation.

[0068] Among them, the loading operation: loading and displaying the first web page associated with the second functional plug-in, the first web page includes a fourth interactive view, and the fourth interactive view includes a video upload control.

[0069] Among them, the simulation operation: simulates the third user interaction operation of the first user terminal uploading the target video through the video upload control, so that the first web page responds to the third user interaction operation, switches from the fourth interaction view to the fifth interaction view, and displays the target video on the fifth interaction view, and the fifth interaction view includes a video publishing control.

[0070] The simulation operation is as follows: simulating the fourth user interaction operation performed by the first user terminal on the video publishing control, so that the first web page responds to the fourth user interaction operation and publishes the target video to the video server corresponding to the first web page, so that the video server provides the target video to the second user terminal, and the second user terminal sends the first interaction information to the first user terminal for the target video. The first interaction information includes at least one of: comment information for the target video and private message information for the target video.

[0071] The specific implementation of the third user interaction operation and the fourth user interaction operation is the same or similar to the first user interaction operation and the second user interaction operation, and details can be referred to the above embodiment. The first user interaction operation, the second user interaction operation, the third user interaction operation and the fourth user interaction operation can be partially the same, all the same or different, and there is no limitation on this.

[0072] The video server is used to maintain videos published by various users, and can push the published videos to the first user terminal or the second user terminal.

[0073] The target video is a video published by the first user terminal. The video content of the target video is not limited, for example, it can be a character video, a landscape video or a cartoon video.

[0074] Optionally, in the case where the second user terminal sends the first interaction information to the first user terminal for the target video, an implementation method of generating second interaction information corresponding to the first interaction information based on the AI ​​language model includes: generating prompt words according to the first interaction information, the target video and the prompt word template, the prompt words including task description information, and the task description information indicating the task that the AI ​​language model needs to complete; calling the AI ​​language model according to the prompt words to generate second interaction information corresponding to the first interaction information according to the task description information.

[0075] For example, the AI ​​language model corresponds to a prompt word template, which includes task description information. The task description information may include: generating comment information for the first interaction information based on the content in the target video and the first interaction information of the second user.

[0076] Wherein, the target video can be encoded to obtain a coding feature, and the coding feature and the second interactive information are filled into the prompt word template to obtain the prompt word. Wherein, the method for encoding the target video is not limited. For example, multiple video frames can be used from the target video, and the multiple video frames are encoded respectively to obtain the coding features of the multiple video frames. According to the coding features of the multiple video frames, the coding features of the target video are generated. For example, the coding features of multiple video frames can be spliced ​​to obtain the coding features of the target video, or the coding features of multiple video frames can be weighted averaged to obtain the coding features of the target video. Alternatively, average pooling operations (Average Pooling), maximum pooling operations (Max Pooling) and other operations are performed on the coding features of multiple video frames to obtain the coding features of the target video.

[0077] Among them, the method of encoding the original image is not limited. For example, the encoding method may include but is not limited to: base64 encoding, lossy compression encoding, lossless compression encoding, and image feature encoding. Among them, Base64 encoding is an encoding method that converts binary data into a string format. Lossy compression is an encoding method that reduces the image size by reducing part of the information in the original image, such as the JPEG format. Lossless compression is an encoding method that reduces the image size without losing image quality, such as PNG and GIF formats. Image feature encoding is a method of extracting feature information from the original image, such as HOG (Histogram of Oriented Gradients), SIFT (Scale Invariant Feature Transform), etc.

[0078] Figure 3a A flowchart of interactions among a first user terminal, a first web page, a workflow plug-in, a server of the first web page, an AI language model, and a clue server is provided for an exemplary embodiment of the present application.

[0079] 30. The first user terminal registers an application resource account on the target application;

[0080] 31. The first user terminal registers a page resource account on the first web page;

[0081] 32. The workflow plug-in triggers execution of the clue acquisition workflow corresponding to the first function plug-in on the first web page;

[0082] 33. The workflow plug-in provides the first interaction information of the second user end sent by the server end corresponding to the first web page to the AI ​​language model;

[0083] 34. The workflow plug-in calls the AI ​​language model to generate second interaction information corresponding to the first interaction information, and provides the second interaction information to the first web page;

[0084] 35. The first web page provides the second interaction information to its corresponding server, so that the server provides the second interaction information to the second user terminal;

[0085] 36. The workflow plug-in identifies clue information from at least one round of interaction information and uploads the clue information to the clue server;

[0086] The above steps 32 to 36 may be repeated multiple times, and each time, steps 32 to 36 may be performed for different web pages;

[0087] 37. The first user terminal queries the clue information from the clue server terminal through the application resource account.

[0088] Regarding the embodiments of the present application Figure 1 The detailed implementation and beneficial effects of each step in the method have been described in detail in the aforementioned embodiments and will not be elaborated here.

[0089] This application also provides a workflow plug-in publishing method as follows:

[0090] Step 11, respond to the workflow construction instruction and display the workflow construction interface to the user; the workflow construction interface includes: multiple work icons; each work icon is associated with a function code.

[0091] Step 12: In response to at least one selection operation on a work icon during each workflow drawing, a plurality of selected target work icons are displayed in the work area of ​​the workflow construction interface to be used for constructing a functional plug-in corresponding to the workflow.

[0092] Step 13: In response to the connection operation for multiple target work icons, a connection relationship is established between the multiple target work icons, and the function codes corresponding to the multiple target work icons are spliced ​​according to the connection relationship to form a function plug-in corresponding to the workflow.

[0093] Step 14: Integrate multiple functional plug-ins obtained by multiple workflow drawings into a workflow plug-in, and publish the workflow plug-in to a target browser on other electronic devices. Different functional plug-ins correspond to different workflows, and different workflows are used to implement different functional operations.

[0094] Among them, the above users may be research and development personnel.

[0095] Among them, each work icon can correspond to an operation in the workflow. For example, taking the clue acquisition workflow including loading operation -> simulation operation -> interaction operation -> acquisition operation as an example, the clue acquisition workflow may include: loading icon -> simulation icon -> interaction icon -> acquisition icon. Furthermore, a delay icon may be inserted between the loading icon -> simulation icon -> interaction icon -> acquisition icon, for example, loading icon -> delay icon -> simulation icon -> delay icon -> interaction icon -> delay icon -> acquisition icon. The delay icon is used to execute the function code corresponding to the next icon after the delay setting time. The delay setting time may include but is not limited to: 1500ms, 5000ms, etc., and there is no limitation on this.

[0096] Among them, workflow plug-ins can include Figure 1 A first functional plug-in and a second functional plug-in in the illustrated embodiment.

[0097] The workflow plug-in publishing method is applicable to the first electronic device, which can be any type of electronic device, such as a server or a terminal device, which can be a mobile phone, a tablet computer or a computer, etc., and this embodiment does not limit this. A target browser can be deployed on the electronic device.

[0098] In this embodiment, in response to the workflow construction instruction, the workflow construction interface can be displayed to the user through the target browser. The workflow construction instruction can be issued by the user corresponding to the first electronic device, or it can be automatically issued based on the rules of timing construction, which is not limited in this embodiment. Figure 3b As shown, the user can click on the interactive control named "New Workflow" to issue a workflow construction instruction.

[0099] The workflow construction interface may include: multiple work icons and work areas. Figure 3c As shown, the left side of the work construction interface displays multiple work icons including "delay", "clipboard" and "parameter prompt", and the right side of the work construction interface can display the work area. Figure 3c The click element in is an example of a simulated icon. Each work icon is associated with a function code. The association here can be understood as the binding of the work icon displayed on the front end and the function code on the back end. It should be noted that Figure 3b and Figure 3c The interface layout shown is only an example and does not limit the actual interface layout.

[0100] Among them, the function code is used to perform the corresponding operation. For example, the function code associated with the work icon named "delay" can be used to delay the setting time. For example, the function code associated with the work icon named "model call" can be used to call a pre-trained deep learning model. For example, Figure 1The AI ​​language model in the illustrated embodiment.

[0101] Users can build one or more functional plug-ins corresponding to workflows on the workflow construction interface according to their own needs. Different functional plug-ins correspond to different workflows, and different workflows are used to implement different functional operations. Figure 3b As shown, on the right side there are displayed a function plug-in A1 named "Automatic Reply to Comments" and a function plug-in A2 named "Live Plug-in". Function plug-in A1 can be used to automatically reply to the comments of the audience in the live broadcast room, and function plug-in A2 can be used for automatic reply, looping and sending welcome messages to users who have just entered the live broadcast room.

[0102] In this embodiment, the user can draw a workflow one or more times. In each workflow drawing, the user can perform one or more selection operations on the work icon. The user can drag at least one work icon to the work area to complete the operation of the work icon, or directly click the work icon to complete the operation of the work icon. This embodiment does not limit this. Correspondingly, in response to at least one selection operation on the work icon, multiple selected target work icons can be displayed in the work area of ​​the workflow construction interface for use in constructing the functional plug-in corresponding to the workflow. Figure 3d As shown, three target work icons of "click element", "delay" and "insert data" can be displayed in the work area.

[0103] Afterwards, the user may connect multiple target work icons, and correspondingly, in response to the connection operation on multiple target work icons, a connection relationship may be established between the multiple target work icons. Figure 3e As shown, the user can connect between the three target work icons of "Click Element", "Delay" and "Insert Data". Correspondingly, a connection relationship can be established between the "Click Element" icon and the "Delay" icon, as well as a connection relationship between the "Delay" icon and the "Insert Data" icon.

[0104] After establishing the connection relationship, the function codes corresponding to the multiple target work icons can be spliced ​​according to the connection relationship to form a function plug-in corresponding to the workflow. Among them, the function codes corresponding to the multiple target work icons can be directly spliced, and the function plug-in corresponding to the workflow can be formed according to the spliced ​​code. It is also possible to fill the preset code snippet for code connection between the function codes corresponding to any adjacent target work icons, thereby completing the splicing between the function codes corresponding to any adjacent target work icons, and forming the function plug-in corresponding to the workflow according to the spliced ​​code. Among them, when forming the function plug-in corresponding to the workflow according to the spliced ​​code, the preset resource configuration rules can be used to package resources and construct plug-ins according to the spliced ​​code, so as to form the function plug-in corresponding to the workflow, wherein the resource configuration rules are used to specify the configuration files and resource files of the function plug-in.

[0105] Based on the above steps, multiple functional plug-ins obtained by multiple workflow drawing can be integrated into a workflow plug-in, and the workflow plug-in can be published to a target browser on other electronic devices. The integration here can be understood as a packaging operation, that is, the workflow plug-in includes multiple functional plug-ins.

[0106] After publishing the workflow plug-in to the target browser on other electronic devices, merchants of other electronic devices can execute at least one of the functional plug-ins through the workflow plug-in according to their own needs, thereby realizing the automation of functions. For example, after the merchant clicks on the functional plug-in called "live broadcast plug-in", the target browser on other electronic devices can automatically reply, send out the script in a loop, and send a welcome message to the new users entering the live broadcast room, thereby improving the business efficiency of the merchant.

[0107] In this embodiment, a workflow construction interface can be displayed; in each workflow drawing, in response to at least one selection operation for a work icon, multiple selected target work icons are displayed in the work area of ​​the workflow construction interface; in response to an icon connection operation, a connection relationship is established between multiple target work icons, and the function codes corresponding to the multiple target work icons are spliced ​​according to the connection relationship to form a function plug-in corresponding to the workflow; multiple function plug-ins obtained by multiple workflow drawings are integrated into a workflow plug-in, and the workflow plug-in is published to a target browser on other electronic devices, and different function plug-ins correspond to different workflows, and different workflows are used to implement different functional operations. In this way, a workflow plug-in can be generated more efficiently to improve the business efficiency of merchants.

[0108] In some optional embodiments, the work icons may include first-category icons and second-category icons. Correspondingly, the function codes may also be divided into two categories, namely, first-category function codes associated with each first-category icon and second-category function codes associated with each first-category icon.

[0109] Among them, the first type of function code is a function code generated based on the function interface of the target browser, and this type of code can be used to implement functions based on the inherent functions of the target browser. For example, the target browser may have an inherent function of "click trigger", and a first type of function code can further implement the "simulated click" function based on the inherent function of "page element click". For example, a simulated icon can be an example of a first type of icon. Different from the first type of function code, the second type of function code is generated based on the user's custom operation and can be used to implement custom functions.

[0110] The following will explain how to generate the first type of function code and the second type of function code respectively according to different situations:

[0111] (1) First-class function code: The first-class function code can be obtained by encapsulating the script encapsulated based on the function interface of the target browser. The function interface here refers to the API (Application Programming Interface) provided by the target browser that allows developers to interact with it. The following is an example:

[0112] ① Encapsulate the first script as the first function code for creating a new tab, and establish an association relationship between the first function code and the first icon; the first script is used to control the target browser to open the live page. By establishing the association relationship, the first icon can have the function of controlling the target browser to open the live page. If the first script is used to control the target browser to open a web page and play a short video on the web page. By establishing the association relationship, the first icon can have the function of controlling the target browser to open a web page and play a short video on the web page.

[0113] ② Encapsulate the second script as a second function code for clicking a page element, and establish an association relationship between the second function code and the second icon; the second script is used to control the target browser to click the target page element. By establishing the association relationship, the second icon can have the function of controlling the target browser to click the target page element. The target page element can be any page element, such as a confirmation control and an expansion control, etc.

[0114] Based on this, the first function code and the second function code can be used as the first category function code, and the first icon and the second icon are the first category icons. The first script and the second script are encapsulated based on the functional interface of the target browser. The above ① and ② are only exemplary descriptions, and the first category function code can also include other function codes, such as function codes for simulating page scrolling effects, etc.

[0115] (2) Second-class function code: Display the editing window of the second-class function code to the user so that the user can input the target script; encapsulate the target script to obtain the second-class function code corresponding to the target script. The following is an example:

[0116] ③ Encapsulate the third script as a third function code for capturing comment information, and establish an association relationship between the third function code and the third icon; the third script is used to determine the comment information from the live page according to the preset information screening rules. The information screening rules can be used to identify which information is comment information on the live page. Through the establishment of the association relationship, the third icon can have the function of capturing comment information. Alternatively, encapsulate the third script as a third function code for capturing private message information, and establish an association relationship between the third function code and the third icon; the third script is used to determine the private message information from the private message list according to the preset information screening rules. The information screening rules can be used to identify which information is private message information in the private message list. Through the establishment of the association relationship, the third icon can have the function of capturing private message information.

[0117] ④ Encapsulate the fourth script as a fourth function code for calling the deep learning model, and establish an association relationship between the fourth function code and the fourth icon; the fourth script is used to input the comment information into the deep learning model to output reply information to the comment information in the live broadcast page. Among them, the deep learning model can be used to generate corresponding reply information for the comment information in the live broadcast page. By establishing an association relationship, the fourth icon may have the function of calling the deep learning model. Alternatively, encapsulate the fourth script as a fourth function code for calling the deep learning model (or, AI language model), and establish an association relationship between the fourth function code and the fourth icon; the fourth script is used to input the first interaction information sent by the second user terminal into the deep learning model (or, AI language model) to display it in the private message interaction view (such as, Figure 1 In the second interaction view of the embodiment shown, the second interaction information in response to the first interaction information sent by the second user terminal is output. By establishing the association relationship, the fourth icon can have the function of calling the deep learning model (or, AI language model).

[0118] ⑤ Encapsulate the fifth script as the fifth function code for sending a welcome message; the fifth script is used to send a welcome message to the newly added viewers when the number of newly added viewers on the live page is detected. By establishing the association relationship, the fifth icon can have the function of sending a welcome message.

[0119] ⑥ Encapsulate the sixth script as the sixth function code for sending the advertising slogan, and establish an association relationship between the sixth function code and the sixth icon; the sixth script is used to output the advertising slogan in the live broadcast page according to the set sending cycle. By establishing the association relationship, the sixth icon can have the function of sending the advertising slogan.

[0120] The third icon to the sixth icon above are the second category icons. The above ③-⑥ are only exemplary descriptions, and the second category function codes may also include other function codes, such as function codes for outputting set pictures in the live broadcast room, etc.

[0121] In this way, more abundant work icons with various functions can be provided to users, so that users can generate workflows with richer functions.

[0122] Of course, the above classification method of work icons is only an optional method. In actual scenarios, work icons can also be divided into other categories according to the functions that can be realized by the corresponding function codes. For example, work icons can include Figure 3f The various work icons of the Web interaction class shown, the work icons may also include Figure 3g The various work icons of the data class shown can be classified in any way according to actual needs, which will not be repeated here.

[0123] On the basis of the above embodiment, step 12 in the above embodiment "responding to at least one selection operation on the work icon, displaying multiple selected target work icons in the work area of ​​the workflow construction interface" can be implemented based on the following method: when the workflow is a live broadcast customer acquisition workflow, responding to multiple selection operations on the work icon, dragging the first icon, the second icon, the third icon, the fourth icon, the fifth icon and the sixth icon to the work area, wherein each icon is used to implement the function of its associated function code. Among them, live broadcast customer acquisition can be understood as the process of attracting potential customers and converting them into actual customers when displaying and promoting products in the live broadcast room.

[0124] Correspondingly, step 13 in the aforementioned embodiment, "In response to the connection operation for multiple target work icons, a connection relationship is established between the multiple target work icons, and the function codes corresponding to the multiple target work icons are spliced ​​according to the connection relationship to form a function plug-in corresponding to the workflow" can be implemented based on the following method: In response to the connection operation for the first icon to the sixth icon, a connection relationship is established between the first icon and the sixth icon, and the function codes corresponding to the first icon to the sixth icon are spliced ​​according to the connection relationship to form a function plug-in corresponding to the live broadcast customer acquisition workflow. The function plug-in can be used to execute the following steps in sequence: control the target browser to open the live broadcast page, control the target browser to click on the target page element (such as the open control corresponding to the comment area), capture comment information, input the comment information into the deep learning model to output reply information to the comment information in the live broadcast page, send a welcome message to the newly added viewers when the number of new viewers on the live broadcast page is detected, and output advertising slogans in the live broadcast page according to the set sending cycle.

[0125] The functional plug-in corresponding to the live broadcast customer acquisition workflow generated in this way can help merchants convert viewers who watch the live broadcast (i.e. potential customers) into actual customers.

[0126] Of course, the above is only an example. Different work icons selected by users will generate different functions of the workflow based on the work icons. For example, if the user only selects the first icon and the sixth icon and connects them, the function plug-in generated based on the function codes corresponding to the first icon and the sixth icon can control the target browser to open the live page and output the advertising slogan in the live page according to the set sending cycle, but will not execute other functions such as "capturing comment information".

[0127] In an optional embodiment, step 13 in the aforementioned embodiment "in response to connection operations on multiple target work icons, a connection relationship is established between the multiple target work icons, and the function codes corresponding to the multiple target work icons are spliced ​​according to the connection relationship to form a function plug-in corresponding to the workflow" can be implemented based on the following method: in response to connection operations on loading icons, simulation icons, interactive icons, and identification icons, a connection relationship is established between the loading icons, simulation icons, interactive icons, and identification icons, and the function codes corresponding to the loading icons, simulation icons, interactive icons, and identification icons are spliced ​​according to the connection relationship to form a function plug-in corresponding to the clue acquisition workflow. This functional plug-in can be used to execute the following steps in sequence: load and display the first web page associated with the workflow plug-in to which the first functional plug-in belongs, and the first web page displays a first interactive view; simulate the first user terminal performing a first user interactive operation on the first web page, so that the first web page switches from the first interactive view to the second interactive view in response to the first user interactive operation; obtain the first interactive information sent by the second user terminal from the second interactive view; generate the second interactive information corresponding to the first interactive information based on the AI ​​language model, and send it to the second user terminal to replace the first user terminal to perform at least one round of interaction with the second user terminal to obtain at least one round of interaction information; when clue information is identified based on at least one round of interaction information, the identified clue information is sent to the clue server, so that the first user terminal can subsequently obtain the required clue information from the clue server. For detailed introduction, please refer to Figure 1 The illustrated embodiments will not be described in detail here.

[0128] In some optional embodiments, the workflow construction interface may also include multiple types of connection lines for the user to choose from, and different types of connection lines may be used to establish different connection relationships between target task icons. The following is an example:

[0129] The first type of connecting line can be used to establish a connection relationship for sequential execution between target work icons. For example, the target work icon T1 and the target work icon T2 are connected by the first type of connecting line. Based on this, the two target work icons can be executed sequentially in the order of the target work icon T1 and the target work icon T2.

[0130] The second type of connection line can be used to establish a connection relationship for parallel execution between target work icons. For example, the target work icon T3 and the target work icon T4 are connected using the second type of connection line. Based on this, the target work icon T3 and the target work icon T4 can be executed simultaneously.

[0131] The third type of connection line can be used to establish a jump connection relationship between target work icons. For example, by connecting the target work icon T5 and the target work icon T6 using the third type of connection line, you can jump directly to the target work icon T4 without executing the target work icon T3 to execute the target work icon T4.

[0132] The fourth type of connecting line can be used to establish a connection relationship between target work icons based on custom conditions. For example, the target work icon T7 and the target work icon T7 can be connected using the fourth type of connecting line. Based on this, the target work icon T7 can be executed first, and then it can be determined whether the output value of the function code corresponding to the target work icon T7 meets the conditions set by the user. If yes, the target work icon T8 is executed, and if not, the execution of the target work icon is stopped.

[0133] Based on this, in response to the connection operation for multiple target work icons, when a connection relationship is established between multiple target work icons, a target connection relationship can be established between the multiple target work icons pointed to by the target connection line in response to the drag operation of the target connection line for the target type.

[0134] In this way, a more functional plug-in with richer functions can be generated for the user through different types of connection lines.

[0135] On the basis of the above embodiments, a workflow plug-in publishing method is also provided in an embodiment of the present application, which is applicable to a second electronic device, on which a target browser is deployed, and the method includes:

[0136] Step 51, respond to the plug-in start operation, and obtain the workflow plug-in; the workflow plug-in is obtained by integrating multiple functional plug-ins obtained by multiple workflow drawings; any functional plug-in is formed by splicing the functional codes corresponding to multiple target work icons according to the connection relationship of multiple target work icons.

[0137] Step 52: Display a function plug-in selection window corresponding to the workflow plug-in on the target browser, so that the user can select multiple function plug-ins; each function plug-in corresponds to a workflow.

[0138] Step 53: In response to the user's function plug-in selection operation, multiple target task icons corresponding to the target function plug-in selected by the user and their respective connection relationships are displayed in the workflow details window.

[0139] Step 54: In response to the user's workflow start operation, execute the target code corresponding to the target function plug-in selected by the user; wherein, after each execution of any function code in the target code, highlight the target work icon corresponding to any function code to allow the user to perceive the progress of the workflow execution. Figure 3h As shown, users can click Figure 3h Launch control in the upper right corner to start the workflow called "Live Plug-in".

[0140] In this way, the business efficiency of the merchant of the second electronic device can be improved based on the workflow plug-in generated by the first electronic device, and the user can perceive the execution progress by visualizing the execution progress of the workflow.

[0141] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 101 to 103 can be device A; for another example, the execution subject of steps 101 and 102 can be device A, and the execution subject of step 103 can be device B; and so on.

[0142] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and the sequence numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0143] Figure 4 A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of the present application, wherein the electronic device is deployed with multiple workflow plug-ins corresponding to a target application, and different workflow plug-ins are associated with different web pages in the target application, and the workflow plug-ins include a first functional plug-in for executing a clue acquisition workflow for a first user terminal; Figure 4As shown, the device includes: a memory 44 and a processor 45.

[0144] The memory 44 is used to store computer programs and can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method used to operate on the electronic device.

[0145] The processor 45 is coupled to the memory 44 and is used to execute the computer program in the memory 44, so as to: the method includes: during the operation of the target application, in response to the first workflow start event, determining the first function plug-in pointed to by the first workflow start event; obtaining the workflow according to the clue corresponding to the first function plug-in, and at least performing the following operations: loading and displaying the first web page associated with the workflow plug-in to which the first function plug-in belongs, the first web page displaying a first interaction view; simulating the first user terminal to perform a first user interaction operation on the first web page, so that the first web page switches from the first interaction view to the second interaction view in response to the first user interaction operation; obtaining the first interaction information sent by the second user terminal from the second interaction view; generating the second interaction information corresponding to the first interaction information based on the AI ​​language model, and sending it to the second user terminal, so as to replace the first user terminal to perform at least one round of interaction with the second user terminal to obtain at least one round of interaction information; in the case where clue information is identified according to the at least one round of interaction information, sending the identified clue information to the clue service end, so that the first user terminal can subsequently obtain the required clue information from the clue service end.

[0146] In an optional embodiment, the first interactive view includes at least one message item, and the clue acquisition workflow indicates performing a first user interactive operation on at least one message item; when simulating the first user terminal performing the first user interactive operation on the first web page so that the first web page switches from the first interactive view to the second interactive view in response to the first user interactive operation, the processor 45 is specifically used to: for any message item, locate the first node corresponding to the any message item in the page node tree corresponding to the first web page according to the identification information of the any message item; construct a first event object representing the first user interactive operation, and pass the first event object to the first node through the event object dispatcher corresponding to the page node tree to simulate the first user terminal performing the first user interactive operation on the first web page, so that the first web page switches from the first interactive view to the second interactive view in response to the first user interactive operation.

[0147] In an optional embodiment, the first web page includes a third interactive view, the third interactive view includes at least one interactive control, and the clue acquisition workflow indicates performing a second user interaction operation on at least one interactive control; the processor 45 is also used to: for any interactive control, simulate the first user terminal performing a second user interaction operation on the any interactive control, so that the first web page switches from the third interactive view to the first interactive view corresponding to the any interactive control in response to the second user interaction operation.

[0148] Optionally, when simulating the first user terminal performing the second user interaction operation on any interactive control so that the first web page switches from the third interaction view to the first interaction view corresponding to any interactive control in response to the second user interaction operation, the processor 45 is specifically used to: locate the second node corresponding to any interactive control in the page node tree corresponding to the first web page according to the identification information of any interactive control; construct a second event object representing the second user interaction operation, and pass the second event object to the second node through the event object dispatcher corresponding to the page node tree, so as to simulate the first user terminal performing the second user interaction operation on any interactive control, so that the first web page switches from the third interaction view to the first interaction view corresponding to any interactive control in response to the second user interaction operation.

[0149] In an optional embodiment, the processor 45 is also used to: if clue information is not identified from the at least one round of interaction information, continue to generate second interaction information corresponding to the first interaction information based on the AI ​​language model to replace the first user terminal and the second user terminal for the next round of interaction until clue information is identified.

[0150] In an optional embodiment, the processor 45 is also used to: input the at least one round of interaction information into the clue information extraction model to extract clue information to obtain at least one candidate clue information; determine whether there is target clue information that meets the set clue format conditions in the at least one candidate clue information, and if so, use the target clue information as clue information identified from the at least one round of interaction information.

[0151] In an optional embodiment, the workflow plug-in also includes: a second functional plug-in for executing a video publishing workflow for the first user terminal; the processor 45 is also used to: during the operation of the target application, in response to the second workflow start event, determine the second functional plug-in pointed to by the second workflow start event, and perform at least the following operations according to the video publishing workflow corresponding to the second functional plug-in: load and display the first web page associated with the second functional plug-in, the first web page including a fourth interactive view with a video upload control; simulate the third user interaction operation of the first user terminal uploading the target video through the video upload control, so that the first web page switches from the fourth interactive view to the fifth interactive view in response to the third user interaction operation, and displays the target video on the fifth interactive view, the fifth interactive view including a video publishing control; simulate the fourth user interaction operation performed by the first user terminal on the video publishing control, so that the first web page responds to the fourth user interaction operation and publishes the target video to the video server corresponding to the first web page, so that the video server provides the target video to the second user terminal, and the second user terminal sends first interaction information to the first user terminal for the target video.

[0152] Optionally, when the second user terminal sends the first interaction information to the first user terminal for the target video, the processor 45 generates the second interaction information corresponding to the first interaction information based on the AI ​​language model, and is specifically used to: generate a prompt word according to the first interaction information, the target video and a prompt word template, the prompt word including task description information, and the task description information indicating the task that the AI ​​language model needs to complete; call the AI ​​language model according to the prompt word to generate the second interaction information corresponding to the first interaction information according to the task description information.

[0153] Regarding the embodiments of the present application Figure 4 The detailed implementation and beneficial effects of each step in the illustrated device have been described in detail in the aforementioned embodiments and will not be elaborated on here.

[0154] Further, if Figure 4 As shown, the electronic device also includes: a communication component 46, a display 47, a power component 48, an audio component 49 and other components. Figure 4 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 4 In addition, Figure 4The components in the dashed box are optional components, not mandatory components, and the specific components depend on the product form of the electronic device. The electronic device of this embodiment can be implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, or an IOT (Internet of Things) device and a smart wearable device (such as a smart watch, a smart bracelet), or a conventional server, a cloud server, or a server array. If the electronic device of this embodiment is implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, etc., it can include Figure 4 If the electronic device of this embodiment is implemented as a server device such as a conventional server, a cloud server or a server array, it may not include Figure 4 Components within the dashed box.

[0155] Accordingly, the embodiment of the present application further provides a computer-readable storage medium storing a computer program, which can achieve the above-mentioned Figure 1 The steps in the method embodiment shown may be executed by an electronic device.

[0156] Accordingly, the embodiment of the present application further provides a computer program product, including a computer program / instruction, which, when executed by a processor, causes the processor to implement the above Figure 1 The steps in the method embodiment shown may be executed by an electronic device.

[0157] The above-mentioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0158] The above-mentioned communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

[0159] The above-mentioned display includes a screen, and the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0160] The power supply assembly provides power to various components of the device where the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply assembly is located.

[0161] The above-mentioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (Microphone, MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal.

[0162] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-readable storage media (including but not limited to disk storage, compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage, etc.) containing computer-usable program code.

[0163] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0164] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0165] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0166] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interface, network interface and memory.

[0167] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0168] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0169] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0170] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for obtaining clue information, characterized in that: A plurality of workflow plug-ins are deployed in a target application, and different workflow plug-ins are associated with different web pages in the target application. The workflow plug-in includes a first functional plug-in for executing a clue acquisition workflow for a first user terminal. The method includes: During the running of the target application, in response to a first workflow start event, determining a first function plug-in pointed to by the first workflow start event; obtaining a workflow according to a clue corresponding to the first function plug-in, and performing at least the following operations: Loading and displaying a first web page associated with the workflow plug-in to which the first functional plug-in belongs, wherein the first web page displays a first interactive view; Simulating a first user terminal performing a first user interaction operation on the first web page, so that the first web page switches from the first interaction view to a second interaction view in response to the first user interaction operation; Acquire the first interaction information sent by the second user terminal from the second interaction view; generate second interaction information corresponding to the first interaction information based on the AI ​​language model, and send the second interaction information to the second user terminal, so as to replace the first user terminal in performing at least one round of interaction with the second user terminal, and obtain at least one round of interaction information; When clue information is identified according to the at least one round of interaction information, the identified clue information is sent to the clue server, so that the first user terminal can subsequently obtain required clue information from the clue server.

2. The method according to claim 1, characterized in that The first interaction view includes at least one message item, and the clue acquisition workflow indicates to perform a first user interaction operation on the at least one message item; The method of simulating a first user terminal performing a first user interaction operation on the first web page so that the first web page switches from the first interaction view to the second interaction view in response to the first user interaction operation includes: For any message item, according to the identification information of the any message item, locate a first node corresponding to the any message item in the page node tree corresponding to the first web page; Construct a first event object representing the first user interaction operation, and pass the first event object to the first node through the event object dispatcher corresponding to the page node tree to simulate the first user terminal performing the first user interaction operation on the first web page, so that the first web page switches from the first interaction view to the second interaction view in response to the first user interaction operation.

3. The method according to claim 1, characterized in that The first web page includes a third interactive view, the third interactive view includes at least one interactive control, and the clue acquisition workflow indicates performing a second user interactive operation on the at least one interactive control; the method further includes: For any interactive control, simulate the first user terminal performing a second user interaction operation on the any interactive control, so that the first web page switches from the third interaction view to the first interaction view corresponding to the any interactive control in response to the second user interaction operation.

4. The method according to claim 3, characterized in that The method of simulating the first user terminal to perform a second user interaction operation on any interactive control, so that the first web page switches from the third interaction view to the first interaction view corresponding to any interactive control in response to the second user interaction operation, includes: Locating a second node corresponding to any interactive control in a page node tree corresponding to the first web page according to the identification information of any interactive control; Construct a second event object representing the second user interaction operation, and pass the second event object to the second node through the event object dispatcher corresponding to the page node tree to simulate the first user end performing the second user interaction operation on any interactive control, so that the first web page switches from the third interactive view to the first interactive view corresponding to any interactive control in response to the second user interaction operation.

5. The method according to claim 1, characterized in that The method further comprises: If clue information is not identified from the at least one round of interaction information, second interaction information corresponding to the first interaction information is continuously generated based on the AI ​​language model to replace the first user terminal and the second user terminal for the next round of interaction until clue information is identified.

6. The method according to claim 1, characterized in that Also includes: Inputting the at least one round of interaction information into a clue information extraction model to extract clue information, and obtaining at least one candidate clue information; It is determined whether there is target clue information that meets the set clue format conditions in the at least one candidate clue information. If so, the target clue information is used as clue information identified from the at least one round of interaction information.

7. The method according to claim 1, characterized in that The workflow plug-in further includes: a second function plug-in for executing a video publishing workflow for the first user terminal; the method further includes: During the operation of the target application, in response to a second workflow start event, a second function plug-in pointed to by the second workflow start event is determined, and according to the video publishing workflow corresponding to the second function plug-in, at least the following operations are performed: Loading and displaying a first web page associated with the second function plug-in, wherein the first web page includes a fourth interactive view with a video upload control; Simulate a third user interaction operation of uploading a target video by the first user terminal through the video upload control, so that the first web page switches from the fourth interaction view to the fifth interaction view in response to the third user interaction operation, and displays the target video on the fifth interaction view, wherein the fifth interaction view includes a video publishing control; Simulate the fourth user interaction operation performed by the first user terminal on the video publishing control, so that the first web page responds to the fourth user interaction operation and publishes the target video to the video server corresponding to the first web page, so that the video server provides the target video to the second user terminal, and the second user terminal sends first interaction information to the first user terminal for the target video.

8. The method according to claim 7, characterized in that In a case where the second user terminal sends first interaction information to the first user terminal for the target video, generating second interaction information corresponding to the first interaction information based on the AI ​​language model includes: Generate a prompt word according to the first interaction information, the target video, and a prompt word template, wherein the prompt word includes task description information, and the task description information indicates a task that the AI ​​language model needs to complete; The AI ​​language model is called according to the prompt word to generate second interaction information corresponding to the first interaction information according to the task description information.

9. An electronic device, characterized in that: include: Memory and processor; The memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement the steps in the method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to implement the steps in the method according to any one of claims 1 to 8.

11. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the processor is caused to implement the steps in any one of the methods of claims 1-8.

Images