AB experiment platform intelligent interaction method and device, equipment, storage medium and product
By generating query information vectors and using a large language model for analysis and processing, the problem of reliance on human experience in A/B testing platforms has been solved, achieving automated knowledge acquisition and in-depth interpretation, thus improving user experience and efficiency.
Patent Information
- Application Number
- CN202511503036.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-21
- Publication Date
- 2026-03-03
AI Technical Summary
Existing A/B testing platforms rely mainly on human experience and manual analysis for in-depth interpretation of experimental results, discovery of key influencing factors, and diagnosis of potential anomalies. Users need to consult a large number of related knowledge documents, resulting in a decline in user experience.
By generating query information vectors, retrieving target knowledge block vectors from a vector database, and inputting query prompts into a language big data model for analysis and processing, query answers are generated, thus achieving automated knowledge acquisition and solution.
Users can accurately understand the relevant knowledge of the A/B testing platform without having to browse through a large number of documents, which improves the user experience and analysis efficiency, and supports multi-turn dialogue and multi-scenario interaction.
Smart Images
Figure CN121599092A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to intelligent interaction methods, devices, equipment, storage media, and products for AB experimental platforms. Background Technology
[0002] A / B testing (or comparative testing) is a core technical tool for internet product iteration, operational optimization, and data-driven decision-making. A / B testing involves randomly assigning target users to different scenarios (e.g., version A or version B), collecting and comparing user behavior metrics (such as click-through rate, conversion rate, retention rate, etc.) under different scenarios, and evaluating the effectiveness of different scenarios based on these metrics.
[0003] While traditional A / B testing platforms can meet the statistical requirements of core indicators, they still rely heavily on human experience and manual analysis for in-depth interpretation of experimental results, identification of key influencing factors, and diagnosis of potential anomalies. Using A / B testing platforms requires users to have considerable background knowledge in the field, often necessitating extensive reading of relevant A / B testing documentation to understand the platform, thus reducing the user experience. Summary of the Invention
[0004] This application provides an intelligent interaction method, apparatus, device, storage medium, and product for an AB experiment platform. It generates a query information vector based on experiment query information provided by the user's front end and the AB experiment platform. Based on the query information vector, it retrieves a target knowledge block vector from a vector database, obtains the target knowledge block corresponding to the target knowledge block vector, generates query prompts based on the experiment query information and the target knowledge block, and inputs the query prompts into a preset language model. The language model analyzes and processes the query prompts to obtain the query answer. The vector database records knowledge block vectors corresponding to multiple knowledge blocks. These knowledge blocks are obtained by segmenting and processing based on preset AB experiment knowledge documents. By combining the experiment query information and the target knowledge block to generate query prompts, the language model can accurately output query answers for AB experiment-related knowledge. This effectively solves the technical problem that users need to browse through a large number of related AB experiment knowledge documents to understand the AB experiment platform, leading to a decline in the user experience. It allows users to accurately understand the AB experiment platform without having to browse through a large number of related AB experiment knowledge documents, greatly improving the user experience of the AB experiment platform.
[0005] In a first aspect, embodiments of this application provide an intelligent interaction method for an A / B testing platform, comprising: The experiment query information provided by the user front-end based on the A / B experiment platform is obtained through a microservice cluster, and a query information vector is generated based on the experiment query information. The user front-end submits the experiment query information through an interactive interface. The target knowledge block vector is retrieved from the vector database server based on the query information vector. The vector database server is configured with one or more vector databases. The vector database records knowledge block vectors corresponding to multiple knowledge blocks. The knowledge blocks are obtained by segmenting based on a preset AB experiment knowledge document. Obtain the target knowledge block corresponding to the target knowledge block vector, and generate query prompts based on the experimental query information and the target knowledge block; The query suggestions are input into a preset language model, which then analyzes and processes them to obtain the query answer.
[0006] In a second aspect, embodiments of this application provide an intelligent interactive device for an A / B testing platform, including a vector generation module, a vector retrieval module, a prompt word generation module, a query processing module, and a result feedback module, wherein: The vector generation module is configured to obtain experimental query information provided by the user front-end based on the AB experiment platform through a microservice cluster, and generate a query information vector based on the experimental query information, wherein the user front-end submits the experimental query information through an interactive interface; The vector retrieval module is configured to retrieve target knowledge block vectors from a vector database server based on the query information vector. The vector database server is configured with one or more vector databases, and the vector databases record knowledge block vectors corresponding to multiple knowledge blocks. The knowledge blocks are obtained by segmenting a preset AB experiment knowledge document. The prompt word generation module is configured to obtain the target knowledge block corresponding to the target knowledge block vector, and generate query prompt words based on the experimental query information and the target knowledge block. The query processing module is configured to input the query suggestion words into a preset language model, and the language model analyzes and processes the query suggestion words to obtain the query answer. The result feedback module is configured to send the query answer to the user front-end, so that the user front-end can display the query answer on the interactive interface.
[0007] In a third aspect, embodiments of this application provide an intelligent interactive device for an AB experimental platform, including: a memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the AB experimental platform intelligent interaction method as described in the first aspect.
[0008] In a fourth aspect, embodiments of this application provide a non-volatile storage medium for storing computer-executable instructions, which, when executed by a computer processor, are used to perform the AB experimental platform intelligent interaction method as described in the first aspect.
[0009] In a fifth aspect, embodiments of this application provide a computer program product comprising a computer program stored in a computer-readable storage medium, wherein at least one processor of the device reads from and executes the computer program from the computer-readable storage medium, causing the device to perform the AB experimental platform intelligent interaction method as described in the first aspect.
[0010] This application embodiment generates a query information vector based on the experiment query information provided by the user's front end based on the AB experiment platform. It then retrieves the target knowledge block vector from the vector database based on the query information vector, obtains the target knowledge block corresponding to the target knowledge block vector, and generates query prompts based on the experiment query information and the target knowledge block. These query prompts are input into a preset language model, which analyzes and processes them to obtain the query answer. The vector database records knowledge block vectors corresponding to multiple knowledge blocks. These knowledge blocks are obtained by segmenting them based on preset AB experiment knowledge documents. By combining the experiment query information and the target knowledge block to generate query prompts, the language model can accurately output query answers for AB experiment-related knowledge. This effectively solves the technical problem that users need to browse through a large number of related AB experiment knowledge documents to understand the AB experiment platform, leading to a decline in the user experience. This allows users to accurately understand the AB experiment platform without having to browse through a large number of related AB experiment knowledge documents, greatly improving the user experience of the AB experiment platform. Attached Figure Description
[0011] Figure 1 This is a flowchart of an intelligent interaction method for an AB experimental platform provided in an embodiment of this application; Figure 2 This is a flowchart of another intelligent interaction method for an AB experimental platform provided in an embodiment of this application; Figure 3 This is a flowchart of another intelligent interaction method for an AB experimental platform provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an intelligent interactive device for an AB experimental platform provided in an embodiment of this application; Figure 5This is a schematic diagram of the structure of an intelligent interactive device for an AB experimental platform provided in an embodiment of this application. Detailed Implementation
[0012] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but additional steps not included in the drawings may also be present. The above processes can correspond to methods, functions, procedures, subroutines, subroutines, etc.
[0013] While existing A / B testing platforms have developed relatively mature and standardized solutions for experiment creation, traffic segmentation, data collection, and statistical calculation, significant technical bottlenecks remain in user interaction, in-depth analysis, and intelligent decision-making. Although A / B testing platforms can meet the statistical significance calculation requirements for most core indicators, they still rely heavily on human experience and manual analysis for in-depth interpretation of experimental results, identification of key influencing factors, and diagnosis of potential anomalies. These platforms lack the ability to connect experimental background knowledge (such as past experimental conclusions, product documents, and business rules). They also lack the ability to dynamically generate in-depth analysis conclusions or insights based on users' natural language queries regarding experimental results, and cannot schedule and combine different professional analysis tools (such as experimental indicator analysis and attribution models) for data processing based on complex user query intentions, thus limiting the depth and coverage of A / B testing analysis. Furthermore, existing A / B testing platforms lack built-in experimental analysis capabilities, hindering users' ability to view experimental results effectively. In practice, users can only obtain indicator results data and rely on subjective analysis and judgment based on their personal knowledge and experience. This not only leads to inconsistencies in the analysis results but also makes it difficult for users without a statistical background to draw reliable conclusions. Furthermore, the experimental conclusions, operating procedures, business backgrounds, and platform function introductions accumulated during the use of A / B testing platforms are usually scattered in different places (e.g., platform FAQ reports, document systems, chat logs, etc.), lacking effective unified management and intelligent classification and integration. Based on this, this application provides an intelligent interaction method for A / B testing platforms to solve the technical problem that existing A / B testing platforms require users to browse through a large number of related A / B testing knowledge documents to understand the relevant knowledge, resulting in a decline in the user experience of the A / B testing platform.
[0014] Figure 1 A flowchart of the first intelligent interaction method for an AB experimental platform provided in this application embodiment is given. The intelligent interaction method for an AB experimental platform provided in this application embodiment can be executed by an intelligent interaction device for an AB experimental platform. The intelligent interaction device for an AB experimental platform can be implemented by hardware and / or software and integrated into the intelligent interaction device for an AB experimental platform.
[0015] The following description uses the intelligent interactive device of the AB experimental platform to illustrate the intelligent interactive method of the AB experimental platform. (Reference) Figure 1 The intelligent interaction method of this AB experimental platform includes: S110: Obtain the experiment query information provided by the user front-end based on the AB experiment platform through the microservice cluster, and generate a query information vector based on the experiment query information. The user front-end submits the experiment query information through the interactive interface.
[0016] The intelligent interaction method for the A / B testing platform provided in this application can be applied to an intelligent interaction system for the A / B testing platform. This system is configured with components such as a microservice cluster and a vector database server. The microservice cluster can be used to deploy Retrieval-Augmented Generation (RAG) microservices and pre-set databases (e.g., MySQL). The microservice cluster can also be used to deploy the intelligent interaction device for the A / B testing platform provided in this application and execute business logic related to the intelligent interaction method. Furthermore, it can configure the Retrieval-Augmented Generation microservice based on the business logic related to the intelligent interaction method, utilizing the data processing capabilities of the Retrieval-Augmented Generation microservice to implement the intelligent interaction method based on the A / B testing platform. The vector database server is configured with one or more vector databases and can retrieve corresponding vector data from one or more vector databases based on received search requests.
[0017] For example, the experiment query information provided by the user front-end based on the AB experiment platform is obtained through the microservice cluster. The experiment query information is used to record queries on the AB experiment platform and related knowledge. Users can input the experiment query information in the user front-end based on natural language. The user front-end sends the experiment query information to the intelligent interactive device of the AB experiment platform through HTTP request or server push event (SSE).
[0018] Optionally, after the user frontend establishes communication with the microservice cluster, the microservice cluster provides an interactive interface (e.g., a natural language interactive interface) to the user frontend for display. Users can input information into the interactive interface. This input (including experiment query information, experiment requirement information, and intelligent dialogue information) can be text or voice (the corresponding text can be extracted based on speech recognition technology). For example, after the user frontend establishes communication with the intelligent interactive device of the AB experiment platform, the corresponding interactive interface of the intelligent interactive device is displayed on the user frontend. Users can input experiment query information into the interactive interface, such as "How to push the entire experiment?", "How to conduct experiment canary release?", or related questions.
[0019] In one embodiment, after obtaining the experimental query information provided by the user's front-end based on the A / B testing platform, a query information vector can be generated based on the experimental query information. Optionally, the experimental query information can be processed based on a preset vector generation algorithm to obtain the query information vector corresponding to the experimental query information, or the experimental query information can be processed by a trained vector generation model (which can be deployed locally or call a vector generation model in the cloud) to obtain the query information vector corresponding to the experimental query information.
[0020] S120: Retrieve the target knowledge block vector from the vector database server based on the query information vector. The vector database server is configured with one or more vector databases. The vector databases record knowledge block vectors corresponding to multiple knowledge blocks. The knowledge blocks are obtained by segmenting based on the preset AB experiment knowledge documents.
[0021] The vector database provided in this application records multiple knowledge block vectors. These knowledge block vectors are obtained by extracting vectors from knowledge blocks (e.g., processing knowledge blocks using a pre-defined vector generation algorithm to obtain their corresponding vectors, or extracting vectors from knowledge blocks using a trained vector generation model). The knowledge blocks are obtained by segmenting pre-defined A / B experiment knowledge documents. These A / B experiment knowledge documents can include experimental conclusions, product documents, business rules, platform FAQ reports, document systems, chat logs, operation procedures, business background, platform function introduction documents, platform operation manuals, etc., all related to A / B experiments.
[0022] For example, after obtaining the query information vector corresponding to the experimental query information, a retrieval request for the target knowledge block vector is sent to the vector database server to retrieve the target knowledge block vector in the vector database based on the query information vector. For example, using the query information vector as the query condition, a similarity (e.g., cosine similarity) search is initiated to the vector database server. The vector database server determines one or more knowledge block vectors in the vector database that are closest to the query information vector as the target knowledge block vector and feeds back the retrieved target knowledge block vector to the intelligent interactive device of the AB experimental platform.
[0023] S130: Obtain the target knowledge block corresponding to the target knowledge block vector, and generate query prompts based on the experimental query information and the target knowledge block.
[0024] For example, the target knowledge block corresponding to the target knowledge block vector is obtained, and a query prompt is generated based on the experimental query information and the target knowledge block. For instance, the query prompt is obtained by concatenating the experimental query information and the target knowledge block. Optionally, the query prompt can be obtained by concatenating the experimental query information and the target knowledge block based on a preset query prompt format. For example, if the experimental query information is A, and the target knowledge blocks are B, C, and D, the generated query prompt could be "Answer A based on knowledge B, C, and D".
[0025] S140: Input the query suggestions into the preset language model, and the language model will analyze and process the query suggestions to obtain the query answer.
[0026] S150: Sends the query answer to the user front-end, so that the user front-end can display the query answer on the interactive interface.
[0027] For example, the API interface of a preset language model is called to send the determined query suggestions to the preset language model, which then analyzes and processes the query suggestions to obtain the query answer. After obtaining the query answer, it can be sent to the user's front end, where it can be displayed in the interactive interface. At this time, the interactive interface will display the experimental query information and query answer in the form of a dialogue. Users can also enter new experimental query information in the interactive interface to conduct multi-turn dialogue-based intelligent interaction on the A / B experimental platform.
[0028] The above describes a process where, based on the experimental query information provided by the user's front-end using the AB experiment platform, a query information vector is generated. This vector is then used to retrieve the target knowledge block vector from the vector database, obtaining the target knowledge block corresponding to the vector. Query prompts are generated based on the experimental query information and the target knowledge block, and then input into a pre-defined language model. The language model analyzes and processes these prompts to obtain the query answer. The vector database records knowledge block vectors corresponding to multiple knowledge blocks, which are obtained by segmenting pre-defined AB experiment knowledge documents. By combining the experimental query information and the target knowledge block to generate query prompts, the language model can accurately output query answers for AB experiment-related knowledge. This effectively solves the technical problem of users needing to consult numerous AB experiment knowledge documents to understand the platform, leading to a decreased user experience. The system allows users to accurately understand the AB experiment platform without having to consult a large number of documents, significantly improving the user experience.
[0029] Based on the above embodiments, Figure 2 A flowchart of another intelligent interaction method for an AB experimental platform provided in this application embodiment is given. This intelligent interaction method for an AB experimental platform is a concretization of the above-described intelligent interaction method for an AB experimental platform. (Reference) Figure 2 The intelligent interaction method of this AB experimental platform includes: S210: Divide the AB experiment knowledge documents in the AB experiment knowledge base into blocks to obtain multiple knowledge blocks.
[0030] S220: Determine the knowledge block vector for each knowledge block and save the knowledge block vector as the payload of the knowledge block to the vector database.
[0031] For example, the AB experiment knowledge documents in the AB experiment knowledge base are segmented into multiple knowledge blocks. Each AB experiment knowledge document can be divided into one or more knowledge blocks, and the content within a knowledge block records the same or similar knowledge. Optionally, the AB experiment knowledge documents can be segmented into knowledge blocks based on regular expression segmentation, semantic clustering, summary indexing, or multi-granularity segmentation. Optionally, the AB experiment knowledge documents can be in file formats such as txt, PDF, Word, or Markdown.
[0032] In one embodiment, vectors are extracted from each knowledge block based on a preset vector generation algorithm or a trained vector generation model to obtain the corresponding knowledge block vectors. The knowledge block vectors and knowledge block identification information (such as the filename, file ID, and file category identifier of the AB experiment knowledge document corresponding to the knowledge block) are added to the payload of the corresponding knowledge block, and the knowledge block vectors and payloads are saved in a vector database. The source files can be indexed based on the information in the payload of the knowledge block vectors. This application obtains multiple knowledge blocks by segmenting the AB experiment knowledge documents in the AB experiment knowledge base, and saves the knowledge block vectors as the knowledge block payloads in a vector database. Subsequently, the target knowledge block vector can be accurately retrieved from the vector database based on the query information vector, improving the accuracy and efficiency of intelligent interaction on the AB experiment platform, enhancing the accuracy and user-friendliness of knowledge queries, reducing learning and maintenance costs, and allowing users to directly query relevant information through natural language dialogue. The large language model can understand and optimize based on the knowledge base content, transforming the search results into dialogue-friendly output, making information expression more concise and understandable.
[0033] In one embodiment, when the vector database needs to be initialized, the AB experiment knowledge documents in the AB experiment knowledge base can be segmented, and the vector database can be initialized based on the knowledge blocks obtained from the segmentation. When a user uploads a new AB experiment knowledge document to the AB experiment knowledge base, or updates an AB experiment knowledge document, the newly added or updated AB experiment knowledge document can be segmented, and the vector database can be updated based on the knowledge blocks obtained from the segmentation.
[0034] Accordingly, in the intelligent interaction method of the AB experimental platform provided in this application, obtaining the target knowledge block corresponding to the target knowledge block vector can be: determining the target payload corresponding to the target knowledge block in the vector database, and obtaining the target knowledge block corresponding to the target knowledge block vector in the target payload.
[0035] For example, after retrieving the target knowledge block vector from the vector database, the payload stored in the vector database for the target knowledge block vector is determined, and this payload is identified as the target payload. The knowledge block is then extracted from the target payload and identified as the target knowledge block. This application improves the accuracy and efficiency of intelligent interaction in the A / B testing platform by quickly and accurately extracting the target knowledge block from the knowledge block payload recorded in the vector database based on the target knowledge block vector.
[0036] S230: Obtain the experiment query information provided by the user front-end based on the AB experiment platform through the microservice cluster, and generate a query information vector based on the experiment query information. The user front-end submits the experiment query information through the interactive interface.
[0037] S240: Retrieve the target knowledge block vector from the vector database server based on the query information vector. The vector database server is configured with one or more vector databases. The vector databases record knowledge block vectors corresponding to multiple knowledge blocks. The knowledge blocks are obtained by segmenting based on the preset AB experiment knowledge documents.
[0038] S250: Obtain the target knowledge block corresponding to the target knowledge block vector, and generate query prompts based on the experimental query information and the target knowledge block.
[0039] In one possible embodiment, the intelligent interaction method for the A / B testing platform provided in this application, before generating query suggestions based on the experimental query information and the target knowledge block, further includes: obtaining the context information of the user's front end in the current session. Furthermore, generating query suggestions based on the experimental query information and the target knowledge block includes concatenating the experimental query information, context information, and the target knowledge block to obtain the query suggestions.
[0040] For example, the context information of the user's front end in the current session is obtained (e.g., historical experiment query information and historical query answers entered by the user in the current interactive interface). The user can have multiple sessions in the current interactive interface, with different sessions corresponding to different session pages. The user can enter experiment query information on the session page and can continue to enter experiment query information after the query answer is displayed. When subsequently entering experiment query information, the previously entered experiment query information and the generated query answer constitute the context information. Optionally, based on a preset context length threshold, the earliest context information with time information (e.g., input time, generation time) can be obtained to generate query suggestions. Optionally, the context information can be stored in a preset database (e.g., a MySQL database).
[0041] In one embodiment, the experimental query information, context information, and target knowledge block are concatenated to obtain query suggestions. Optionally, the experimental query information, context information, and target knowledge block can be concatenated based on a preset query suggestion format to obtain query suggestions. For example, if the experimental query information is A, the target knowledge blocks are B, C, and D, and the context information is E, then the generated query suggestion could be "Answer A based on knowledge B, C, and D and context E".
[0042] This application obtains query prompts by concatenating experimental query information, contextual information, and target knowledge blocks. This allows the large language model to answer experimental queries based on contextual information and target knowledge blocks. Users can ask follow-up questions based on historical queries and answers, improving the contextual relevance of the large language model's answers. This enables multi-turn dialogue for intelligent interaction on the A / B testing platform, improving the accuracy and efficiency of intelligent interaction on the A / B testing platform and enhancing the user experience.
[0043] S260: Input the query suggestions into the preset language model, and the language model will analyze and process the query suggestions to obtain the query answer.
[0044] S270: Sends the query answer to the user front-end, so that the user front-end can display the query answer on the interactive interface.
[0045] In one possible embodiment, the intelligent interaction method of the AB experimental platform provided in this application, after inputting query prompts into a preset language model and having the language model analyze and process the query prompts to obtain query answers, further includes: continuously receiving query answer fragments returned by the language model based on a streaming dialogue protocol, wherein the query answer includes multiple query answer fragments; and continuously sending query answer fragments to the user front-end based on the streaming dialogue protocol, for the user front-end to splice the received query answer fragments and display the spliced query answer fragments in the form of a streaming dialogue.
[0046] The AB experimental platform intelligent interactive device provided in this application can establish a connection with the platform where the large language model is located based on a streaming dialogue protocol (such as the streaming call protocol SSE (Server-Sent Events)). The user front end can establish a connection with the AB experimental platform intelligent interactive device based on the streaming dialogue protocol.
[0047] For example, the large language model generates query answer fragments in a "generate-as-you-go" manner and continuously sends these fragments to the intelligent interactive device of the AB experimental platform based on a streaming dialogue protocol. Upon receiving the query answer fragments, the intelligent interactive device of the AB experimental platform sends them to the user's front-end based on the streaming dialogue protocol. The user's front-end concatenates one or more continuously received query answer fragments and displays the concatenated fragments in a streaming dialogue format. After receiving and displaying multiple query answer fragments, the concatenated fragments form the complete query answer. This application effectively improves the flow of intelligent interaction on the AB experimental platform and enhances the user experience by displaying query answer fragments in real time through streaming dialogue.
[0048] The above describes a process where, based on the experimental query information provided by the user's front-end using the AB experiment platform, a query information vector is generated. This vector is then used to retrieve the target knowledge block vector from the vector database, obtaining the target knowledge block corresponding to the vector. Query prompts are generated based on the experimental query information and the target knowledge block, and then input into a pre-defined language model. The language model analyzes and processes these prompts to obtain the query answer. The vector database records knowledge block vectors corresponding to multiple knowledge blocks, which are obtained by segmenting pre-defined AB experiment knowledge documents. By combining the experimental query information and the target knowledge block to generate query prompts, the language model can accurately output query answers for AB experiment-related knowledge. This effectively solves the technical problem of users needing to consult numerous AB experiment knowledge documents to understand the platform, leading to a decreased user experience. The system allows users to accurately understand the AB experiment platform without having to consult a large number of documents, significantly improving the user experience. Meanwhile, by segmenting the AB experiment knowledge documents in the AB experiment knowledge base into blocks, multiple knowledge blocks are obtained, and the knowledge block vectors are saved as the payloads of the knowledge blocks in the vector database. Subsequently, the target knowledge block vector can be accurately retrieved from the vector database based on the query information vector, thereby improving the accuracy and efficiency of intelligent interaction on the AB experiment platform.
[0049] Based on the above embodiments, Figure 3 A flowchart of another intelligent interaction method for an AB experimental platform provided in this application embodiment is given. This intelligent interaction method for an AB experimental platform is a concretization of the above-described intelligent interaction method for an AB experimental platform. (Reference) Figure 3 The intelligent interaction method of this AB experimental platform includes: S310: Receive dialogue scene selection operation and determine the target dialogue scene corresponding to the dialogue scene selection operation.
[0050] In one embodiment, a dialogue scenario selection button can be displayed on the user's front-end interactive interface. The user clicks the dialogue scenario selection button to initiate a dialogue scenario selection operation. The dialogue scenarios provided in this application include an operation assistant scenario and an experimental analysis scenario.
[0051] For example, the system receives a dialogue scenario selection operation from the user's front end, determines the target dialogue scenario corresponding to the operation, and determines the construction method of the prompt words based on the target dialogue scenario. For instance, when the target dialogue scenario is an operation assistant scenario, the system jumps to step S320, obtains the experiment query information provided by the user's front end based on the AB experiment platform through the microservice cluster, generates a query information vector based on the experiment query information, and generates prompt words based on the operation assistant scenario; when the target dialogue scenario is an experiment analysis scenario, the system jumps to step S360 to generate prompt words based on the experiment analysis scenario.
[0052] Optionally, the dialogue scenarios provided in this application may also include AI dialogue scenarios. When the target dialogue scenario is an AI dialogue scenario, dialogue prompts can be generated based on the AI dialogue information provided by the user's front end. These prompts are then input into a preset language model, which analyzes and processes them to obtain the dialogue answer, which is then returned to the user's front end for display. This application supports multi-scenario dialogue by determining the construction method of the prompts based on the target dialogue scenario corresponding to the dialogue scenario.
[0053] S320: When the target dialogue scenario is an operation assistant scenario, the microservice cluster obtains the experiment query information provided by the user front-end based on the AB experiment platform, and generates a query information vector based on the experiment query information. The user front-end submits the experiment query information through the interactive interface.
[0054] S330: Retrieve the target knowledge block vector from the vector database server based on the query information vector. The vector database server is configured with one or more vector databases. The vector databases record knowledge block vectors corresponding to multiple knowledge blocks. The knowledge blocks are obtained by segmenting based on the preset AB experiment knowledge documents.
[0055] S340: Obtain the target knowledge block corresponding to the target knowledge block vector, and generate query prompts based on the experimental query information and the target knowledge block.
[0056] S350: Input the query suggestion words into the preset language model, which analyzes and processes the query suggestion words to obtain the query answer and sends the query answer to the user front-end for display on the user front-end interactive interface.
[0057] S360: When the target dialogue scenario is an experimental analysis scenario, obtain the experimental requirement information provided by the user's front end based on the AB experimental platform, and determine whether to call the data processing tool based on the experimental requirement information.
[0058] In one embodiment, after selecting an experimental analysis scenario and entering the corresponding session page, the user can input experimental requirement information on the session page. This requirement information can be text or voice (or corresponding text extracted based on speech recognition technology). For example, after the user's front end establishes communication with the AB experimental platform's intelligent interactive device, the corresponding interactive interface of the AB experimental platform's intelligent interactive device is displayed on the user's front end. The user can select an experimental analysis scenario on the interactive interface and enter the corresponding session page, inputting experimental requirement information on the session page. For example, inputting phrases such as "Help me query the retention rate metric of Experiment A and analyze the results," "Generate a detailed analysis report for Experiment ID-123," "Calculate the attribution of the purchase rate metric in Experiment ID-456," and "Analyze the attribution results just now" as experimental requirement information.
[0059] In one embodiment, the A / B testing platform provided in this application is configured with multiple data processing tools (Agents) that provide different data processing functions. These tools can be invoked to perform corresponding data processing functions. For example, the data processing functions that different data processing tools can perform include: generating rule-based experimental analysis reports, performing experimental indicator attribution calculations, analyzing attribution calculation results, conducting specific indicator analysis, and querying detailed experimental information. Different data processing tools can be configured according to actual needs.
[0060] For example, when the target dialogue scenario is an experimental analysis scenario, the system obtains the experimental requirement information provided by the user's front end based on the A / B testing platform, and determines whether to call a data processing tool based on the experimental requirement information. Optionally, a trained neural network model can be used to analyze and process the experimental requirement information to determine whether to call a data processing tool. For example, when the experimental requirement information is "Help me query the retention rate index of experiment A and analyze the results," it can be determined that a data processing tool with the ability to "analyze the retention rate index" needs to be called.
[0061] In one embodiment, when it is determined that data processing tools do not need to be called based on the experimental requirements information, dialogue prompts can be generated based on the experimental requirements information provided by the user's front end. The dialogue prompts are then input into a preset language model, which analyzes and processes the experimental requirements information to obtain the experimental answer and returns it to the user's front end for display.
[0062] S370: If it is determined that a data processing tool should be called, the target data processing tool is determined according to the experimental requirements information, and the target data processing tool is called to process the experimental requirements information and obtain the tool execution result.
[0063] S380: The experimental requirement information and the tool execution results are concatenated to obtain experimental prompts. The experimental prompts are then input into a preset language model, which analyzes and processes them to obtain the experimental answers.
[0064] For example, when determining which data processing tool to call, the target data processing tool is determined based on the experimental requirements information, and the target data processing tool is called to process the experimental requirements information to obtain the tool execution result.
[0065] Optionally, multiple data processing tools can be integrated into an experimental analysis toolset. Utilizing large language model function calling technology and RPC (Remote Procedure Call) cross-service calls, experimental requirement information can be analyzed. The target data processing tool can then be scheduled within the experimental analysis toolset to process the experimental requirement information and obtain the tool execution results. By establishing a connection channel with the platform's underlying business rules through function calling technology and RPC cross-service calls, the experimental analysis toolset can be invoked across processes via RPC within the experimental analysis toolset logic implementation. This allows for real-time acquisition of the latest data from the AB experimental platform (such as experimental configurations, result data, business rules, etc.) and integration with platform control operation interfaces. This enables automatic API calls to obtain experimental metrics and automatically generates statistically sound analysis reports and optimization suggestions, significantly improving the efficiency and accuracy of experimental analysis.
[0066] After obtaining the tool execution result, the experimental requirement information and the tool execution result are concatenated to obtain experimental prompts. These prompts can be generated by concatenating experimental query information and target knowledge blocks based on a preset prompt format. For example, if the experimental query information is S and the tool execution result is M, the generated prompt could be "Answer S based on tool execution result M".
[0067] In one embodiment, the generated experimental prompts are input into a preset language model, which analyzes and processes the prompts to obtain the experimental answers. After obtaining the answers, they can be sent to the user's front-end, where they can be displayed on the interactive interface. Users can also input new experimental requirements on the interactive interface for multi-turn dialogue within the A / B testing platform's intelligent interaction.
[0068] This application, when the target dialogue scenario is an experimental analysis scenario, obtains the experimental requirement information provided by the user's front end based on the A / B testing platform, and calls the target data processing tool to process the experimental requirement information to obtain the tool execution result. The experimental requirement information and the tool execution result are concatenated to obtain experimental prompt words. The experimental prompt words are input into a preset language model, which analyzes and processes the experimental prompt words to obtain the experimental answer. No manual intermediate data preprocessing is required. The large language model, retrieval enhancement generation, and data processing tools are deeply integrated into the core interaction layer of the A / B testing platform. The language model can accurately provide experimental answers based on the experimental requirement information and the tool execution result, which greatly improves the efficiency of A / B testing and realizes the fully automated generation from experimental requirement information to experimental answer. The analysis can be completed without manual query writing or reliance on personal knowledge background. The consistency and interpretability of the analysis results are significantly improved, enabling users without data analysis background to quickly obtain experimental answers as an actionable decision-making basis.
[0069] The above describes a process where, based on the experimental query information provided by the user's front-end using the AB experiment platform, a query information vector is generated. This vector is then used to retrieve the target knowledge block vector from the vector database, obtaining the target knowledge block corresponding to the vector. Query prompts are generated based on the experimental query information and the target knowledge block, and then input into a pre-defined language model. The language model analyzes and processes these prompts to obtain the query answer. The vector database records knowledge block vectors corresponding to multiple knowledge blocks, which are obtained by segmenting pre-defined AB experiment knowledge documents. By combining the experimental query information and the target knowledge block to generate query prompts, the language model can accurately output query answers for AB experiment-related knowledge. This effectively solves the technical problem of users needing to consult numerous AB experiment knowledge documents to understand the platform, leading to a decreased user experience. The system allows users to accurately understand the AB experiment platform without having to consult a large number of documents, significantly improving the user experience. When the target dialogue scenario is an experimental analysis scenario, the system obtains the experimental requirement information provided by the user's front end based on the A / B testing platform. It then calls the target data processing tool to process the experimental requirement information and obtain the tool execution results. The experimental requirement information and the tool execution results are concatenated to obtain experimental prompts. These prompts are input into a pre-defined language model, which analyzes and processes them to obtain the experimental answers. No manual data preprocessing is required; the language model can accurately provide experimental answers based on the experimental requirement information and tool execution results, greatly improving the efficiency of A / B testing. Figure 4 A schematic diagram of the structure of an intelligent interactive device for an AB experimental platform provided in an embodiment of this application is given. (Reference) Figure 4 The intelligent interactive device of the AB experimental platform includes a vector generation module 41, a vector retrieval module 42, a prompt word generation module 43, a query processing module 44, and a result feedback module 45.
[0070] The system includes the following modules: Vector Generation Module 41, configured to obtain experimental query information provided by the user frontend based on the AB experiment platform through a microservice cluster, and generate query information vectors based on the experimental query information, wherein the user frontend submits experimental query information through an interactive interface; Vector Retrieval Module 42, configured to retrieve target knowledge block vectors from a vector database server based on the query information vectors, wherein the vector database server is configured with one or more vector databases, and the vector databases record knowledge block vectors corresponding to multiple knowledge blocks, which are obtained by segmenting based on a preset AB experiment knowledge document; Prompt Word Generation Module 43, configured to obtain the target knowledge block corresponding to the target knowledge block vector, and generate query prompt words based on the experimental query information and the target knowledge block; Query Processing Module 44, configured to input the query prompt words into a preset language model, and have the language model analyze and process the query prompt words to obtain the query answer; and Result Feedback Module 45, configured to send the query answer to the user frontend for the user frontend to display the query answer on the interactive interface.
[0071] The above describes a process where, based on the experimental query information provided by the user's front-end using the AB experiment platform, a query information vector is generated. This vector is then used to retrieve the target knowledge block vector from the vector database, obtaining the target knowledge block corresponding to the vector. Query prompts are generated based on the experimental query information and the target knowledge block, and then input into a pre-defined language model. The language model analyzes and processes these prompts to obtain the query answer. The vector database records knowledge block vectors corresponding to multiple knowledge blocks, which are obtained by segmenting pre-defined AB experiment knowledge documents. By combining the experimental query information and the target knowledge block to generate query prompts, the language model can accurately output query answers for AB experiment-related knowledge. This effectively solves the technical problem of users needing to consult numerous AB experiment knowledge documents to understand the platform, leading to a decreased user experience. The system allows users to accurately understand the AB experiment platform without having to consult a large number of documents, significantly improving the user experience.
[0072] In one possible embodiment, the AB experimental platform intelligent interactive device further includes a database management module, which is configured as follows: The AB experiment knowledge documents in the AB experiment knowledge base are divided into blocks to obtain multiple knowledge blocks. Determine the knowledge block vector for each knowledge block, and save the knowledge block vector as the payload of the knowledge block to the vector database.
[0073] In one possible embodiment, the prompt word generation module 43 obtains the target knowledge block corresponding to the target knowledge block vector, configured as follows: Determine the target payload corresponding to the target knowledge block in the vector database, and retrieve the target knowledge block corresponding to the target knowledge block vector from the target payload.
[0074] In one possible embodiment, the intelligent interactive device of the AB experimental platform further includes an information acquisition module, which is configured as follows: Obtain the context information of the user's front end in the current session; Furthermore, the prompt word generation module 43 generates query prompt words based on the experimental query information and the target knowledge block, configured as follows: The query suggestions are obtained by concatenating the experimental query information, context information, and target knowledge blocks.
[0075] In one possible embodiment, the AB experiment platform intelligent interactive device further includes an answer return module, which is configured as follows: Based on the streaming dialogue protocol, query answer fragments returned by the large language model are continuously received. The query answer includes multiple query answer fragments. Based on a streaming dialogue protocol, query answer fragments are continuously sent to the user's front end. The user's front end then splices the received query answer fragments and displays the spliced query answer fragments in the form of a streaming dialogue.
[0076] In one possible embodiment, the intelligent interactive device of the AB experimental platform further includes a scene determination module, which is configured as follows: Receive a dialogue scenario selection operation and determine the target dialogue scenario corresponding to the dialogue scenario selection operation. Furthermore, the vector generation module 41 obtains the experiment query information provided by the user's front end based on the AB experiment platform and configures it as follows: When the target dialogue scenario is an operation assistant scenario, obtain the experiment query information provided by the user's front end based on the A / B experiment platform.
[0077] In one possible embodiment, the intelligent interactive device of the AB experimental platform further includes an analysis and interaction module, which is configured as follows: When the target dialogue scenario is an experimental analysis scenario, obtain the experimental requirement information provided by the user's front end based on the A / B experimentation platform. Determine whether to call data processing tools based on the experimental requirements; If it is determined that a data processing tool should be called, the target data processing tool is selected based on the experimental requirements information, and the target data processing tool is called to process the experimental requirements information and obtain the tool execution results. The experimental requirements information and tool execution results are concatenated to obtain experimental prompts. These prompts are then input into a pre-defined language model, which analyzes and processes them to obtain the experimental answers.
[0078] It is worth noting that in the embodiments of the above-mentioned AB experimental platform intelligent interactive device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this application.
[0079] This application also provides an intelligent interactive device for an AB experimental platform, which can integrate the intelligent interactive device for an AB experimental platform provided in this application. Figure 5 This is a schematic diagram of the structure of an intelligent interactive device for an AB experimental platform provided in an embodiment of this application.
[0080] refer to Figure 5 The AB experimental platform intelligent interaction device includes: an input device 53, an output device 54, a memory 52, and one or more processors 51; the memory 52 is used to store one or more programs; when one or more programs are executed by one or more processors 51, the one or more processors 51 implement the AB experimental platform intelligent interaction method provided in the above embodiments. The AB experimental platform intelligent interaction device, equipment, and computer provided above can be used to execute the AB experimental platform intelligent interaction method provided in any of the above embodiments, and have corresponding functions and beneficial effects.
[0081] This application also provides a non-volatile storage medium for storing computer-executable instructions. These computer-executable instructions, when executed by a computer processor, are used to perform the AB experimental platform intelligent interaction method provided in the above embodiments. Of course, the computer-executable instructions provided in this application are not limited to the AB experimental platform intelligent interaction method provided above; they can also perform related operations in the AB experimental platform intelligent interaction method provided in any embodiment of this application. The AB experimental platform intelligent interaction device, equipment, and storage medium provided in the above embodiments can execute the AB experimental platform intelligent interaction method provided in any embodiment of this application. Technical details not described in detail in the above embodiments can be found in the AB experimental platform intelligent interaction method provided in any embodiment of this application.
[0082] Based on the above embodiments, this application also provides a computer program product. The technical solution of this application, in essence or in other words, the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes several instructions to cause a computer device, mobile terminal, or processor therein to execute all or part of the steps of the AB experimental platform intelligent interaction method provided in the various embodiments of this application.
Claims
1. An intelligent interaction method for an A / B testing platform, characterized in that, include: The experiment query information provided by the user front-end based on the A / B experiment platform is obtained through a microservice cluster, and a query information vector is generated based on the experiment query information. The user front-end submits the experiment query information through an interactive interface. The target knowledge block vector is retrieved from the vector database server based on the query information vector. The vector database server is configured with one or more vector databases. The vector database records knowledge block vectors corresponding to multiple knowledge blocks. The knowledge blocks are obtained by segmenting based on a preset AB experiment knowledge document. Obtain the target knowledge block corresponding to the target knowledge block vector, and generate query prompts based on the experimental query information and the target knowledge block; The query suggestions are input into a preset language model, which analyzes and processes the query suggestions to obtain the query answer. The query answer is sent to the user front-end, so that the user front-end can display the query answer on the interactive interface.
2. The intelligent interaction method for the AB experimental platform according to claim 1, characterized in that, Before obtaining the experiment query information provided by the user front-end based on the AB experiment platform, the following is also included: The AB experiment knowledge documents in the AB experiment knowledge base are divided into blocks to obtain multiple knowledge blocks. Determine the knowledge block vector for each knowledge block, and save the knowledge block vector as the payload of the knowledge block to the vector database.
3. The intelligent interaction method for the AB experimental platform according to claim 2, characterized in that, The step of obtaining the target knowledge block corresponding to the target knowledge block vector includes: The target payload corresponding to the target knowledge block is determined in the vector database, and the target knowledge block corresponding to the target knowledge block vector is obtained from the target payload.
4. The intelligent interaction method for the AB experimental platform according to claim 1, characterized in that, Before generating query suggestions based on the experimental query information and the target knowledge block, the method further includes obtaining the context information of the user front-end in the current session; Furthermore, the step of generating query prompts based on the experimental query information and the target knowledge block includes concatenating the experimental query information, the context information, and the target knowledge block to obtain query prompts.
5. The intelligent interaction method for the AB experimental platform according to claim 1, characterized in that, After inputting the query suggestion words into a preset language model, and having the language model analyze and process the query suggestion words to obtain the query answer, the method further includes: Based on the streaming dialogue protocol, the system continuously receives query answer fragments returned by the language big model, and the query answer includes multiple query answer fragments; Based on the streaming dialogue protocol, the query answer fragments are continuously sent to the user front-end, so that the user front-end can splice the received query answer fragments and display the spliced query answer fragments in the form of streaming dialogue.
6. The intelligent interaction method for the AB experimental platform according to claim 1, characterized in that, Before obtaining the experiment query information provided by the user front-end based on the AB experiment platform, the method also includes receiving a dialogue scenario selection operation and determining the target dialogue scenario corresponding to the dialogue scenario selection operation. Furthermore, the acquisition of experimental query information provided by the user front-end based on the AB experiment platform includes, when the target dialogue scenario is an operation assistant scenario, acquiring experimental query information provided by the user front-end based on the AB experiment platform.
7. The intelligent interaction method for the AB experimental platform according to claim 6, characterized in that, After determining the target dialogue scenario corresponding to the dialogue scenario selection operation, the method further includes: When the target dialogue scenario is an experimental analysis scenario, obtain the experimental requirement information provided by the user's front end based on the A / B experiment platform; Determine whether to invoke the data processing tool based on the experimental requirements information; If it is determined that a data processing tool should be called, the target data processing tool is determined according to the experimental requirements information, and the target data processing tool is called to process the experimental requirements information to obtain the tool execution result; The experimental requirements information and the tool execution results are concatenated to obtain experimental prompts. The experimental prompts are then input into a preset language model, which analyzes and processes them to obtain the experimental answers.
8. An intelligent interactive device for an AB experimental platform, characterized in that, It includes a vector generation module, a vector retrieval module, a prompt word generation module, a query processing module, and a result feedback module, among which: The vector generation module is configured to obtain experimental query information provided by the user front-end based on the AB experiment platform through a microservice cluster, and generate a query information vector based on the experimental query information, wherein the user front-end submits the experimental query information through an interactive interface; The vector retrieval module is configured to retrieve target knowledge block vectors from a vector database server based on the query information vector. The vector database server is configured with one or more vector databases, and the vector databases record knowledge block vectors corresponding to multiple knowledge blocks. The knowledge blocks are obtained by segmenting a preset AB experiment knowledge document. The prompt word generation module is configured to obtain the target knowledge block corresponding to the target knowledge block vector, and generate query prompt words based on the experimental query information and the target knowledge block. The query processing module is configured to input the query suggestion words into a preset language model, and the language model analyzes and processes the query suggestion words to obtain the query answer. The result feedback module is configured to send the query answer to the user front-end, so that the user front-end can display the query answer on the interactive interface.
9. An intelligent interactive device for an AB experimental platform, characterized in that, include: Memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the AB experimental platform intelligent interaction method as described in any one of claims 1-7.
10. A non-volatile storage medium for storing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the AB experimental platform intelligent interaction method as described in any one of claims 1-7.
11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the AB experimental platform intelligent interaction method according to any one of claims 1-7.