A / B test data analysis method and device based on large model agent

Abstract

The invention discloses an A / B test data analysis method and device based on a large model agent, and the method comprises the steps: displaying an intelligent interaction page associated with the agent, and the agent is used for carrying out the joint analysis of different data indexes through a large model; in response to an input operation on the at least two data indexes on the intelligent interaction page, determining the at least two data indexes as target data indexes to be subjected to conjoint analysis; and in response to a conjoint analysis demand triggered on the intelligent interaction page and aiming at the target data index, displaying a data analysis result aiming at the conjoint analysis demand. Therefore, automatic analysis of the data can be realized through relatively simple input operation, so that the data analysis process is simplified, and the analysis efficiency is improved. Besides, input of the data indexes and conjoint analysis of different data indexes are supported, so that targeted analysis can be performed on different data indexes in a user-defined manner according to user requirements, and the flexibility of data analysis and the user experience are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular, to a method and apparatus for analyzing A / B test data based on a large model agent. Background Art

[0002] With the advent of the big data era, the data in business scenarios presents diversity and complexity. In order to extract meaningful information from the data, data analysis is generally required. For example, in the content delivery scenario, an A / B experiment can be set up, and then by comparing and analyzing the experimental data of the A / B experiment, a content delivery strategy for content delivery can be obtained.

[0003] However, in the related art, when analyzing the data in business scenarios, generally a single indicator group is used as the analysis dimension, and thus a holistic drill-down analysis is performed on all the indicators in the indicator group, which has certain limitations. Summary of the Invention

[0004] This Summary of the Invention section is provided to introduce concepts in a brief form, which will be described in detail in the subsequent Detailed Description section. This Summary of the Invention section is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to be used to limit the scope of the claimed technical solution.

[0005] In a first aspect, the present disclosure provides a method for analyzing A / B test data based on a large model agent. The method for analyzing A / B test data based on a large model agent includes: Displaying an intelligent interaction page associated with an agent, where the agent is used to jointly analyze different data indicators through a large model; In response to an input operation on at least two data indicators on the intelligent interaction page, determining the at least two data indicators as target data indicators to be jointly analyzed; In response to a joint analysis requirement for the target data indicators triggered on the intelligent interaction page, displaying an analysis result of the data for the joint analysis requirement on the intelligent interaction page.

[0006] In a second aspect, the present disclosure provides a data analysis apparatus. The data analysis apparatus includes: A first display module for displaying an intelligent interaction page associated with an agent, where the agent is used to jointly analyze different data indicators through a large model; A determination module for, in response to an input operation on at least two data indicators on the intelligent interaction page, determining the at least two data indicators as target data indicators to be jointly analyzed; A second display module, configured to respond to a joint analysis requirement for the target data metric triggered on the intelligent interaction page, and display, on the intelligent interaction page, a data analysis result for the joint analysis requirement.

[0007] In a third aspect, the present disclosure provides a computer-readable medium, on which a computer program is stored, and when the computer program is executed by a processing device, the steps of the method described in the first aspect are implemented.

[0008] In a fourth aspect, the present disclosure provides an electronic device, including: A storage device, on which a computer program is stored; A processing device, configured to execute the computer program in the storage device to implement the steps of the method described in the first aspect.

[0009] In a fifth aspect, the present disclosure provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] Through the above technical solutions, the target data metrics to be jointly analyzed and the joint analysis requirements for the target data metrics can be input on the intelligent interaction page, and the joint analysis requirements for the target data metrics can be automatically analyzed by the large model to obtain a data analysis result for the joint analysis requirement. Thus, automatic analysis of data can be achieved through relatively simple input operations, thereby simplifying the data analysis process and improving the analysis efficiency. On the other hand, since this solution supports the input of data metrics and can perform joint analysis on different data metrics, thus, targeted analysis of different data metrics can be customized according to user requirements, thereby improving the flexibility of data analysis and the user experience.

[0011] Other features and advantages of the present disclosure will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In combination with the drawings and with reference to the following specific implementation manners, the above and other features, advantages, and aspects of the embodiments of the present disclosure will become more obvious. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the original components and elements are not necessarily drawn to scale. In the drawings: Figure 1 is a flowchart of a method for A / B test data analysis based on a large model agent according to an exemplary embodiment of the present disclosure; Figure 2 is a schematic diagram of the input of a data metric according to an exemplary embodiment of the present disclosure; Figure 3It is a display schematic diagram of an analysis task script shown according to an exemplary embodiment of the present disclosure; Figure 4 It is a content reference schematic diagram shown according to an exemplary embodiment of the present disclosure; Figure 5 It is a display schematic diagram of an interaction link map shown according to an exemplary embodiment of the present disclosure; Figure 6 It is a schematic diagram of style switching of an intelligent interaction page shown according to an exemplary embodiment of the present disclosure; Figure 7 It is a structural block diagram of an A / B test data analysis device based on a large model agent shown according to an exemplary embodiment of the present disclosure; Figure 8 It is a structural schematic diagram of an electronic device shown according to an exemplary embodiment of the present disclosure. Detailed implementation manners

[0013] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0014] It should be understood that the various steps recited in the method embodiments of the present disclosure can be executed in a different order and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard.

[0015] The term "including" and its variations used herein are open-ended, that is, "including but not limited to". The term "based on" is "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0016] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependent relationships.

[0017] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly stated in the context, it should be understood as "one or more".

[0018] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are for illustrative purposes only and are not used to limit the scope of these messages or information.

[0019] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to users and the authorization of users should be obtained through appropriate means in accordance with relevant laws and regulations.

[0020] For example, when responding to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested by the user will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, an application program, a server, or a storage medium that performs the operations of the technical solutions of the present disclosure according to the prompt message.

[0021] As an optional but non-limiting implementation manner, the manner of sending a prompt message to the user in response to receiving an active request from the user may be, for example, a pop-up window manner, and the prompt message may be presented in text in the pop-up window. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0022] It can be understood that the above process of notifying and obtaining user authorization is only illustrative and does not constitute a limitation on the implementation manners of the present disclosure, and other manners that meet relevant laws and regulations can also be applied to the implementation manners of the present disclosure.

[0023] At the same time, it can be understood that the data involved in the technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the corresponding laws, regulations and related provisions.

[0024] As mentioned in the background art, in the related art, when performing data analysis on the data in a business scenario, generally a single index group is used as the analysis dimension, so as to perform an overall analysis on all the indexes in the index group, which has certain limitations.

[0025] Exemplarily, in a content delivery scenario, multiple different content delivery strategies may be provided. For example, content delivery strategy A and content delivery strategy B may be provided. In order to determine the optimal content delivery strategy from content delivery strategy A and content delivery strategy B, an A / B test may be set up, that is: for the target content to be delivered, let a part of the target content to be delivered be delivered based on content delivery strategy A, and let another part of the target content to be delivered be delivered based on content delivery strategy B, and finally determine the optimal content delivery strategy through comparison of experimental data (i.e., delivery data).

[0026] However, in related technologies, many data analysis tools are built based on the concept of a single metric group, that is, they can only perform overall dimension drilling or overall analysis on the entire metric group. However, in actual business scenarios, users' data analysis needs are often more complex and diverse, and they may need to analyze by combining data metrics from different metric groups. For example, since different businesses may have different key metrics and influencing factors, and the launch of a new business may affect other businesses, users need to pay attention to specific metrics in different metric groups of different businesses to comprehensively evaluate the operation status and potential impact of the business.

[0027] Therefore, in order to achieve cross-metric-group data analysis, in related technologies, generally, an artificial method is used to view and extract data metrics from different metric groups, and then artificial analysis is performed based on the extracted data metrics. When performing data analysis of metric groups based on this method, users need to switch between different data systems or data analysis tools and manually organize and compare the data in each metric group, resulting in problems such as complex data analysis operations and low analysis efficiency.

[0028] In view of this, the present disclosure provides an A / B test data analysis method and device based on a large model agent to solve the above technical problems.

[0029] The following further explains and illustrates the embodiments of the present disclosure with reference to the accompanying drawings.

[0030] Figure 1 is a flowchart of an A / B test data analysis method based on a large model agent shown according to an exemplary embodiment of the present disclosure. Referring to Figure 1 , the A / B test data analysis method based on a large model agent may include the following steps: S101: Display an intelligent interaction page associated with the agent, where the agent is used to perform joint analysis on different data metrics through a large model.

[0031] Exemplarily, when opening the data analysis platform, the home page of the data analysis platform may be displayed, and an analysis module "Data Analysis" for data analysis may be displayed on the home page of the data analysis platform. When the analysis module "Data Analysis" is triggered through operations such as clicking, an intelligent interaction page associated with the agent may be displayed, such as Figure 2As shown, where the intelligent agent is the XX intelligent assistant in the figure. Or, when triggering the "data analysis" of the analysis module through operations such as clicking, the data analysis page can be displayed, and the identifier of the intelligent agent can be displayed on the data analysis page. When triggering the identifier of the intelligent agent through operations such as clicking, the intelligent interaction page associated with the intelligent agent is displayed. Or, when triggering the "data analysis" of the analysis module through operations such as clicking, the data analysis page can be displayed, and the data index information and the function entry for triggering the large model to analyze the data index information can be displayed on the data analysis page. When triggering the function entry through operations such as clicking, the intelligent interaction page associated with the intelligent agent is displayed. Of course, it can also be to trigger the display of the intelligent interaction page associated with the intelligent agent in other ways, and the embodiments of the present disclosure do not make any restrictions on this.

[0032] In this embodiment, the joint analysis of different data indicators can be the joint analysis of at least two data indicators in the same data indicator group, or the joint analysis of at least two data indicators in different data indicator groups. Of course, it can also be others, and the embodiments of the present disclosure do not make any restrictions on this.

[0033] S102: In response to an input operation on at least two data indicators on the intelligent interaction page, determine the at least two data indicators as the target data indicators to be jointly analyzed.

[0034] In this embodiment, the input operation on at least two data indicators can be that the user manually inputs the data indicators through an external input device. For example, the data indicators can be input through a keyboard to obtain the target data indicators. It can also be that a selection control for selecting data indicators is pre-displayed on the intelligent interaction page. When the selection control is triggered through operations such as clicking, the index page can be displayed, and different data indicators can be displayed on the index page. When the index page is triggered through operations such as clicking, the target data indicators are obtained. Of course, it can also be to input the data indicators in other ways, and the embodiments of the present disclosure do not make any restrictions on this.

[0035] S103: In response to the joint analysis requirement for the target data indicators triggered on the intelligent interaction page, display the data analysis result for the joint analysis requirement on the intelligent interaction page.

[0036] In this embodiment, the joint analysis requirement for the target data metric triggered on the intelligent interaction page can be triggered through a session, or after determining the target data metric, a preset joint analysis requirement for the target data metric can be displayed on the intelligent interaction page. When the preset joint analysis requirement is triggered through operations such as clicking or dragging, the joint analysis requirement is triggered on the intelligent interaction page. Of course, it can also be triggered in other ways on the intelligent interaction page, and the embodiments of the present disclosure do not impose any restrictions on this. When the joint analysis requirement is triggered through a session, it can be triggered by voice input of the joint analysis requirement, or by inputting the joint analysis requirement in the form of natural language text in the text input box. Of course, it can also be triggered through other session methods, and the embodiments of the present disclosure do not impose any restrictions on this.

[0037] Through the above technical solution, the target data metrics to be jointly analyzed and the joint analysis requirements for the target data metrics can be input on the intelligent interaction page, and the large model can automatically analyze the joint analysis requirements for the target data metrics to obtain the data analysis results for the joint analysis requirements. Thus, through relatively simple input operations, automatic data analysis can be achieved, thereby simplifying the data analysis process and improving the analysis efficiency. On the other hand, since this solution can support the input of data metrics and can perform joint analysis on different data metrics, thus, according to user needs, different data metrics can be customarily and specifically analyzed, thereby improving the flexibility of data analysis and the user experience.

[0038] To facilitate understanding of the A / B test data analysis method based on the large model agent provided by the present disclosure, the possible implementation manners in the present disclosure are described below.

[0039] In a possible manner, in response to an input operation on at least two data metrics on the intelligent interaction page, determining the at least two data metrics as the target data metrics to be jointly analyzed may include: In response to a shortcut instruction triggered on the intelligent interaction page, different data metrics are displayed; in response to a selection operation on at least two data metrics among the different data metrics, the at least two data metrics corresponding to the selection operation are determined as the target data metrics to be jointly analyzed.

[0040] In this embodiment, the shortcut instruction can be determined according to the actual situation, and the embodiments of the present disclosure do not impose any restrictions thereon. By way of example, the shortcut instruction can be a preset character. For example, it can be the character "#". Thus, when the character "#" is input on the intelligent interaction page, different data metrics can be displayed. By way of example, the shortcut instruction can be a preset voice instruction. For example, it can be "Select data metrics". Thus, when the intelligent interaction page detects the language instruction "Select data metrics", different data metrics can be displayed. By way of example, the shortcut instruction can be a shortcut key. For example, it can be the shortcut key "Ctrl+Q". Thus, when the intelligent interaction page detects that an external input device triggers the shortcut key "Ctrl+Q", different data metrics can be displayed.

[0041] In this embodiment, different data metrics can be determined according to the actual situation, and the embodiments of the present disclosure do not impose any restrictions thereon. By way of example, different data metrics can be different data metrics in the same data metric group, or different data metrics in different data metric groups. Of course, they can also be others, and the embodiments of the present disclosure do not impose any restrictions thereon.

[0042] By way of example, the shortcut instruction can be the character "#". The intelligent interaction page can include a content display area and a content input area. Among them, a send control "Send" for sending the content in the content input area to the content display area is displayed in the content input area. Thus, when the character "#" is input in the content input area, a drop-down panel can be displayed, and multiple different data metric groups can be displayed in the drop-down panel. When "Data Metric Group 2" in the multiple data metric groups is triggered by an operation such as clicking, multiple individual data metrics corresponding to "Data Metric Group 2" can be displayed. When "Data Metric 2" and "Data Metric 3" are triggered by an operation such as clicking, "Data Metric 2" and "Data Metric 3" can be determined as the target data metrics and displayed in the content input area, as Figure 2 shown.

[0043] It should be understood that in a business scenario, there are dozens or even hundreds of data metrics. If users manually input data metrics, there may be a situation where users cannot accurately remember the data metrics due to the large number of data metrics, and then they need to repeatedly check and re-enter the data metrics, which increases the complexity of user operations. In this embodiment, since different data metrics can be displayed when a quick command is triggered, corresponding data metrics can be selected for joint analysis according to user needs, thus reducing the problem of repeated input due to manual input errors, and further improving the analysis efficiency and user experience. In addition, since the same data metric can exist in different data metric groups, in order to enable users to quickly and accurately find the required data metrics and reduce repeated searches and confusion, when selecting data metrics through a quick command, after triggering the quick command, the data metric group can be displayed first, and then after triggering the data metric group, the data metrics under the data metric group can be displayed, which can improve the efficiency and accuracy of data metric selection, reduce the complexity of user operations, and further improve the user experience.

[0044] In a possible way, the joint analysis requirement may include multiple sub-joint analysis requirements with logical relationships. Correspondingly, in response to the joint analysis requirement for the target data metric triggered on the intelligent interaction page, displaying the data analysis result for the joint analysis requirement on the intelligent interaction page may include: In response to the joint analysis requirement for the target data metric triggered on the intelligent interaction page, displaying the analysis task script for implementing the joint analysis requirement on the intelligent interaction page; in response to the trigger operation on the analysis task script, displaying the data analysis result based on the analysis task script on the intelligent interaction page.

[0045] In this embodiment, the trigger operation on the analysis task script may be: associating a target control for confirming or executing the analysis task script with the analysis task script. Thus, after displaying the analysis task script and the target control, the trigger operation on the analysis task script can be realized by triggering the target control. It may also be associating a voice command for confirming or executing the analysis task script with the analysis task script. Thus, after displaying the analysis task script, the analysis task script can be triggered by the voice command. For example, when the intelligent interaction page detects the voice command "confirm and execute the analysis task script", the trigger on the analysis task script is realized. Of course, the trigger operation on the analysis task script may also be other, and the embodiments of the present disclosure do not make any restrictions on this.

[0046] Exemplarily, continuing to refer to the above example, after obtaining the target data metrics, the combined analysis requirements for the target data metrics can be input in the content input area of the intelligent interaction page. For example, the combined analysis requirement "I want to see the performance of these metrics under Dimension 1, Dimension 2, and... " can be input, and by triggering the "Send" control, "I want to see the performance of these metrics under Dimension 1, Dimension 2, and... " can be sent to the content display area. Subsequently, the large model can automatically decompose "I want to see the performance of these metrics under Dimension 1, Dimension 2, and... " into an analysis task script for realizing "I want to see the performance of these metrics under Dimension 1, Dimension 2, and... " according to the analysis granularity in the combined analysis requirement, and display the analysis task script and the target control "Confirm and Start Execution" for executing the analysis task script in the content display area of the intelligent interaction page. When "Confirm and Start Execution" is triggered through operations such as clicking, the data analysis result obtained based on the analysis task script can be displayed in the content display area of the intelligent interaction page, such as Figure 3 as shown.

[0047] In this embodiment, since the combined analysis requirement may include multiple sub-combined analysis requirements with logical relationships, thus, the data analysis result can include an overall data analysis result, or a data analysis result can be generated for each sub-combined analysis requirement. The embodiments of the present disclosure do not impose any restrictions on this. When a data analysis result is generated for each sub-combined analysis requirement, each data analysis result can be displayed in a card style to facilitate the user to switch and view the data analysis results corresponding to each sub-combined analysis requirement.

[0048] Through the above method, the deep understanding ability of the large model for natural language can be utilized to automatically decompose the combined analysis requirement including multiple sub-combined analysis requirements into an analysis task script for realizing the combined analysis requirement. Thus, by executing the analysis task script once, the data analysis results of multiple sub-combined analysis requirements can be obtained. Compared with the related art, when multiple combined analysis requirements with logical relationships need to be executed, the combined analysis requirements need to be sequentially analyzed based on the logical order. This solution can simplify the analysis operation process and further improve the analysis efficiency.

[0049] In a possible way, in order to enable the user to clarify the analysis process, when the user triggers the combined analysis requirement on the intelligent interaction page, the large model can recognize the intention of the combined analysis requirement, and then automatically call the corresponding intelligent analysis tool for data analysis based on the recognized intention, and during the data analysis process, display the name of the intelligent analysis tool currently performing the data analysis and / or the analysis process, thereby enhancing the credibility of the data analysis result.

[0050] In possible ways, in response to a triggering operation on an analysis task script, data analysis results obtained based on the analysis task script are displayed on the intelligent interaction page, which may include: In response to an editing operation on the analysis task script, a target analysis task script is determined according to the editing operation; in response to a triggering operation on the target analysis task script, data analysis results obtained from the target analysis task script are displayed on the intelligent interaction page.

[0051] In this embodiment, the editing operation on the analysis task script can be determined according to the actual situation, and the embodiments of the present disclosure do not impose any restrictions on this. Exemplarily, as Figure 3 shown, the editing operation on the analysis task script can be: adding, modifying, or deleting execution conditions, adding, modifying, or deleting analysis content, and adding, modifying, or deleting analysis metrics, etc.

[0052] Through the above method, the analysis task script generated by the large model can be edited, so that users can make targeted modifications to the analysis task script according to the actual situation, thereby realizing personalized data analysis and further improving the user experience.

[0053] In possible ways, the A / B test data analysis method based on a large model agent may further include: A content reference control is displayed on the intelligent interaction page, where the content reference control is used to reference the historical session content displayed on the intelligent interaction page; Correspondingly, in response to a joint analysis requirement for a target data metric triggered on the intelligent interaction page, data analysis results for the joint analysis requirement are displayed on the intelligent interaction page, which may include: In response to a triggering operation on the content reference control, target historical session content to be referenced is determined in the historical session content; in response to a joint analysis requirement for a target data metric triggered on the intelligent interaction page, data analysis results for the joint analysis requirement are displayed on the intelligent interaction page based on the target historical session content.

[0054] In this embodiment, displaying the content reference control on the intelligent interaction page can be: associating a content reference control with each piece of historical session content displayed on the intelligent interaction page, or displaying a content reference control on the intelligent interaction page. Of course, it can also be others, and the embodiments of the present disclosure do not impose any restrictions on this.

[0055] When a content reference control is associated with each piece of historical session content, determining the target historical session content to be referenced in the historical session content in response to a triggering operation by the user on the content reference control can be: determining the historical session content associated with the triggered content reference control as the target historical session content.

[0056] When a content reference control is displayed on the intelligent interaction page, in response to a trigger operation by the user on the content reference control, the target historical session content to be referenced determined from the historical session content may be: after triggering the content reference control, a selection control for selecting historical session content may be displayed. After selecting historical session content through the selection control, the selected historical session content is determined as the target historical session content. Or, it may be to automatically reference the previous historical session content. Of course, it may also be others, and the embodiments of the present disclosure do not impose any restrictions on this.

[0057] Exemplarily, as Figure 4 shown, a content reference control "Reference" may be associated and displayed for each historical session content in the content display area. When the reference control "Reference" associated with the historical session content "Under 'Axx...', data metric 1 and not..." is triggered by an operation such as clicking, "Under 'Axx...', data metric 1 and not..." can be marked and displayed in the content input area of the intelligent interaction page, so that the user can intuitively understand the referenced historical session content based on the content marked and displayed in the content input area. Subsequently, when the user triggers a joint analysis requirement on the intelligent interaction page, the large model can generate a data analysis result for the joint analysis requirement based on the target historical session content and display it in the content display area of the intelligent interaction page.

[0058] Through the above method, a content reference control can be displayed on the intelligent interaction page. Thus, when performing data analysis, data analysis can be combined with the referenced content and the joint analysis requirement, thereby further improving the accuracy of data analysis. Especially when there are different joint data analysis results on the intelligent interaction page, through content reference, the current data analysis can pay more attention to the referenced joint data analysis results, thereby further improving the accuracy of data analysis.

[0059] In a possible way, the A / B test data analysis method based on the large model agent may further include: Displaying an interaction link graph on the intelligent interaction page, where the interaction link graph is determined based on the reference relationship between different session contents on the intelligent interaction page; in response to a trigger operation on the target session content in the interaction link graph, the target session content is located and displayed on the intelligent interaction page.

[0060] In this embodiment, the interactive link graph refers to a visual structure constructed based on the reference relationships between conversation contents to reflect the reference paths between conversation contents. By way of example, if there are 6 conversation contents displayed on the intelligent interaction page, denoted as conversation content 1, conversation content 2, conversation content 3, conversation content 4, conversation content 5, and conversation content 6 respectively, where conversation content 5 references conversation content 4, and conversation content 4 references conversation content 2; conversation content 3 references conversation content 2, and conversation content 2 references conversation content 1; conversation content 6 does not have content references. Then the intelligent interaction page can display three interactive link graphs, denoted as interactive link graph 1, interactive link graph 2, and interactive link graph 3, where interactive link graph 1 can be represented as: conversation content 2 → conversation content 4 → conversation content 5, interactive link graph 2 can be represented as: conversation content 1 → conversation content 2 → conversation content 3, and interactive link graph 3 can be represented as: conversation content 6.

[0061] In this embodiment, the display of the interactive link graph on the intelligent interaction page can be: only display the name of the interactive link graph, and when the name of the interactive link graph is triggered, display the conversation contents in the interactive link graph in the conversation reference order, as Figure 5 shown. It can also be directly display the conversation contents in each interactive link graph on the intelligent interaction page in the conversation reference order. Of course, it can also be other ways, and the embodiments of the present disclosure do not impose any restrictions on this. However, no matter which display method is used, as long as the conversation content in the interactive link graph is triggered, the triggered conversation content can be located and displayed on the intelligent interaction page.

[0062] Through the above method, an interactive link graph determined based on the reference relationships between different conversation contents on the intelligent interaction page can be displayed on the intelligent interaction page. Thus, the interactive link graph can assist users in sorting out and analyzing ideas in multi-round conversations, further improving the analysis efficiency and user experience. In addition, since when the conversation content in the interactive link graph is triggered, the triggered conversation content can be located and displayed on the intelligent interaction page, it is possible to quickly jump to the corresponding conversation content without manually searching through a large number of conversation records, further improving the user experience.

[0063] In a possible way, the intelligent interaction page includes a full-screen display style and a floating window display style. The intelligent interaction page in the full-screen display style provides a data table display function. Correspondingly, the A / B test data analysis method based on the large model agent can also include: When the intelligent interaction page is displayed in the full-screen display style and a data table is displayed on the intelligent interaction page, in response to the style switching operation of the intelligent interaction page, display the intelligent interaction page in the floating window display style and update the display of the data table to text content, where the text content is used to describe the table content of the data table.

[0064] In this embodiment, the operation of switching the style of the intelligent interaction page can be to input a shortcut key in an external device to switch the style of the intelligent interaction page. For example, it can be to input the shortcut key "Ctrl+D" on the keyboard to switch the style of the intelligent interaction page, or a style switching control can be displayed on the intelligent interaction page. Thus, by triggering the style switching control, the style of the intelligent interaction page can be switched. Of course, the operation of switching the style of the intelligent interaction page can also be other, and the embodiments of the present disclosure do not impose any restrictions on this.

[0065] Exemplarily, as Figure 6 shown, the intelligent interaction page is displayed in a full-screen display style, and a data table is displayed in the middle of the intelligent interaction page, text content is displayed below the data table, and a style switching control "Style Switching" is displayed in the upper right corner of the intelligent interaction page. Thus, when "Style Switching" is triggered by an operation such as clicking, the intelligent interaction page can be displayed in a floating window display style, and the display of the data table can be cancelled.

[0066] In a possible way, in order to be able to quickly view the data table in the intelligent interaction page, when the display content in the intelligent interaction page includes a data table and the intelligent interaction page is displayed in a floating window display style, a quick switching instruction for switching the intelligent interaction page from the floating window display style to the full-screen display style can be displayed in the intelligent interaction page. For example, a quick switching instruction "View the data table in full-screen display mode" can be displayed in the intelligent interaction page. As Figure 6 shown, thus, by triggering the quick switching instruction "View the data table in full-screen display mode", the intelligent interaction page can be quickly switched from the floating window display style to the full-screen display style, thereby further improving the user experience.

[0067] It should be understood that when the intelligent interaction page is displayed in a floating window display style, the display window of the intelligent interaction page is smaller than the full-screen display style. Thus, in order to completely display the data table in the intelligent interaction page, there is a problem that the table content is small and the readability of the table content is poor. Thus, in this embodiment, when the intelligent interaction page is switched from the full-screen display style to the floating window display style, the data table in the intelligent interaction page is updated and displayed as text content, thereby increasing the readability of the display content in the intelligent interaction page and further improving the user experience.

[0068] Based on the same concept, the embodiments of the present disclosure also provide an A / B test data analysis device based on a large model agent. As Figure 7 shown, the A / B test data analysis device 700 based on a large model agent may include: A first display module 701, configured to display an intelligent interaction page associated with an agent, where the agent is used to perform joint analysis on different data metrics through a large model; A determination module 702, configured to determine at least two data metrics as target data metrics to be jointly analyzed in response to an input operation on at least two data metrics on the intelligent interaction page; A second display module 703, configured to display an analysis result of data for the joint analysis requirement on the intelligent interaction page in response to a joint analysis requirement for the target data metrics triggered on the intelligent interaction page.

[0069] Through the above-mentioned A / B test data analysis device 700 based on a large model agent, target data metrics to be jointly analyzed and a joint analysis requirement for the target data metrics can be input on the intelligent interaction page, and the large model can automatically analyze the joint analysis requirement for the target data metrics to obtain an analysis result of data for the joint analysis requirement. Thus, through relatively simple input operations, automatic data analysis can be achieved, thereby simplifying the data analysis process and improving the analysis efficiency. On the other hand, since this solution can support the input of data metrics and can perform joint analysis on different data metrics, accordingly, targeted analysis of different data metrics can be customized according to user requirements, thereby improving the flexibility of data analysis and the user experience.

[0070] In a possible manner, the determination module 702 may include: A first display unit, configured to display different data metrics in response to a shortcut instruction triggered on the intelligent interaction page; A first determination unit, configured to determine at least two data metrics corresponding to the selection operation as target data metrics to be jointly analyzed in response to a selection operation on at least two data metrics among different data metrics.

[0071] In a possible manner, the joint analysis requirement may include multiple sub-joint analysis requirements with logical relationships. Correspondingly, the second display module 703 may include: A second display unit, configured to display an analysis task script for implementing the joint analysis requirement on the intelligent interaction page in response to a joint analysis requirement for the target data metrics triggered on the intelligent interaction page; A third display unit, configured to display an analysis result of data obtained based on the analysis task script on the intelligent interaction page in response to a trigger operation on the analysis task script.

[0072] In a possible manner, the third display unit may include: A determination subunit, configured to determine a target analysis task script according to an edit operation in response to an edit operation on the analysis task script; A display subunit, configured to display an analysis result of data obtained from the target analysis task script on the intelligent interaction page in response to a trigger operation on the target analysis task script, In a possible way, the A / B test data analysis device 700 based on the large model agent may further include: A third display module, configured to display a content reference control on the intelligent interaction page, where the content reference control is used to reference the historical session content displayed on the intelligent interaction page; Correspondingly, the second display module 703 may include: A first determination unit, configured to determine the target historical session content to be referenced in the historical session content in response to a trigger operation on the content reference control; A fourth display unit, configured to display the data analysis result for the joint analysis requirement on the intelligent interaction page based on the target historical session content in response to the joint analysis requirement for the target data metric triggered on the intelligent interaction page.

[0073] In a possible way, the A / B test data analysis device 700 based on the large model agent may further include: A fourth display module, configured to display an interaction link graph on the intelligent interaction page, where the interaction link graph is determined based on the reference relationship between different session contents on the intelligent interaction page; A fifth display module, configured to position and display the target session content on the intelligent interaction page in response to a trigger operation on the target session content in the interaction link graph.

[0074] In a possible way, the intelligent interaction page includes a full-screen display style and a floating window display style. The intelligent interaction page in the full-screen display style provides a data table display function. Correspondingly, the A / B test data analysis device 700 based on the large model agent may further include: A sixth display module, configured to, when the intelligent interaction page is displayed in the full-screen display style and a data table is displayed on the intelligent interaction page, in response to a style switching operation on the intelligent interaction page, display the intelligent interaction page in the floating window display style and cancel the display of the data table.

[0075] Based on the same concept, an embodiment of the present disclosure also provides a computer-readable medium, on which a computer program is stored, and when the program is executed by a processing device, the steps of any of the above A / B test data analysis methods based on the large model agent are implemented.

[0076] Based on the same concept, an embodiment of the present disclosure also provides an electronic device, which may include: A storage device, on which a computer program is stored; A processing device, configured to execute the computer program in the storage device to implement the steps of any of the above A / B test data analysis methods based on the large model agent.

[0077] Based on the same concept, an embodiment of the present disclosure also provides a computer program product, including a computer program, which when executed by a processor implements the steps of any of the above-mentioned A / B test data analysis methods based on large model agents.

[0078] Reference is made below Figure 8 , which shows a schematic structural diagram of an electronic device 800 suitable for implementing the embodiments of the present disclosure. The terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The electronic device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present disclosure.

[0079] As Figure 8 shown, the electronic device 800 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 802 or the program loaded from the storage device 808 into the random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The processing device 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. The input / output (I / O) interface 805 is also connected to the bus 804.

[0080] Generally, the following devices may be connected to the I / O interface 805: an input device 806 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 808 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 809. The communication device 809 can allow the electronic device 800 to communicate with other devices wirelessly or wirelessly to exchange data. Although Figure 8 shows the electronic device 800 having various devices, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices may be implemented or had.

[0081] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program including program code for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by a processing device 801, the above-described functions defined in the methods of the embodiments of the present disclosure are performed.

[0082] It should be noted that the above-mentioned computer-readable medium in the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of a computer-readable storage medium can include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0083] In some embodiments, communication can be carried out using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0084] The above computer-readable medium can be included in the above electronic device; it can also exist separately without being assembled into the electronic device.

[0085] The above computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: display an intelligent interaction page associated with an agent, where the agent is used to perform joint analysis on different data metrics through a large model; in response to an input operation on at least two data metrics on the intelligent interaction page, determine the at least two data metrics as target data metrics to be jointly analyzed; in response to a joint analysis requirement for the target data metrics triggered on the intelligent interaction page, display the data analysis result for the joint analysis requirement on the intelligent interaction page.

[0086] Computer program code for performing the operations of the present disclosure can be written in one or more programming languages or combinations thereof. The programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., by using an Internet service provider to connect through the Internet).

[0087] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or by a combination of dedicated hardware and computer instructions.

[0088] The modules described in the embodiments of the present disclosure can be implemented in software or in hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0089] The functions described above herein can be performed, at least in part, by one or more hardware logic components. By way of example, and without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Arrays (FPGA), Application Specific Integrated Circuits (ASIC), Application Specific Standard Products (ASSP), System on a Chip (SOC), Complex Programmable Logic Devices (CPLD), and the like.

[0090] In the context of the present disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM or Flash memory), an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0091] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

[0092] In addition, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although a number of specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0093] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms for implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be elaborated here.

Claims

1. A method for analyzing A / B test data based on large model agents, characterized in that, The A / B test data analysis method based on a large model agent includes: Display an intelligent interaction page associated with the agent, where the agent is used to jointly analyze different data metrics in an A / B test experiment through a large model; In response to an input operation on at least two data metrics on the intelligent interaction page, determine the at least two data metrics as target data metrics to be jointly analyzed; In response to a joint analysis requirement for the target data metrics triggered on the intelligent interaction page, display the data analysis result for the joint analysis requirement on the intelligent interaction page.

2. The A / B test data analysis method based on large model agents according to claim 1, wherein The step of determining the at least two data metrics as target data metrics to be jointly analyzed in response to an input operation on at least two data metrics on the intelligent interaction page includes: In response to a shortcut instruction triggered on the intelligent interaction page, display different data metrics; In response to a selection operation on at least two data metrics among the different data metrics, determine the at least two data metrics corresponding to the selection operation as target data metrics to be jointly analyzed.

3. The A / B test data analysis method based on large model agents according to claim 1 or 2, characterized in that, The joint analysis requirement includes multiple sub-joint analysis requirements with logical relationships. The step of displaying the data analysis result for the joint analysis requirement on the intelligent interaction page in response to a joint analysis requirement for the target data metrics triggered on the intelligent interaction page includes: In response to a joint analysis requirement for the target data metrics triggered on the intelligent interaction page, display an analysis task script for implementing the joint analysis requirement on the intelligent interaction page; In response to a trigger operation on the analysis task script, display the data analysis result obtained based on the analysis task script on the intelligent interaction page.

4. The A / B test data analysis method based on large model agents according to claim 3, wherein, The step of displaying the data analysis result obtained based on the analysis task script on the intelligent interaction page in response to a trigger operation on the analysis task script includes: In response to an editing operation on the analysis task script, determine a target analysis task script according to the editing operation; In response to a trigger operation on the target analysis task script, display the data analysis result obtained from the target analysis task script on the intelligent interaction page.

5. The A / B test data analysis method based on large model agents according to claim 1 or 2, characterized in that, The A / B test data analysis method based on a large model agent further includes: Display a content reference control on the intelligent interaction page, where the content reference control is used to reference the historical session content displayed on the intelligent interaction page; The step of displaying the data analysis result for the joint analysis requirement on the intelligent interaction page in response to a joint analysis requirement for the target data metrics triggered on the intelligent interaction page includes: In response to a trigger operation on the content reference control, determine target historical session content to be referenced in the historical session content; In response to a joint analysis requirement for the target data metrics triggered on the intelligent interaction page, display the data analysis result for the joint analysis requirement on the intelligent interaction page based on the target historical session content.

6. The A / B test data analysis method based on large model agents according to claim 1 or 2, wherein The A / B test data analysis method based on a large model agent further includes: Display an interaction link graph on the intelligent interaction page, where the interaction link graph is determined based on the reference relationship between different session contents in the intelligent interaction page; In response to a trigger operation on the target session content in the interaction link graph, locate and display the target session content on the intelligent interaction page.

7. The A / B test data analysis method based on large model agents according to claim 1 or 2, characterized in that, The intelligent interaction page includes a full-screen display style and a floating window display style. The intelligent interaction page in the full-screen display style provides a data table display function. The A / B test data analysis method based on a large model agent further includes: When the intelligent interaction page is displayed in the full-screen display style and a data table is displayed on the intelligent interaction page, in response to a style switching operation on the intelligent interaction page, display the intelligent interaction page in the floating window display style and cancel the display of the data table.

8. An A / B test data analysis device based on a large model intelligent agent, characterized in that, The data analysis device includes: A first display module for displaying an intelligent interaction page associated with an agent, where the agent is used to jointly analyze different data metrics through a large model; A determination module for, in response to an input operation on at least two data metrics on the intelligent interaction page, determining the at least two data metrics as target data metrics to be jointly analyzed; A second display module for, in response to a joint analysis requirement for the target data metrics triggered on the intelligent interaction page, displaying an analysis result of the joint analysis requirement on the intelligent interaction page.

9. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processing device, it implements the steps of the method according to any one of claims 1-7.

10. An electronic device, characterized in that, It includes: A storage device on which a computer program is stored; A processing device for executing the computer program in the storage device to implement the steps of the method according to any one of claims 1-7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-7.

Images