Visual A / B test data analysis method based on large model and corresponding device

Abstract

The invention discloses a visual A / B test data analysis method and device based on a large model, and the method comprises the steps: displaying A / B test data index information of a to-be-analyzed object under different analysis dimensions on a data analysis page, and carrying out the associated display of an analysis function entry used for triggering the large model to analyze the A / B test data index information for each analysis dimension; and in response to a trigger operation on the analysis function entry associated with the target analysis dimension, displaying a data analysis result of the A / B test data index information under the target analysis dimension. Therefore, the analysis function entry can be triggered in a visual mode, and the analysis efficiency and the analysis accuracy can be improved by automatically analyzing the A / B test data index information through a large model. Besides, each analysis dimension displays an analysis function entry in an associated manner, so that the analysis function entry associated with the corresponding analysis dimension can be triggered based on user requirements, targeted A / B test data index information analysis is realized, and the user experience is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and in particular, to a method and corresponding device for visual A / B test data analysis based on a large model. Background Art

[0002] With the development of computer technologies, the complexity of data has been increasing day by day. In order to extract valuable information from data, it is generally necessary to analyze the data index information of the data. For example, in the content delivery scenario, an A / B experiment can generally be set up, and then by comparing and analyzing the data index information in the experimental data, a content delivery strategy for content delivery can be obtained.

[0003] However, in the related art, the A / B test data index information is generally analyzed based on manual analysis, which has problems of low analysis efficiency and poor analysis accuracy. Summary of the Invention

[0004] This summary of the invention is provided to introduce concepts in a brief form, which will be described in detail in the following detailed implementation section. This summary of the invention is not intended to identify the key features or essential features of the claimed technical solution, nor is it intended to be used to limit the scope of the claimed technical solution.

[0005] In a first aspect, the present disclosure provides a method for visual A / B test data analysis based on a large model, and the method for visual A / B test data analysis based on a large model includes:

[0006] Displaying, on a data analysis page, A / B test data index information of an object to be analyzed under different analysis dimensions, and associating and displaying an analysis function entry for each of the analysis dimensions, where the analysis function entry is used to trigger the large model to perform analysis based on the A / B test data index information of the object to be analyzed;

[0007] Responding to a trigger operation on the analysis function entry associated with a target analysis dimension, and displaying a data analysis result of the A / B test data index information under the target analysis dimension.

[0008] In a second aspect, the present disclosure provides a device for visual A / B test data analysis based on a large model, and the device for A / B test data analysis based on a large model includes:

[0009] A first display module, configured to display, on a data analysis page, A / B test data index information of an object to be analyzed under different analysis dimensions, and associating and displaying an analysis function entry for each of the analysis dimensions, where the analysis function entry is used to trigger the large model to perform analysis based on the A / B test data index information of the object to be analyzed;

[0010] The second display module is configured to display the data analysis result of the A / B test data metrics information under the target analysis dimension in response to a trigger operation on the analysis function entry associated with the target analysis dimension.

[0011] In a third aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, and when the computer program is executed by a processing device, the steps of the method described in the first aspect are implemented.

[0012] In a fourth aspect, the present disclosure provides an electronic device, including:

[0013] A storage device having a computer program stored thereon;

[0014] A processing device configured to execute the computer program in the storage device to implement the steps of the method described in the first aspect.

[0015] In a fifth aspect, the present disclosure provides a computer program product including a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0016] Through the above technical solutions, the A / B test data metrics information of the object to be analyzed and the analysis function entry for triggering the large model to analyze the A / B test data metrics information can be displayed on the data analysis page. Thus, the analysis function entry can be triggered in a visual manner to implement the automatic analysis of the A / B test data metrics information. On the one hand, the analysis process can be simplified and the analysis efficiency can be improved. On the other hand, since the data analysis result is obtained based on the large model's analysis of the A / B test data metrics information, the analysis accuracy can be improved. Additionally, since each analysis dimension is associated with and displays an analysis function entry, thus, the analysis function entry associated with the corresponding analysis dimension can be triggered based on user needs, thereby implementing targeted analysis of the A / B test data metrics information and further improving the user experience.

[0017] Other features and advantages of the present disclosure will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In combination with the drawings and with reference to the following specific implementation, the above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic and the original elements and elements are not necessarily drawn to scale. In the drawings:

[0019] Figure 1 is a flowchart of a method for visual A / B test data analysis based on a large model shown according to an exemplary embodiment of the present disclosure;

[0020] Figure 2 It is a schematic diagram showing A / B test data metric information according to an exemplary embodiment of the present disclosure;

[0021] Figure 3 It is a schematic diagram showing the display of an analysis function entry according to an exemplary embodiment of the present disclosure;

[0022] Figure 4 It is a schematic diagram showing the display of data analysis results according to an exemplary embodiment of the present disclosure;

[0023] Figure 5 It is a schematic diagram showing the display of an information query control and / or an analysis function trigger control according to an exemplary embodiment of the present disclosure;

[0024] Figure 6 It is a schematic diagram showing the display of a candidate analysis object according to an exemplary embodiment of the present disclosure;

[0025] Figure 7 It is a schematic diagram showing the display of an analysis task script according to an exemplary embodiment of the present disclosure;

[0026] Figure 8 It is a schematic diagram of content citation according to an exemplary embodiment of the present disclosure;

[0027] Figure 9 It is a schematic diagram showing the display of an interaction link map according to an exemplary embodiment of the present disclosure;

[0028] Figure 10 It is a schematic diagram showing the style switching of an intelligent interaction page according to an exemplary embodiment of the present disclosure;

[0029] Figure 11 It is a schematic diagram showing the display of data metric information according to an exemplary embodiment of the present disclosure;

[0030] Figure 12 It is a schematic diagram showing the display of data detail information according to an exemplary embodiment of the present disclosure;

[0031] Figure 13 It is a block diagram of a visualization A / B test data analysis device based on a large model according to an exemplary embodiment of the present disclosure;

[0032] Figure 14 It is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present disclosure. Detailed implementation manners

[0033] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0034] It should be understood that the various steps recited in the method embodiments of the present disclosure can be executed in a different order and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard.

[0035] As used herein, the term "comprising" and its variations are open-ended, i.e., "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0036] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependent relationships.

[0037] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise clearly specified in the context, it should be understood as "one or more".

[0038] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0039] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0040] For example, when receiving the user's active request, a prompt message is sent to the user to clearly prompt the user that the operation requested by the user will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, an application program, a server or a storage medium that executes the operation of the technical solution of the present disclosure according to the prompt message.

[0041] As an optional but non-limiting implementation, in response to receiving an active request from a user, the way to send a prompt message to the user can be, for example, in the form of a pop-up window, and the prompt message can be presented in text in the pop-up window. In addition, the pop-up window can also carry selection controls for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0042] It can be understood that the above notification and the process of obtaining user authorization are only illustrative and do not constitute a limitation on the implementation of the present disclosure. Other ways that comply with relevant laws and regulations can also be applied to the implementation of the present disclosure.

[0043] At the same time, it can be understood that the data involved in the present technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the corresponding laws, regulations and related provisions.

[0044] As mentioned in the background art, in the related art, the A / B test data index information is generally analyzed based on manual analysis, which has problems such as low analysis efficiency and poor analysis accuracy.

[0045] Exemplarily, in the content delivery scenario, multiple different content delivery strategies can be provided. For example, content delivery strategy A and content delivery strategy B can be provided. In order to determine the optimal content delivery strategy from content delivery strategy A and content delivery strategy B, an A / B experiment can be set up, that is: for the target content to be delivered, let a part of the target content to be delivered be delivered based on content delivery strategy A, and let another part of the target content to be delivered be delivered based on content delivery strategy B. Finally, the optimal content delivery strategy is determined by comparing the experimental data (i.e., delivery data). However, in the actual scenario, the optimal content delivery strategy cannot be obtained by simply comparing a single set of experimental data. Instead, dozens of data in the experimental data need to be comprehensively evaluated, and various analysis tools are used to drill down and attribute the data indicators that do not meet the expectations in order to obtain the optimal content delivery strategy. However, this process requires data analysts to have certain professional analysis knowledge and a deep understanding of the business, so there are problems such as high analysis difficulty. In addition, in the related art, most of the experimental analysis tools are mainly based on the Graphical User Interface (GUI). When abnormal changes occur in the data indicators and in-depth analysis is required, data analysts often need to rely on business experience to select appropriate analysis tools, and then search for other data indicators and dimensions related to the change of this data indicator. Since this process requires continuous attempts to locate the cause, there are problems such as low analysis efficiency and poor accuracy.

[0046] In view of this, the present disclosure provides a visualization A / B test data analysis method and corresponding device based on a large model to solve the above technical problems.

[0047] The following further explains the embodiments of the present disclosure with reference to the accompanying drawings.

[0048] Figure 1 is a flowchart of a visualization A / B test data analysis method based on a large model shown according to an exemplary embodiment of the present disclosure. Referring to Figure 1 this, the visualization A / B test data analysis method based on a large model may include the following steps:

[0049] S101: Display the A / B test data index information of the object to be analyzed under different analysis dimensions on the data analysis page, and associatively display an analysis function entry for each analysis dimension, where the analysis function entry is used to trigger the large model to perform analysis based on the A / B test data index information of the object to be analyzed.

[0050] In this embodiment, the object to be analyzed, different analysis dimensions, and A / B test data index information can all be flexibly determined according to the actual application scenario, and the embodiments of the present disclosure do not impose any restrictions on this.

[0051] Exemplarily, in a controlled experiment scenario, the object to be analyzed can be a single data index, or a data index group including multiple single data indexes. Of course, it can also be others, and the embodiments of the present disclosure do not impose any restrictions on this. Correspondingly, the A / B test data index information can be the data performance of a single data index or data index group in the experimental group, or the data performance of a single data index or data index group in the control group, or the data comparison performance between a single data index and data index group in the experimental group and the control group. Of course, it can also be others, and the embodiments of the present disclosure do not impose any restrictions on this.

[0052] It should be understood that in a controlled experiment scenario, generally, there may be a reference group and at least one experimental group. Therefore, in order to be able to give an overall summary of the experimental results, or analyze the comparison situation between the experimental group, the reference group, and / or different experimental groups and the reference group, different analysis dimensions may include the experimental group dimension. Thus, by triggering the analysis function entry associated with the experimental group dimension, the comparison situation between the experimental group, the reference group, and / or different experimental groups and the reference group can be analyzed.

[0053] It should also be understood that when the object to be analyzed is a data index group, since a data index group can include multiple single data indexes with link associations, thus, in order to analyze a single data index, different analysis dimensions can include a single index dimension, so that the single data index in the data index group to be analyzed can be analyzed by triggering the analysis function entry associated with the single index dimension. At the same time, in order to analyze the data index group as a whole, different analysis dimensions can also include an index group dimension, thus, the overall analysis of the data index group to be analyzed can be performed by triggering the analysis function entry associated with the index group dimension. Further, since the data performance of a single data index in the data index group may be different in different experimental groups and / or control groups, thus, in order to analyze the single data index in different experimental groups and / or control groups, different analysis dimensions can also include a combination dimension, which can be determined according to the experimental group dimension and the single index dimension, thus, the data performance of the single data index in the data index group to be analyzed in different experimental groups and / or control groups can be analyzed by triggering the analysis function entry associated with the combination dimension.

[0054] That is to say, in a possible way, displaying the A / B test data index information of the object to be analyzed under different analysis dimensions on the data analysis page may include:

[0055] Displaying the A / B test data index information of the object to be analyzed on the data analysis page under at least one of the experimental group dimension, the index group dimension, the single index dimension, and the combination dimension, where the combination dimension is determined according to the experimental group dimension and the single index dimension.

[0056] Exemplarily, as Figure 2 shown, after opening the data analysis platform, the home page of the data analysis platform can be displayed, and an analysis module "Index Data" for analyzing the A / B test data index information can be displayed on the home page of the data analysis platform. When the analysis module "Index Data" is triggered by an operation such as clicking, the data analysis page can be displayed, and multiple candidate data index groups can be displayed in the left display area of the data analysis page. When the "Fourth Data Index Group" in the multiple data index groups is triggered by an operation such as clicking, the "Fourth Data Index Group" is used as the object to be analyzed, and the A / B test data index information of the "Fourth Data Index Group" under the experimental group dimension, the index group dimension, the single index dimension, and the combination dimension can be displayed in the right display area of the data analysis page, and the analysis function entry is associated and displayed for the experimental group dimension, the index group dimension, the single index dimension, and the combination dimension. Among them, the A / B test data index information under the experimental group dimension corresponds to the display content in the Figure 2 "Data Summary" part; the A / B test data index information under the index group dimension corresponds to Figure 2The display content of the "Fourth Data Index Group" part; the A / B test data index information under a single index dimension corresponds to Figure 2 The display content corresponding to data index 1 or data index 2 in; the A / B test data index information under the combined dimension corresponds to the display content of data index 1 under the control group or experimental group 1, or corresponds to the display content of data index 2 under the control group or experimental group 1.

[0057] It should be understood that the analysis function entry associated with the experimental group dimension, index group dimension, single index dimension, and combined dimension can be displayed when displaying the corresponding A / B test data index information, or can be displayed when triggering the A / B test data index information. The embodiments of the present disclosure do not impose any restrictions on this. By way of example, as Figure 2 shown, since the display areas of the experimental group dimension and the index group dimension are relatively large, the corresponding analysis function entry can be displayed when displaying the A / B test data index information corresponding to the experimental group dimension and the index group dimension. And since the data indexes in the data index group are displayed in a table format, therefore, in order to increase the readability of the table content, the analysis function entry associated with the single index dimension and the combined dimension can be displayed when triggering the corresponding A / B test data index information. For example, when the mouse hovers over a single data index, the analysis function entry associated with the single data index can be displayed, and when the mouse hovers over the index value of a single data index in the control group, the analysis function entry associated with the combined dimension can be displayed, as Figure 3 shown.

[0058] S102: In response to a trigger operation on the analysis function entry associated with the target analysis dimension, display the data analysis result of the A / B test data index information under the target analysis dimension.

[0059] In this embodiment, the target analysis dimension is any analysis dimension displayed on the data analysis page. By way of example, continuing to refer to the above example, since the data analysis page displays the experimental group dimension, index group dimension, single index dimension, and combined dimension, therefore, the target analysis dimension can be any one of the experimental group dimension, index group dimension, single index dimension, and combined dimension.

[0060] In this embodiment, the data analysis results can be determined according to the actual situation, and the embodiments of the present disclosure do not impose any restrictions on this. By way of example, when the target analysis dimension is the experimental group dimension, in order to give an overall summary of the experimental results, the data analysis results may include: summary content obtained by summarizing preset types of data indicators respectively at the single experimental group granularity, for example, summarizing business indicators, revenue indicators, product core indicators, and / or guardrail indicators. When the target analysis dimension is the indicator group dimension, the data analysis results may include at least one of the following: data indicators with positive significance or negative significance in the data indicator group; data indicators with current significance but possibly insignificant in the future in the data indicator group; summary content summarizing the overall performance of the data indicator group; data indicators associated with the data indicator group. When the target analysis dimension is the single indicator dimension, the data analysis results may include at least one of the following: summary content of the data indicator in each experimental group or control group; significance information of the data indicator, such as "significantly less than the preset number of days", "negative significant trend", "positive significant trend", and / or "dominant convergence", etc.; correlation attribution and / or dimension attribution of the data indicator; common analysis dimensions of the data indicator; data indicators associated with the data indicator. When the target analysis dimension is the combined dimension, the data analysis results may include at least one of the following: summary content of the data indicator in a specific experimental group; significance information of the data indicator in a specific experimental group, such as "significantly less than the preset number of days", "negative significant trend", "positive significant trend", and / or "dominant convergence", etc.; correlation attribution and / or dimension attribution of the data indicator in a specific experimental group; common analysis dimensions of the data indicator in a specific experimental group; data indicators associated with the data indicator in a specific experimental group.

[0061] Through the above technical solution, the A / B test data indicator information of the object to be analyzed can be displayed on the data analysis page, as well as the analysis function entry for triggering the large model to analyze the A / B test data indicator information. Thus, by triggering the analysis function entry, the automatic analysis of the A / B test data indicator information can be realized. On the one hand, the analysis process can be simplified and the analysis efficiency can be improved. On the other hand, since the data analysis results are obtained based on the large model's analysis of the A / B test data indicator information, the analysis accuracy can be improved. In addition, since an analysis function entry is associated and displayed for each analysis dimension, thus, the analysis function entry associated with the corresponding analysis dimension can be triggered based on the user's needs, so as to realize the targeted analysis of the A / B test data indicator information, and further improve the user experience.

[0062] To facilitate the understanding of the large model-based visual A / B test data analysis method provided by the present disclosure, the possible implementation manners in the present disclosure will be described below.

[0063] Among possible ways, the data analysis result showing the A / B test data metric information under the target analysis dimension may include:

[0064] Showing at least one of the first data analysis result, the second data analysis result, and the third data analysis result, where the first data analysis result is used to indicate the change trend of the A / B test data metric information under the target analysis dimension, the second data analysis result is used to indicate the reason for the change trend of the A / B test data metric information under the target analysis dimension, and the third data analysis result is used to indicate the analysis direction when the A / B test data metric information under the target analysis dimension shows a change trend.

[0065] In this embodiment, the change trend can be displayed in the display style of a data table or a data graph, or can be displayed in the display style of text. For example, Figure 4 as shown, of course, it can also be displayed through other display styles, and the embodiments of the present disclosure do not impose any restrictions on this. When the change trend is displayed in the text display style, the change trend can be marked for display. For example, the change trend can be highlighted or marked with a preset color, etc., so that the user's attention can be attracted through the marked display, enabling the user to more quickly notice the change trend of the A / B test data metric information, and then the corresponding business can be adjusted according to the change trend. By way of example, in the content delivery scenario, the content delivery strategy can be adjusted or improved accordingly based on the change trend of the A / B test data metric information, thereby improving the effect of content delivery based on the content delivery strategy.

[0066] Through the above technical solution, the change trend and / or the reason for the change can be displayed. Compared with the related art that only provides static data analysis results, this solution can perform data analysis from the dimensions of the change trend and the reason for the change. Thus, the change trend and / or the reason for the change of the A / B test data metric information can be intuitively displayed. On the one hand, the depth of data analysis can be improved, and on the other hand, the user experience can be improved. In addition, since the analysis direction analysis for the change trend can also be displayed, the indicator data information can be analyzed again based on the analysis direction. Compared with the related art where other data indicators and dimensions related to the change of this data indicator are found for further analysis based on the business experience of data analysts, not only can the analysis efficiency and analysis accuracy be improved, but also the user experience can be improved.

[0067] Among possible ways, the visualization A / B test data analysis method based on a large model may further include:

[0068] When displaying the data analysis results of the A / B test data metrics information under the target analysis dimension, display the information query control and / or the analysis function trigger control associated with the target analysis dimension. Among them, the information query control is used to trigger the large model to query the A / B test data metrics information under the target analysis dimension, and the analysis function trigger control is used to trigger the large model to analyze the A / B test data metrics information under the target analysis dimension based on the data analysis results.

[0069] In this embodiment, the information query controls and / or the analysis function trigger controls associated with different analysis dimensions may be the same or different, and the embodiments of the present disclosure do not impose any restrictions on this. In a possible manner, in order to enable the information query control and / or the analysis function trigger control to better adapt to the analysis characteristics of different analysis dimensions, the information query controls and / or the analysis function trigger controls associated with different analysis dimensions are different. Thus, personalized operations and efficient interactions for each analysis dimension can be achieved through the information query control and / or the analysis function trigger control, thereby improving the interaction efficiency and user experience.

[0070] In this embodiment, the display quantity and display position of the information query control and / or the analysis function trigger control may be determined according to the actual situation, and the embodiments of the present disclosure do not impose any restrictions on this. By way of example, as Figure 5 shown, the information query control "experimental data query" and the analysis function trigger control "dimension drill-down" can be displayed below the data analysis results.

[0071] Through the above method, when displaying the data analysis results, the information query control and / or the analysis function trigger control can be displayed. Thus, information query can be performed through the information query control, and / or data analysis can be performed through the analysis function trigger control. On the one hand, the information query efficiency and / or data analysis efficiency can be improved, and on the other hand, the user experience can be further improved.

[0072] In a possible manner, in response to a trigger operation on the analysis function entry associated with the target analysis dimension, displaying the data analysis results of the A / B test data metrics information under the target analysis dimension may include:

[0073] In response to a trigger operation on the analysis function entry associated with the target analysis dimension, display an intelligent interaction page, and display the data analysis results of the A / B test data metrics information under the target analysis dimension on the intelligent interaction page, where the intelligent interaction page is used for the user to have a conversation with an intelligent agent associated with a large model;

[0074] Correspondingly, the visual A / B test data analysis method based on a large model may further include:

[0075] In response to the data analysis requirements triggered by the user on the intelligent interaction page, display the data analysis results for the data analysis requirements on the intelligent interaction page.

[0076] In this embodiment, the data analysis requirements triggered by the user on the intelligent interaction page can be triggered through a session. For example, the data analysis requirements can be triggered by voice input, or can be triggered by entering the data analysis requirements in the form of natural language in the text input box. It can also be that preset data analysis requirements are pre-displayed on the intelligent interaction page, and when the preset data analysis requirements are triggered through operations such as clicking or dragging, the data analysis requirements are triggered on the intelligent interaction page. Of course, it can also be that the data analysis requirements are triggered on the intelligent interaction page in other ways, and the embodiments of the present disclosure do not make any restrictions on this.

[0077] In this embodiment, the data analysis requirements triggered by the user on the intelligent interaction page can be data query, metric link attribution, dimension drill-down attribution, analysis content summary, metric impact range, or experiment summary. Of course, it can also be others, and the embodiments of the present disclosure do not make any restrictions on this.

[0078] In this embodiment, when the analysis function entry is triggered, an intelligent interaction page for the user to communicate with the intelligent agent associated with the large model can be displayed, so that the user can trigger data analysis requirements on the intelligent interaction page according to their own needs, thereby realizing targeted data analysis and further improving the user experience.

[0079] In a possible way, in response to the data analysis requirements triggered by the user on the intelligent interaction page, displaying the data analysis results for the data analysis requirements on the intelligent interaction page may include:

[0080] In response to the shortcut command triggered by the user on the intelligent interaction page, display candidate analysis objects; in response to the selection operation among the candidate analysis objects, determine the candidate analysis object corresponding to the selection operation; in response to the first data analysis requirement entered by the user on the intelligent interaction page for the candidate analysis object corresponding to the selection operation, display the data analysis results for the first data analysis requirement on the intelligent interaction page.

[0081] In this embodiment, the shortcut instruction can be determined according to the actual situation, and the embodiments of the present disclosure do not impose any restrictions thereon. By way of example, the shortcut instruction can be a preset character. For example, it can be "#". Thus, when "#" is input on the intelligent interaction page, candidate analysis objects can be displayed. By way of example, the shortcut instruction can be a preset voice instruction. For example, it can be "object selection". Thus, when the language instruction "object selection" is detected on the intelligent interaction page, candidate analysis objects can be displayed. By way of example, the shortcut instruction can be a shortcut key. For example, it can be "Ctrl+Q". Thus, when the intelligent interaction page detects that the external input device triggers the shortcut key "Ctrl+Q", candidate analysis objects can be displayed.

[0082] In this embodiment, the candidate analysis objects can be determined according to the actual situation, and the embodiments of the present disclosure do not impose any restrictions thereon. By way of example, in a controlled experiment scenario, the candidate analysis object can be a single data indicator, or a data indicator group including multiple single data indicators. Of course, it can also be other things, and the embodiments of the present disclosure do not impose any restrictions thereon.

[0083] It should be understood that when the candidate analysis object is a single data indicator, since the single data indicator can exist in different data indicator groups, in order to accurately analyze the single data indicator, when the candidate analysis object is displayed, the candidate data indicator group can be displayed first. When the candidate data indicator group is triggered by an operation such as clicking, the single data indicators in the candidate data indicator group are displayed, so as to help the user clarify their analysis intention and avoid confusion caused by homonymous indicators.

[0084] By way of example, as Figure 6 shown, the intelligent interaction page can include a content display area and a content input area. Among them, a send control "send" for sending the content in the content input area to the content display area is displayed in the content input area. Thus, when "#" is input in the content input area, a drop-down panel can be displayed, and multiple candidate data indicator groups can be displayed in the drop-down panel. When the "candidate data indicator group 2" in the candidate data indicator group is triggered by an operation such as clicking, the multiple single data indicators corresponding to the "candidate data indicator group 2" can be displayed. When the "data indicator 2" is triggered by an operation such as clicking, "data indicator 2" can be displayed in the content input area. When the user inputs a first data analysis requirement for the "data indicator 2" in the content input area, the first data analysis requirement can be sent to the content display area by clicking the "send" control, and the data analysis result for the first data analysis requirement is displayed in the display area below the first data analysis requirement.

[0085] It should be understood that in the data analysis scenario, the number of objects to be analyzed is at least several and at most dozens. When the user manually inputs the objects to be analyzed, there may be input errors due to inaccurate memory, and thus the need for repeated verification and re - input. In this embodiment, since the candidate analysis objects can be displayed when a quick instruction is triggered, the user can select the interested analysis objects from the candidate analysis objects for analysis according to the user's needs. Compared with the method of manually inputting the objects to be analyzed by the user, the problem of repeated input due to manual input errors can be reduced, thereby improving the analysis efficiency and user experience.

[0086] In this embodiment, the number of candidate analysis objects can be one or multiple, and the embodiments of the present disclosure do not impose any restrictions on this. When there are multiple candidate analysis objects, after selecting the candidate analysis objects, the multiple candidate analysis objects can be used as the target objects to be jointly analyzed. Thus, after triggering the joint - analysis requirement for the target objects on the intelligent interaction page, the large - model can perform a joint analysis on the target objects based on the joint - analysis requirement and display the data - analysis result for the joint - analysis requirement on the intelligent interaction page.

[0087] It should be understood that in the related art, many data - analysis tools are constructed based on the concept of a single indicator group, that is: they can only perform overall - dimension drilling or overall analysis on the entire indicator group. However, in actual business scenarios, the user's data - analysis requirements are often more complex and diverse. It may be necessary to analyze by combining data indicators in different indicator groups, or to analyze by combining different indicator groups. For example, since different businesses may have different key indicators and influencing factors, and the launch of a new business may affect other businesses, the user needs to pay attention to specific indicators in different indicator groups of different businesses to comprehensively evaluate the operation status and potential impact of the business. In this embodiment, multiple objects to be analyzed can be selected on the intelligent interaction page through a shortcut key, and the joint - analysis requirement for the multiple objects to be analyzed can be input on the intelligent interaction page, and the large - model can automatically analyze the joint - analysis requirement for the multiple objects to be analyzed to obtain the data - analysis result for the joint - analysis requirement. Thus, joint analysis of different data indicators or data - indicator groups can be performed, thereby improving the flexibility of data analysis and the user experience.

[0088] In a possible way, in response to the data - analysis requirement triggered by the user on the intelligent interaction page, displaying the data - analysis result for the data - analysis requirement on the intelligent interaction page may include:

[0089] In response to a second data analysis requirement triggered by a user on an intelligent interaction page, an analysis task script for implementing the second data analysis requirement is displayed on the intelligent interaction page, where the second data analysis requirement includes multiple data analysis requirements with logical relationships; in response to a triggering operation by the user on the analysis task script, a data analysis result obtained based on the analysis task script is displayed on the intelligent interaction page.

[0090] In this embodiment, the triggering operation on the analysis task script may be: associating and displaying a target control for confirming or executing the analysis task script with the analysis task script. Thus, the triggering operation on the analysis task script can be achieved by performing a triggering operation on the target control. It may also be triggering the analysis task script through a preset voice command. For example, when the intelligent interaction page detects the voice command "confirm and execute", the triggering of the analysis task script is realized. Of course, the triggering operation on the analysis task script may also be other, and the embodiments of the present disclosure do not impose any restrictions on this.

[0091] Exemplarily, as Figure 7 shown, when the user inputs the second data analysis requirement "I want to see the performance of these metrics...." on the intelligent interaction page, the large model can automatically decompose "I want to see the performance of these metrics...." into an analysis task script for implementing "I want to see the performance of these metrics...." according to the analysis granularity, and display the analysis task script and the target control "confirm and start execution" for executing the analysis task script on the intelligent interaction page. When "confirm and start execution" is triggered through operations such as clicking, the data analysis result obtained based on the analysis task script can be displayed on the intelligent interaction page.

[0092] In this embodiment, since the second data analysis requirement includes multiple data analysis requirements, thus, the data analysis result may include an overall data analysis result, or a data analysis result may be generated for each data analysis requirement. The embodiments of the present disclosure do not impose any restrictions on this. When a data analysis result is generated for each data analysis requirement, each data analysis result can be displayed in a card style to facilitate the user to switch and view the data analysis results corresponding to each data analysis requirement.

[0093] Through the above method, the deep understanding ability of the large model for natural language can be utilized to automatically decompose the second data analysis requirement including multiple data analysis requirements into an analysis task script for implementing the second data analysis requirement. Thus, by executing the analysis task script once, the data analysis results of multiple data analysis requirements can be obtained. Compared with the related art, when multiple data analysis requirements with logical relationships need to be executed, it is necessary to perform sequential analysis on the data analysis requirements based on the logical order. This solution can simplify the analysis operation process and improve the analysis efficiency.

[0094] In a possible way, after the user triggers a data analysis requirement on the intelligent interaction page, in order to enable the user to clarify the analysis process, the large model can identify the intention of the data analysis requirement, and then automatically call the corresponding intelligent analysis tool for data analysis based on the identified intention. During the data analysis process, the name of the intelligent analysis tool currently performing the data analysis and / or the analysis process are displayed, thereby enhancing the credibility of the data analysis results.

[0095] In a possible way, in response to the user's trigger operation on the analysis task script, display the data analysis results obtained based on the analysis task script on the intelligent interaction page, which may include:

[0096] In response to the user's editing operation on the analysis task script, determine the target analysis task script according to the editing operation; in response to the user's trigger operation on the target analysis task script, display the data analysis results obtained from the target analysis task script on the intelligent interaction page.

[0097] In this embodiment, the editing operation on the analysis task script can be determined according to the actual situation, and the embodiments of the present disclosure do not impose any restrictions on this. Exemplarily, as Figure 7 shown, the editing operation on the analysis task script can be: adding, modifying, or deleting execution conditions, adding, modifying, or deleting analysis content, and adding, modifying, or deleting analysis metrics, etc.

[0098] Through the above method, the analysis task script generated by the large model can be edited, thereby enabling the user to make targeted modifications to the analysis task script according to the actual situation, so as to achieve personalized data analysis and further improve the user experience.

[0099] In a possible way, the visualization A / B test data analysis method based on the large model may further include:

[0100] Display a content reference control on the intelligent interaction page, where the content reference control is used for the user to reference the historical session content displayed on the intelligent interaction page;

[0101] Correspondingly, in response to the data analysis requirement triggered by the user on the intelligent interaction page, display the data analysis results for the data analysis requirement on the intelligent interaction page, which may include:

[0102] In response to the user's trigger operation on the content reference control, determine the target historical session content to be referenced in the historical session content; in response to the data analysis requirement triggered by the user on the intelligent interaction page, based on the target historical session content, display the data analysis results for the data analysis requirement on the intelligent interaction page.

[0103] In this embodiment, the content reference control displayed on the intelligent interaction page may be: associating and displaying a content reference control for each piece of historical session content displayed on the intelligent interaction page, or displaying a content reference control on the intelligent interaction page. Of course, it may also be other, and the embodiments of the present disclosure do not impose any restrictions on this. When a content reference control is associated and displayed for each piece of historical session content, in response to the user's triggering operation on the content reference control, the target historical session content to be referenced determined from the historical session content may be: determining the historical session content associated with the triggered content reference control as the target historical session content. When a content reference control is displayed on the intelligent interaction page, in response to the user's triggering operation on the content reference control, the target historical session content to be referenced determined from the historical session content may be: after triggering the content reference control, a selection control for selecting historical session content may be displayed. After selecting historical session content through the selection control, the selected historical session content is determined as the target historical session content. Or, it may be to automatically reference the previous historical session content. Of course, it may also be other, and the embodiments of the present disclosure do not impose any restrictions on this.

[0104] Exemplarily, as Figure 8 shown, a content reference control "Reference" may be associated and displayed for each piece of historical session content on the intelligent interaction page. When the reference control "Reference" associated with the historical session content "Under 'Axx...', data metric 1 and not..." is triggered by an operation such as clicking, "Under 'Axx...', data metric 1 and not..." can be displayed in the content input area of the intelligent interaction page, enabling the user to intuitively understand the referenced historical session content based on the summary content. Subsequently, when the user triggers a data analysis requirement on the intelligent interaction page, a large model can be used to generate a data analysis result for the data analysis requirement based on the target historical session content and display it on the intelligent interaction page.

[0105] Through the above method, a content reference control can be displayed on the intelligent interaction page. Thus, when performing data analysis, data analysis can be carried out in combination with the reference content and the data analysis requirement, further improving the accuracy of data analysis. Especially when there are data analysis results for different data metrics on the intelligent interaction page, through content reference, the current data analysis can pay more attention to the referenced data metrics or data analysis results, further improving the accuracy of data analysis.

[0106] In a possible way, the visualization A / B test data analysis method based on a large model may further include:

[0107] An interaction link graph is displayed on the intelligent interaction page, where the interaction link graph is determined based on the reference relationships between different session contents in the intelligent interaction page; in response to a trigger operation on the target session content in the interaction link graph, the target session content is located and displayed on the intelligent interaction page.

[0108] In this embodiment, the interaction link graph refers to a visual structure constructed based on the analysis of the reference relationships between session contents and used to reflect the reference paths between session contents. For example, if there are 6 session contents displayed on the intelligent interaction page, denoted as session content 1, session content 2, session content 3, session content 4, session content 5, and session content 6 respectively, where session content 5 references session content 4, and session content 4 references session content 2; session content 3 references session content 2, and session content 2 references session content 1; session content 6 does not have content references. Then the intelligent interaction page can display three interaction link graphs, denoted as interaction link graph 1, interaction link graph 2, and interaction link graph 3, where interaction link graph 1 can be expressed as: session content 2 → session content 4 → session content 5, interaction link graph 2 can be expressed as: session content 1 → session content 2 → session content 3, and interaction link graph 3 can be expressed as: session content 6.

[0109] In this embodiment, displaying the interaction link graph on the intelligent interaction page can be: only displaying the name of the interaction link graph, and when the name of the interaction link graph is triggered, the session contents in the interaction link graph are displayed in the reference order, as Figure 9 shown. It can also be directly displaying the session contents in each interaction link graph in the reference order on the intelligent interaction page. Of course, it can also be other ways, and the embodiments of the present disclosure do not impose any restrictions on this. But no matter which display method is used, as long as the session content in the interaction link graph is triggered, the triggered session content can be located and displayed on the intelligent interaction page.

[0110] Through the above method, an interaction link graph determined based on the reference relationships between different session contents in the intelligent interaction page can be displayed on the intelligent interaction page. Thus, the interaction link graph can assist users in sorting out and analyzing ideas in multi-round conversations, further improving the analysis efficiency and user experience. In addition, since when the session content in the interaction link graph is triggered, the triggered session content can be located and displayed on the intelligent interaction page, it is possible to quickly jump to the corresponding session content without manually searching through a large number of session records, further improving the user experience.

[0111] In a possible way, the intelligent interaction page includes a full-screen display style and a floating window display style. The intelligent interaction page in the full-screen display style provides a data table display function. Correspondingly, the visualization A / B test data analysis method based on a large model can further include:

[0112] When the intelligent interaction page is displayed in full - screen display style and a data table is displayed on the intelligent interaction page, in response to a style switching operation on the intelligent interaction page, the intelligent interaction page is displayed in floating - window display style, and the data table is updated and displayed as text content, where the text content is used to describe the table content of the data table.

[0113] In this embodiment, the style switching operation on the intelligent interaction page can be to input a shortcut key in an external device to switch the style of the intelligent interaction page. For example, it can be to input the shortcut key "Ctrl + D" on the keyboard to switch the style of the intelligent interaction page, or a style switching control can be displayed on the intelligent interaction page. Thus, by triggering the style switching control, the style of the intelligent interaction page can be switched. Of course, the style switching operation on the intelligent interaction page can also be other operations, and the embodiments of the present disclosure do not impose any restrictions on this.

[0114] Exemplarily, as Figure 10 shown, the intelligent interaction page is displayed in full - screen display style, and a data table is displayed in the middle of the intelligent interaction page, text content is displayed below the data table, and a style switching control "Style Switching" is displayed in the upper - right corner of the intelligent interaction page. Thus, when "Style Switching" is triggered by an operation such as clicking, the intelligent interaction page can be displayed in floating - window display style, and the display of the data table can be cancelled.

[0115] In a possible way, in order to be able to quickly view the data table in the intelligent interaction page, when the display content in the intelligent interaction page includes a data table and the intelligent interaction page is displayed in floating - window display style, a quick switching instruction for switching the intelligent interaction page from floating - window display style to full - screen display style can be displayed in the intelligent interaction page. For example, a quick switching instruction "View Data Table in Full - Screen Display Mode" can be displayed in the intelligent interaction page. As Figure 10 shown, thus, by triggering the quick switching instruction "View Data Table in Full - Screen Display Mode", the intelligent interaction page can be quickly switched from floating - window display style to full - screen display style, thereby further improving the user experience.

[0116] It should be understood that when the intelligent interaction page is displayed in floating - window display style, the display window of the intelligent interaction page is smaller than the full - screen display style. Thus, in order to display the data table completely in the intelligent interaction page, there is a problem that the table content is small and the readability of the table content is poor. Thus, in this embodiment, when the intelligent interaction page is switched from full - screen display style to floating - window display style, the data table in the intelligent interaction page is updated and displayed as text content, thereby increasing the readability of the display content in the intelligent interaction page and further improving the user experience.

[0117] Among possible ways, the data analysis results include data metrics and / or data values associated with the A / B test data metric information. Correspondingly, the visualization A / B test data analysis method based on the large model may further include:

[0118] Displaying the data metrics in a first display style, and / or displaying the data values in a second display style, where the first display style is different from the second display style.

[0119] In this embodiment, the first display style and the second display style can be determined according to the actual situation, and the embodiments of the present disclosure do not impose any restrictions on this. Exemplarily, the first display style may be that the data metrics are displayed in a label style or the data metrics are highlighted, and the second display style may be that the data values are displayed in a preset display color.

[0120] In the above manner, the data metrics in the data analysis results can be displayed in the first display style, and / or the data values in the data analysis results can be displayed in the second display style. Thus, the data metrics and / or data values associated with the data metric information can be quickly identified in the data analysis results, improving the readability of the data analysis results.

[0121] Among possible ways, the visualization A / B test data analysis method based on the large model may further include:

[0122] In response to a trigger operation on a data metric, displaying the data metric information of the data metric, or in response to a trigger operation on a data value, displaying the data details information of the data value.

[0123] In this embodiment, the data metric information of the data metrics and the data details information of the data values can be determined according to the actual situation, and the embodiments of the present disclosure do not impose any restrictions on this. Exemplarily, as Figure 11 shown, when the data metric "Data Metric 1" is triggered by an operation such as clicking, a display panel can be displayed, and the data metric information such as the data metric group and metric type corresponding to "Data Metric 1" can be displayed in the display panel. Or, as Figure 12 shown, when the data value "-2.27%" is triggered by an operation such as clicking, a display panel can be displayed, and the data details information such as the change trend, metric group, and metric type corresponding to "-2.27%" can be displayed in the display panel.

[0124] In the above - mentioned manner, when a data metric is triggered, the data metric information corresponding to the data metric can be displayed. Alternatively, when a data value is triggered, the data details information corresponding to the data value can be displayed. Thus, the data metric information corresponding to the data metric or the data details information corresponding to the data value can be quickly viewed, further improving the user experience. Additionally, compared with directly displaying the data metric information corresponding to the data metric or the data details information corresponding to the data value in the data analysis result, in the above - mentioned manner, on the one hand, it can avoid displaying too much information at once and reduce the user's reading interest. On the other hand, it enables the user to independently choose whether to view the data metric information corresponding to the data metric or the data details information corresponding to the data value, further enhancing the user experience.

[0125] Based on the same concept, an embodiment of the present disclosure also provides a visualization A / B test data analysis device based on a large model, as Figure 13 shown. The visualization A / B test data analysis device 1300 based on the large model may include:

[0126] A first display module 1301, configured to display the A / B test data metric information of the object to be analyzed under different analysis dimensions on the data analysis page, and associate and display an analysis function entry for each analysis dimension, where the analysis function entry is used to trigger the large model to analyze based on the A / B test data metric information of the object to be analyzed;

[0127] A second display module 1302, configured to display the data analysis result of the A / B test data metric information under the target analysis dimension in response to a trigger operation on the analysis function entry associated with the target analysis dimension.

[0128] Through the above - mentioned visualization A / B test data analysis device 1300 based on the large model, the A / B test data metric information of the object to be analyzed and the analysis function entry for triggering the large model to analyze the A / B test data metric information can be displayed on the data analysis page. Thus, by triggering the analysis function entry, automatic analysis of the A / B test data metric information can be achieved. On the one hand, it can simplify the analysis process and improve the analysis efficiency. On the other hand, since the data analysis result is obtained by the large model analyzing the A / B test data metric information, the analysis accuracy can be improved. Additionally, since an analysis function entry is associated and displayed for each analysis dimension, thus, the analysis function entry associated with the corresponding analysis dimension can be triggered based on the user's needs, so as to realize targeted analysis of the A / B test data metric information, and further improve the user experience.

[0129] In a possible manner, the second display module 1302 can be used to display at least one of the first data analysis result, the second data analysis result, and the third data analysis result. Among them, the first data analysis result is used to indicate the change trend of the A / B test data index information in the target analysis dimension, the second data analysis result is used to indicate the reason for the change trend of the A / B test data index information in the target analysis dimension, and the third data analysis result is used to indicate the analysis direction when the A / B test data index information in the target analysis dimension shows a change trend.

[0130] In a possible manner, the visualization A / B test data analysis device 1300 based on a large model may further include:

[0131] A third display module, configured to display an information query control and / or an analysis function trigger control associated with the target analysis dimension when displaying the data analysis result of the A / B test data index information in the target analysis dimension. Among them, the information query control is used to trigger the large model to query the A / B test data index information in the target analysis dimension, and the analysis function trigger control is used to trigger the large model to analyze the A / B test data index information in the target analysis dimension based on the data analysis result.

[0132] In a possible manner, the second display module 1302 may include:

[0133] A first display unit, configured to display an intelligent interaction page in response to a trigger operation on an analysis function entry associated with the target analysis dimension, and display the data analysis result of the A / B test data index information in the target analysis dimension on the intelligent interaction page. Among them, the intelligent interaction page is used for the user to have a conversation with an intelligent agent associated with the large model;

[0134] Correspondingly, the visualization A / B test data analysis device 1300 based on a large model may further include:

[0135] A fourth display module, configured to display the data analysis result for the data analysis requirement on the intelligent interaction page in response to the data analysis requirement triggered by the user on the intelligent interaction page.

[0136] In a possible manner, the fourth display module may include:

[0137] A second display unit, configured to display candidate analysis objects in response to a quick instruction triggered by the user on the intelligent interaction page;

[0138] A third display unit, configured to determine the candidate analysis object corresponding to the selection operation in response to a selection operation on the candidate analysis objects;

[0139] The fourth display unit is configured to respond to a first data analysis requirement for a candidate analysis object corresponding to a selection operation input by a user on the intelligent interaction page, and display a data analysis result for the first data analysis requirement on the intelligent interaction page.

[0140] In a possible manner, the fourth display module may include:

[0141] The fifth display unit is configured to respond to a second data analysis requirement triggered by a user on the intelligent interaction page, and display an analysis task script for implementing the second data analysis requirement on the intelligent interaction page, where the second data analysis requirement includes multiple data analysis requirements with logical relationships;

[0142] The sixth display unit is configured to respond to a trigger operation on the analysis task script by a user, and display a data analysis result obtained based on the analysis task script on the intelligent interaction page.

[0143] In a possible manner, the sixth display unit may include:

[0144] The first display subunit is configured to respond to an editing operation on the analysis task script by a user, and determine a target analysis task script according to the editing operation;

[0145] The second display subunit is configured to respond to a trigger operation on the target analysis task script by a user, and display a data analysis result obtained from the target analysis task script on the intelligent interaction page.

[0146] In a possible manner, the visualization A / B test data analysis device 1300 based on a large model may further include:

[0147] The fifth display module is configured to display a content reference control on the intelligent interaction page, where the content reference control is used for a user to reference historical session content displayed on the intelligent interaction page;

[0148] Correspondingly, the fourth display module may include:

[0149] The seventh display unit is configured to respond to a trigger operation on the content reference control by a user, and determine target historical session content to be referenced in the historical session content;

[0150] The eighth display unit is configured to respond to a data analysis requirement triggered by a user on the intelligent interaction page, and display a data analysis result for the data analysis requirement on the intelligent interaction page based on the target historical session content.

[0151] In a possible manner, the visualization A / B test data analysis device 1300 based on a large model may further include:

[0152] The sixth display module is used to display an interaction link graph on the intelligent interaction page, where the interaction link graph is determined based on the reference relationship between different session contents on the intelligent interaction page;

[0153] The seventh display module is used to, in response to a triggering operation on the target session content in the interaction link graph, locate and display the target session content on the intelligent interaction page.

[0154] In a possible manner, the intelligent interaction page includes a full-screen display style and a floating window display style. The intelligent interaction page in the full-screen display style provides a data table display function. Correspondingly, the visualization A / B test data analysis device 1300 based on a large model may further include:

[0155] The eighth display module is used to, when the intelligent interaction page is displayed in the full-screen display style and a data table is displayed on the intelligent interaction page, in response to a style switching operation on the intelligent interaction page, display the intelligent interaction page in the floating window display style and cancel the display of the data table.

[0156] In a possible manner, the data analysis result includes data metrics and / or data values associated with the A / B test data metric information. Correspondingly, the visualization A / B test data analysis device 1300 based on a large model may further include:

[0157] The ninth display module is used to display the data metrics according to the first display style and / or display the data values according to the second display style, where the first display style is different from the second display style.

[0158] In a possible manner, the visualization A / B test data analysis device 1300 based on a large model may further include:

[0159] The tenth display module is used to, in response to a triggering operation on the data metrics, display the data metric information of the data metrics, or, in response to a triggering operation on the data values, display the data detail information of the data values.

[0160] In a possible manner, the second display module 1302 may be used to display the A / B test data metric information of the object to be analyzed in at least one dimension of the experimental group dimension, the metric group dimension, the single metric dimension, and the combination dimension on the data analysis page, where the combination dimension is determined according to the experimental group dimension and the single metric dimension.

[0161] Based on the same concept, an embodiment of the present disclosure also provides a computer-readable medium, on which a computer program is stored, and when the program is executed by a processing device, the steps of any of the above visualization A / B test data analysis methods based on a large model are implemented.

[0162] Based on the same concept, an embodiment of the present disclosure also provides an electronic device, which may include:

[0163] A storage device on which a computer program is stored;

[0164] A processing device for executing the computer program in the storage device to implement the steps of any of the above-mentioned large model-based visual A / B test data analysis methods.

[0165] Based on the same concept, an embodiment of the present disclosure also provides a computer program product, including a computer program, which when executed by a processor, implements the steps of any of the above-mentioned large model-based visual A / B test data analysis methods.

[0166] Next, refer to Figure 14 , which shows a schematic structural diagram of an electronic device 1400 suitable for implementing the embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 14 The electronic device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present disclosure.

[0167] As Figure 14 shown, the electronic device 1400 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 1401, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1402 or the program loaded from the storage device 1408 into the random access memory (RAM) 1403. In the RAM 1403, various programs and data required for the operation of the electronic device 1400 are also stored. The processing device 1401, the ROM 1402, and the RAM 1403 are connected to each other through a bus 1404. The input / output (I / O) interface 1405 is also connected to the bus 1404.

[0168] Generally, the following devices may be connected to the I / O interface 1405: an input device 1406 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1409. The communication device 1409 can allow the electronic device 1400 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 14An electronic device 1400 with various devices is shown, but it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices may be implemented or had.

[0169] In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present disclosure includes a computer program product that includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through the communication device 1409, or installed from the storage device 1408, or installed from the ROM 1402. When the computer program is executed by the processing device 1401, the above functions defined in the methods of the embodiments of the present disclosure are executed.

[0170] It should be noted that the above computer-readable medium in the present disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which the computer-readable program code is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable signal medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0171] In some embodiments, any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol) can be used for communication, and it can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks (“LAN”), wide area networks (“WAN”), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed network.

[0172] The above computer-readable medium can be included in the above electronic device; it can also exist separately without being assembled into the electronic device.

[0173] The above computer-readable medium carries one or more programs. When the above one or more programs are executed by the electronic device, the electronic device is caused to: display, on a data analysis page, A / B test data metric information of an object to be analyzed under different analysis dimensions, and associate and display an analysis function entry for each analysis dimension, where the analysis function entry is used to trigger the large model to analyze based on the A / B test data metric information of the object to be analyzed; in response to a trigger operation on the analysis function entry associated with a target analysis dimension, display the data analysis result of the A / B test data metric information under the target analysis dimension.

[0174] Computer program code for performing the operations of the present disclosure can be written in one or more programming languages or combinations thereof. The above programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the “C” language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., by using an Internet service provider to connect through the Internet).

[0175] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or by a combination of dedicated hardware and computer instructions.

[0176] The modules described in the embodiments of the present disclosure can be implemented in software or in hardware. In some cases, the name of a module does not constitute a limitation on the module itself.

[0177] The functions described above in this document can be performed at least in part by one or more hardware logic components. By way of example and not limitation, the types of hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0178] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0179] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

[0180] In addition, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order. In certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments may also be implemented combinatorially in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0181] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms for implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be elaborated herein.

Claims

1. A visualization A / B test data analysis method based on a large model, characterized in that, The visualization A / B test data analysis method based on a large model includes: Displaying the A / B test data metric information of the object to be analyzed under different analysis dimensions on the data analysis page, and associating and displaying an analysis function entry for each of the analysis dimensions, where the analysis function entry is used to trigger the large model to perform analysis based on the A / B test data metric information of the object to be analyzed; In response to a trigger operation on the analysis function entry associated with the target analysis dimension, displaying the data analysis result of the A / B test data metric information under the target analysis dimension.

2. The visualization A / B test data analysis method based on a large model according to claim 1, wherein The displaying the data analysis result of the A / B test data metric information under the target analysis dimension includes: Displaying at least one of a first data analysis result, a second data analysis result, and a third data analysis result, where the first data analysis result is used to indicate the change trend of the A / B test data metric information under the target analysis dimension, the second data analysis result is used to indicate the reason for the change trend of the A / B test data metric information under the target analysis dimension, and the third data analysis result is used to indicate the analysis direction when the A / B test data metric information under the target analysis dimension shows a change trend.

3. The visualization A / B test data analysis method based on a large model according to claim 1, characterized in that The visualization A / B test data analysis method based on a large model further includes: When displaying the data analysis result of the A / B test data metric information under the target analysis dimension, displaying an information query control and / or an analysis function trigger control associated with the target analysis dimension, where the information query control is used to trigger the large model to query the A / B test data metric information under the target analysis dimension, and the analysis function trigger control is used to trigger the large model to perform analysis on the A / B test data metric information under the target analysis dimension based on the data analysis result.

4. The visualization A / B test data analysis method based on a large model according to any one of claims 1-3, characterized in that, The in response to a trigger operation on the analysis function entry associated with the target analysis dimension, displaying the data analysis result of the A / B test data metric information under the target analysis dimension includes: In response to a trigger operation on the analysis function entry associated with the target analysis dimension, displaying an intelligent interaction page, and displaying the data analysis result of the A / B test data metric information under the target analysis dimension on the intelligent interaction page, where the intelligent interaction page is used for the user to have a conversation with an intelligent agent associated with the large model; The visualization A / B test data analysis method based on a large model further includes: In response to the data analysis requirement triggered by the user on the intelligent interaction page, displaying the data analysis result for the data analysis requirement on the intelligent interaction page.

5. The visualization A / B test data analysis method based on a large model according to claim 4, wherein, The in response to the data analysis requirement triggered by the user on the intelligent interaction page, displaying the data analysis result for the data analysis requirement on the intelligent interaction page includes: In response to a quick command triggered by the user on the intelligent interaction page, displaying candidate analysis objects; In response to a selection operation among the candidate analysis objects, determining the candidate analysis object corresponding to the selection operation; In response to the first data analysis requirement of the candidate analysis object corresponding to the selection operation input by the user on the intelligent interaction page, display the data analysis result for the first data analysis requirement on the intelligent interaction page.

6. The visualization A / B test data analysis method based on a large model according to claim 4, wherein The displaying, on the intelligent interaction page, the data analysis result for the data analysis requirement in response to the data analysis requirement triggered by the user on the intelligent interaction page includes: In response to the second data analysis requirement triggered by the user on the intelligent interaction page, display the analysis task script for implementing the second data analysis requirement on the intelligent interaction page, where the second data analysis requirement includes multiple data analysis requirements with logical relationships; In response to the triggering operation of the user on the analysis task script, display the data analysis result obtained based on the analysis task script on the intelligent interaction page.

7. The visualization A / B test data analysis method based on a large model according to claim 6, wherein The displaying, on the intelligent interaction page, the data analysis result obtained based on the analysis task script in response to the triggering operation of the user on the analysis task script includes: In response to the editing operation of the user on the analysis task script, determine the target analysis task script according to the editing operation; In response to the triggering operation of the user on the target analysis task script, display the data analysis result obtained from the target analysis task script on the intelligent interaction page.

8. The visualization A / B test data analysis method based on a large model according to claim 4, wherein The visualization A / B test data analysis method based on a large model further includes: Display a content reference control on the intelligent interaction page, where the content reference control is used for the user to reference the historical session content displayed on the intelligent interaction page; The displaying, on the intelligent interaction page, the data analysis result for the data analysis requirement in response to the data analysis requirement triggered by the user on the intelligent interaction page includes: In response to the triggering operation of the user on the content reference control, determine the target historical session content to be referenced in the historical session content; In response to the data analysis requirement triggered by the user on the intelligent interaction page, based on the target historical session content, display the data analysis result for the data analysis requirement on the intelligent interaction page.

9. The method for visualizing A / B test data analysis based on a large model according to claim 8, wherein, The visualization A / B test data analysis method based on a large model further includes: Display an interaction link map on the intelligent interaction page, where the interaction link map is determined based on the reference relationship between different session contents on the intelligent interaction page; In response to the triggering operation on the target session content in the interaction link map, locate and display the target session content on the intelligent interaction page.

10. The visualization A / B test data analysis method based on a large model according to claim 4, wherein The intelligent interaction page includes a full-screen display style and a floating window display style. The intelligent interaction page in the full-screen display style provides a data table display function. The visualization A / B test data analysis method based on a large model further includes: When the intelligent interaction page is displayed in the full-screen display style and a data table is displayed on the intelligent interaction page, in response to the style switching operation of the intelligent interaction page, display the intelligent interaction page in the floating window display style and cancel the display of the data table.

11. The method for visualizing A / B test data analysis based on a large model according to any one of claims 1-3, characterized in that, The data analysis results include data metrics and / or data values associated with the A / B test data metric information, and the visualization A / B test data analysis method based on a large model further includes: Displaying the data metrics in a first display style, and / or displaying the data values in a second display style, wherein the first display style is different from the second display style.

12. The method for visualizing A / B test data analysis based on a large model according to claim 11, wherein, The visualization A / B test data analysis method based on a large model further includes: In response to a trigger operation on the data metrics, displaying the data metric information of the data metrics, or in response to a trigger operation on the data values, displaying the data detail information of the data values.

13. The visualization A / B test data analysis method based on a large model according to any one of claims 1-3, characterized in that, The displaying of the A / B test data metric information of the object to be analyzed in different analysis dimensions on the data analysis page includes: Displaying the A / B test data metric information of the object to be analyzed in at least one of the experimental group dimension, the metric group dimension, the single metric dimension, and the combined dimension on the data analysis page, wherein the combined dimension is determined according to the experimental group dimension and the single metric dimension.

14. A visualization A / B test data analysis device based on a large model, characterized in that, The visualization A / B test data analysis device based on a large model includes: A first display module, configured to display the A / B test data metric information of the object to be analyzed in different analysis dimensions on the data analysis page, and associatively display an analysis function entry for each of the analysis dimensions, wherein the analysis function entry is used to trigger the large model to perform analysis based on the A / B test data metric information of the object to be analyzed; A second display module, configured to display the data analysis results of the A / B test data metric information in the target analysis dimension in response to a trigger operation on the analysis function entry associated with the target analysis dimension.

15. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processing device, it implements the steps of the method according to any one of claims 1-13.

16. An electronic device, characterized in that, Including: A storage device, on which a computer program is stored; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1-13.

17. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-13.

Images