Information determination method and device, electronic equipment, storage medium and product

Abstract

The embodiment of the invention provides an information determination method and device, electronic equipment, a storage medium and a product. The information determination method comprises the steps that a configuration information set of an application program is determined, the configuration information set comprises multiple pieces of configuration information, and the configuration information indicates configuration which is transmitted by a server for the application program and is applied by the application program; determining a function attribute of a degradation function in the application program; and based on the function attribute, screening the configuration information set for at least two times, and taking the configuration information set screened each time as input of next screening to obtain configuration information causing degradation of the degradation function. According to the information determination method, automatic attribution of degradation of the degradation function is realized.

Description

Information determination methods, devices, electronic equipment, storage media and products Technical Field

[0001] This disclosure relates to computer technology, and more particularly to information determination methods, apparatus, electronic devices, storage media, and products. Background Technology

[0002] Application performance degradation refers to the decline in performance that occurs during the operation of an application. Attributing performance degradation involves analyzing the causes of problems or defects in the software.

[0003] Currently, attributing performance degradation still requires manual investigation of the causes, resulting in low attribution efficiency. Summary of the Invention

[0004] This disclosure provides information determination methods, apparatus, electronic devices, storage media, and products for automatically attributing degradation of degradation functions.

[0005] In a first aspect, embodiments of this disclosure provide an information determination method, including:

[0006] Determine the configuration information set of the application, the configuration information set including multiple configuration information, the configuration information indicating the configuration transmitted by the server for the application to use;

[0007] Determine the function properties of the degraded functions in the application;

[0008] Based on the function attributes, the configuration information set is filtered at least twice, and the configuration information set after each filtering is used as the input for the next filtering to obtain the configuration information that causes the degradation function to degrade.

[0009] Secondly, embodiments of this disclosure also provide an information determining device, comprising:

[0010] The first determining module is used to determine the configuration information set of the application, the configuration information set including multiple configuration information, the configuration information indicating the configuration transmitted by the server for the application to use;

[0011] The second determining module is used to determine the function attributes of the degraded functions in the application;

[0012] The filtering module is used to filter the configuration information set at least twice based on the function attributes, and the configuration information set after each filtering is used as the input for the next filtering to obtain the configuration information that causes the degradation function to degrade.

[0013] Thirdly, embodiments of this disclosure also provide an electronic device, characterized in that the electronic device comprises:

[0014] One or more processing devices;

[0015] Storage device for storing one or more programs.

[0016] When the one or more programs are executed by the one or more processing devices, the one or more processing devices implement the information determination method provided in the embodiments of this disclosure.

[0017] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the information determination method provided in embodiments of this disclosure.

[0018] Fifthly, embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements the information determination method provided in embodiments of this disclosure.

[0019] In this embodiment of the disclosure, by filtering the configuration information set multiple times based on the function attributes of the degradation function, the range of configuration information that causes the degradation function to deteriorate can be gradually narrowed down, and the configuration information that causes the degradation function to deteriorate can be located, thus realizing automatic attribution of the degradation function degradation. Attached Figure Description

[0020] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0021] Figure 1 is a flowchart illustrating an information determination method provided in an embodiment of this disclosure;

[0022] Figure 2 is a graph showing the startup time according to an embodiment of this disclosure;

[0023] Figure 3 is a flowchart illustrating another information determination method provided in an embodiment of this disclosure;

[0024] Figure 4 is a schematic diagram of the screening execution flow provided in an embodiment of this disclosure;

[0025] Figure 5 is a schematic diagram of a relationship diagram provided in an embodiment of this disclosure;

[0026] Figure 6 is a schematic diagram of a scenario for determining distribution information provided in an embodiment of this disclosure;

[0027] Figure 7 is a schematic diagram of another scenario for determining distribution information provided by an embodiment of this disclosure;

[0028] Figure 8 is a schematic diagram of an information determination device provided in an embodiment of this disclosure;

[0029] Figure 9 is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0030] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0031] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0032] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0033] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0034] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0035] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0036] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0037] The current accurate attribution capability is limited to degradation caused by direct code changes, that is, when the method source code has changed and the degradation point corresponds to the modification point in the source code. In this case, mapping can be done directly. However, in most scenarios, degradation points and modification points cannot be mapped, which requires a lot of manpower to manually investigate the root cause of degradation.

[0038] Currently, identifying and addressing degradation issues still requires significant manpower to pinpoint the specific causes, leading to longer fix times and potentially delaying solutions until the next version. In severe cases, this can even cause degradation issues to persist in production. Therefore, this disclosure provides an A / B testing approach to attributing performance degradation in applications (such as Android applications) to more accurately and systematically pinpoint time-consuming degradation.

[0039] Application configuration involves adjusting the functionality and logic of an application to achieve different performance levels for different configurations. Configuration can include A / B testing and setting configurations (such as software settings options). A / B testing involves randomly dividing clients into two or more groups, displaying different software versions or feature settings to each group, and then comparing the performance of each group to determine which version or setting is superior.

[0040] Configuration changes primarily affect online users. Server-side changes typically have network-related impacts; for example, changes to Content Delivery Network (CDN) configurations can cause fluctuations in network requests. This disclosure can detect server-side configuration changes (such as server-side configurations for applications). Configuration changes generally manifest as an increase in the negative penetration rate of application metrics, leading to a deterioration in overall method latency. Overall latency can be considered as the total metric for all users of the application.

[0041] This disclosure uses the A / B test configuration in the server-side configuration as an example for description. The degradation caused by the A / B test and the degradation point (such as a degradation function, which can be considered a function with performance degradation. For example, an increase in the execution time of a function can be considered a degradation function. The increase in the execution time of a function can be an increase in the execution time of a single execution or an increase in execution time after multiple calls) are indirectly related. Therefore, this disclosure provides a scheme to associate the degradation function with the relevant A / B test.

[0042] Degradation attribution involves identifying the change that caused the degradation from among multiple changes. Specifically, this is done by filtering out irrelevant configuration changes layer by layer. If only one configuration change remains, it is highly likely to be the cause of the degradation. If multiple configuration changes remain, manual intervention can be implemented, which improves troubleshooting efficiency and reduces labor costs compared to the past.

[0043] Figure 1 is a flowchart illustrating an information determination method provided in an embodiment of this disclosure. This embodiment is applicable to situations where the degradation of a degradation function is attributed (i.e., the cause of degradation is analyzed). This method can be executed by an information determination device, which can be implemented in software and / or hardware, or optionally by an electronic device, such as a mobile terminal, a PC, or a server.

[0044] As shown in Figure 1, the method includes:

[0045] S110. Determine the set of configuration information for the application.

[0046] An application can be a software program designed to perform a specific task. This disclosure tests the performance of the application. The configuration information set includes multiple configuration details indicating configurations transmitted by the server to the application for its use, such as in an A / B test.

[0047] This operation can determine all configuration information in an application to form a configuration information set, such as a collection of information corresponding to all functions and configurations required for the application to run. Configuration information can be descriptive, such as configuration version numbers. The full configuration information in the application can be the full configuration information at the time of degradation, or the full configuration information throughout the application's entire lifecycle.

[0048] In one embodiment, the configuration information includes information corresponding to the experiment configuration, which includes parameters set for executing the specified experiment. The configuration information can identify the experiment configuration. The experiment configuration can be an experiment sent from the server to the application, such as an A / B experiment.

[0049] S120. Determine the function attributes of the degraded functions in the application.

[0050] Function attributes can be considered as characteristics describing the degradation of a function. These include the degradation time, execution time, and device identifier of the device executing the function. Degradation time can be considered the time it takes for the degradation function to cause degradation. For example, when performing performance testing on an application, if a degradation metric is identified at a certain time and determined to be caused by a degradation function, then that time can be considered the degradation time. Metrics can be software performance metrics. Software performance metrics are measures of the software's runtime performance. Application performance metrics can be used to measure its performance during operation, such as efficiency, speed, and stability.

[0051] Figure 2 is a graph illustrating startup time according to an embodiment of this disclosure. Referring to Figure 2, using startup time as an example, the daily startup time of an application is determined. The daily startup time can be the average or median of multiple startup times for all users within a day; this is not limited here. By comparing the indicators, 0417, i.e., the startup time on April 17th, is equivalent to 0416, meaning the startup time on April 16th has deteriorated. The standard for deterioration is not limited here; however, if the difference in startup times exceeds a certain value, then the startup time on April 17th can be considered deteriorated. Further, a degradation function is identified within the application that caused the startup time degradation, and April 17th is used as the time when the degradation function occurs.

[0052] The execution time of the degradation function can be considered as the time the degradation function takes to execute during the application's runtime. The electronic device executing the configuration information can be the electronic device performing attribution. The device executing the degradation function can be the device running the application; the device identifier can be considered as information uniquely identifying the device.

[0053] This operation can determine different function attributes based on different filtering methods; no restrictions are placed on the function attributes here. Different function attributes correspond to different determination methods. For example, the execution time of the degradation function can be transmitted by the device executing the degradation function. The device identifier can be transmitted by the device executing the degradation function. The degradation time can be transmitted by the device analyzing the degradation function or degradation indicators. When analyzing degradation functions or degradation indicators in electronic devices, the degradation time can be determined locally.

[0054] S130. Based on the function attributes, the configuration information set is filtered at least twice, and the configuration information set after each filtering is used as the input for the next filtering to obtain the configuration information that causes the degradation function to degrade.

[0055] After determining the function attributes, this operation can filter the configuration information set multiple times based on the function attributes. The configuration information set after each filtering is used as the input for the next filtering, so that the configuration information set is filtered through multiple levels to obtain the configuration information that causes the deterioration of the deteriorating function.

[0056] Different function attributes correspond to different filtering methods. This operation can filter the configuration information set using different filtering methods each time, and finally locate the configuration information that causes the deterioration of the deteriorating function.

[0057] Figure 3 is a flowchart illustrating another information determination method provided in this embodiment. Referring to Figure 3, for a configuration information set (such as the set of configuration information corresponding to all configurations of an application), the set is first filtered using filtering method 1 based on function attributes to obtain a filtered configuration information set; then, it is filtered again using filtering method 2 based on function attributes to obtain a filtered configuration information set; finally, it is filtered again using filtering method 3 based on function attributes to obtain the configurations that cause the degradation function to degrade. The number of filtering iterations is not limited here. It can be a pre-set number of iterations or determined by the number of configuration information included in the filtered configuration information set. If the number of configuration information included in the filtered configuration information set is less than a set threshold, the filtering can be terminated, and the filtered configuration information can be determined as the configuration information that causes the degradation function to degrade. The set threshold can be 2.

[0058] The technical solution of this disclosure embodiment, by filtering the configuration information set multiple times based on the function attributes of the degradation function, can gradually narrow down the range of configuration information that causes the degradation function to deteriorate, locate the configuration information that causes the degradation function to deteriorate, and realize automatic attribution of degradation function degradation.

[0059] Based on the above embodiments, modified embodiments of the above embodiments are proposed. It should be noted that, in order to keep the description brief, only the differences from the above embodiments are described in the modified embodiments.

[0060] The filtering methods used in the process of filtering the configuration information set may include time-level change filtering, such as daily change filtering, distribution filtering of device identifiers of devices executing degradation functions (i.e., sample distribution filtering), and configuration impact surface filtering.

[0061] The execution order of the various screening methods is not limited here and can be any order.

[0062] Figure 4 is a schematic diagram of the execution flow of a filtering method provided in an embodiment of this disclosure. Referring to Figure 4, for a configuration information set, daily change filtering, sample distribution filtering, and configuration impact surface filtering can be performed sequentially. Daily change filtering can be considered as filtering configuration information in the configuration information set that has changed on a daily basis, thus identifying the changed configuration information. Sample distribution filtering can be considered as filtering the configuration information set based on the distribution of device identifiers of multiple devices executing the degradation function in the configuration information. Configuration impact surface filtering can be considered as filtering based on the scope of influence of the configuration. The scope of influence reflects the function affected by the configuration.

[0063] In one embodiment, the function attribute includes the degradation time at which the degraded function degrades, and correspondingly, filtering the configuration information set includes:

[0064] Determine the configuration information that was changed in the configuration information group within a set period of time before the degradation time;

[0065] The determined configuration information is selected as the filtered configuration information set.

[0066] The duration can be a preset duration. There is no limit to the value of the duration set here. It can be determined according to the actual situation, such as one day.

[0067] This embodiment uses a daily change filter to filter the configuration information set. First, it identifies configuration information that has changed within a set timeframe prior to the degradation function's degradation time. This identified configuration information is then used as the filtered configuration information set.

[0068] As shown in Figure 2, a degradation point appears on April 17th, thus identifying the degradation function. This allows us to determine the configuration changes made to the application between April 16th and April 17th, such as changes appearing in the application's configuration information set.

[0069] Configuration changes will lead to an increase in related degradation functions. In this embodiment, configuration changes corresponding to the occurrence time of degradation functions, i.e. degradation time, are filtered out to obtain a set of filtered configuration information.

[0070] In one embodiment, determining the configuration information that has changed within a set period of time prior to the degradation time includes:

[0071] Determine the configuration information of the application at the degradation time;

[0072] Determine the configuration information of the application at a target time, where the target time is earlier than the degradation time, and the time difference between the target time and the degradation time is the set duration;

[0073] Configuration information that has changed relative to the target time in the determined configuration information of the degradation time is identified as the configuration information that has changed in the configuration information set.

[0074] This embodiment details the operations for determining the changed configuration information. It determines the application's configuration information at two time points: a degradation time and a target time. By comparing the configuration information at these two time points, it identifies the changed configuration information within a set of settings. Configuration changes can include adding, deleting, and modifying configuration settings.

[0075] If the configuration information for the degradation time has changed compared to the configuration information for the target time, then this configuration information can be used as the changed configuration information, and the changed configuration information can be filtered to obtain the filtered configuration information set. Here, the degradation time can be April 17th, and the target time can be April 16th.

[0076] In one embodiment, the function attribute includes the execution time of the degraded function, and correspondingly, filtering the configuration information set includes:

[0077] Configuration information corresponding to the configuration associated with the execution of the degradation function prior to determining the execution time;

[0078] The determined configuration information is selected as the filtered configuration information set.

[0079] The filtering of the configuration information set in this embodiment can be considered as configuration impact surface filtering. The configuration associated with the execution of the degradation function can be considered as the configuration that may affect the execution of the degradation function, such as the configuration that the degradation function needs to read.

[0080] This embodiment filters configurations from the perspective of the correlation between functions and configurations. It determines whether a degraded function is included within the scope of influence of each configuration in the application. Alternatively, it can filter from a temporal perspective, identifying configuration information corresponding to changes read before the degraded function is executed, thus forming a filtered set of configuration information.

[0081] In one embodiment, the configuration information corresponding to the configuration associated with the execution of the degradation function before determining the execution time includes:

[0082] In the application's relationship graph, the modified configuration is searched forward towards the node corresponding to the degradation function. The relationship graph includes the timing relationship between function execution and configuration reading within the application.

[0083] The information corresponding to the found changed configuration is determined to be the configuration information corresponding to the configuration associated with the execution of the degradation function before the degradation time.

[0084] A relationship diagram can reflect the timing relationship between the execution of functions included in the application and the reading of configurations included in the application.

[0085] A relationship diagram can be time-varying, illustrating the execution sequence of functions and configuration reads within an application over time. Each executed function in the application has a corresponding node in the relationship diagram. Searching backwards towards the node corresponding to the degraded function can be considered as searching before the execution time of that degraded function. The process involves checking for changed configurations; if found, these configurations are selected as the filtered configurations. Filtering can continue until changed configurations are found, or until all configurations before the degraded time have been filtered.

[0086] Figure 5 is a schematic diagram of a relationship diagram provided in an embodiment of this disclosure. Referring to Figure 5, the relationship diagram shows methods C, D, G, H, I, and A included in the application. The methods are arranged from left to right according to their execution time, with methods C, D, and G executed sequentially. Method C calls method H. Method D calls methods I and A. The application reads configuration 1 earlier than configuration 2, and configuration 2 is read earlier than configuration 3. After the configuration is read, it will affect the subsequently executed methods. For example, after configuration 3 is read, its influence includes method G. The influence of configuration 2 includes methods D, I, A, and G. After configuration 1 is read, its influence includes all methods in Figure 5. The configuration only affects functions after the configuration is read; the longer the interval between reading the configuration and executing the function, the lower the probability that the function is affected by the configuration. An influence factor can be established to quantify the relationship between configuration and degraded functions. Functions closer to the configuration reading time are more affected.

[0087] This embodiment can filter the configuration information set by considering the configurations that may affect the execution of the degradation function. Referring to Figure 5, when the degradation function is method G, the configurations can be traversed backwards from the node corresponding to method G (i.e., left in Figure 5). First, configuration 3 is traversed to determine if it has been changed. If so, configuration 3 is selected. After selecting configuration 3, the filtering can continue forward or stop.

[0088] This embodiment allows filtering after the daily change filter. When determining whether a configuration has changed, it can be determined whether the configuration is in the configuration information set after the daily change filter.

[0089] In one embodiment, the function attribute includes a device identifier corresponding to the device executing the degradation function, and correspondingly, filtering the configuration information set includes:

[0090] Determine the distribution information of the device identifier within the configuration information included in the configuration information set;

[0091] Based on the distribution information, determine the number of device identifiers corresponding to each configuration information in the configuration information set;

[0092] The configuration information set is filtered to select configuration information whose corresponding quantity is greater than a set threshold.

[0093] Distribution information characterizes how device identifiers are distributed within configuration information. The distribution of device identifiers within configuration information can be considered as an analysis of the distribution of configurations possessed by the devices identified by those identifiers, determining whether there are clusterings in a particular configuration or a few configurations.

[0094] Determining the distribution information of the device identifier in the configuration information set can be considered as determining which configuration information in the configuration information set the device identified by the device identifier has its configuration distributed in.

[0095] Based on the distribution information, the number of device identifiers corresponding to each configuration information in the configuration information set is determined. This can be considered as determining the number of corresponding device identifiers from the perspective of configuration information. If the number exceeds a set threshold, it can be assumed that most devices executing the degradation function that causes degradation have this configuration, and therefore, this configuration has a high probability of causing the degradation function to degrade. The set threshold can be a pre-defined threshold and is not limited here.

[0096] Based on the distribution of configuration information in the configuration information set of the device executing the degradation function, this embodiment can filter out the configuration information with a large number of corresponding device identifiers in the configuration information set. The filtered information can be considered as the configuration information corresponding to the configuration associated with the degradation function degradation.

[0097] The configuration information set disclosed herein, which is not the first screening, can be the configuration information set output after the previous screening.

[0098] Taking an A / B experiment as an example, based on the degree of relevance of the function to the A / B experiment, the degradation function can be divided into three types: The first type is experiment-specific functions, which are only executed by devices that are involved in the experiment and assigned to the experimental group; the second type is experiment-related but not experiment-specific functions, which are degraded due to indirect influence from the experiment, and are referred to as experiment-related functions; the third type does not belong to the first two types and is classified as non-experiment-related functions. For experiment attribution, the first two types are of concern.

[0099] Experiment-specific functions can be considered as functions that only appear in the experiment. Experiment-related functions can be considered as functions that are not specific to the experiment and can appear both during and outside the experiment. An experiment includes an experimental group and a control group. The experimental group is defined as performing the prescribed experiment. The control group serves as a reference for the experimental group. The experimental group can be based on the control group with the addition of code to perform the prescribed experiment. For example, the experimental group is a group of objects receiving specific treatment in an A / B experiment. Its purpose is to test the effect of a new variable or function on the indicator. The control group is a group of objects not receiving specific treatment in an A / B experiment, serving as a benchmark for comparison with the experimental group. The existence of a control group helps determine whether the experimental results are caused by the experimental variable, rather than other external factors.

[0100] Experiment-specific functions refer to functions that will only be executed by devices that have been included in the experiment and are assigned to the experimental group. If a function is experiment-specific, a device that has been included in the experiment and is within the experimental group is likely to execute the function, and the function will be included in the relationship graph reported by the device.

[0101] Figure 6 is a schematic diagram of a scenario for determining distribution information provided by an embodiment of this disclosure. Referring to Figure 6, the distribution information of the device identifier in the configuration information included in the configuration information set may include the device identifier of device 1 distributed in experimental group 1, experimental group 2 and experimental group 3, the device identifier of device 2 distributed in experimental group 2 and experimental group 3, and the device identifier of device 3 distributed in experimental group 1 and experimental group 3. For details, please refer to the connection between each device and the experimental group.

[0102] Based on the distribution information, the number of device identifiers corresponding to each configuration information in the configuration information set is determined. This can be considered as follows: the number of device identifiers corresponding to the experiments in Experiment Group 1 is 2, i.e., the device identifiers of Device 1 and Device 3. The number of device identifiers corresponding to the experiments in Experiment Group 2 is 4, and the number of device identifiers corresponding to Experiment Group 3 is 10. Since the total number of device identifiers corresponding to all experiments in Experiment Group 3 is 10, which is greater than a set threshold (e.g., 6), the experiments in Experiment Group 3 can be identified as the filtered configurations. Figure 6 only uses experimental groups as an example; control groups can also be included. It is sufficient to analyze the device identifiers corresponding to the experiments. For example, for an experiment, the number of device identifiers corresponding to the experimental groups and the control groups can be counted, and the sum of these two counts can be used to determine the total number of device identifiers corresponding to that experiment.

[0103] In Figure 6, the strategy of Experiment Group 3 is that all devices with this degradation function will be included. The experiment to which Experiment Group 3 belongs is a configuration that can affect the degradation of the degradation function, and its configuration information is determined as the filtered configuration information.

[0104] For experiment-related functions, since the functions are not specific to the experiment, there is no necessary correlation between whether a device hits a particular experimental group and whether the function is included in the relationship graph reported by that device. These functions are affected by the experiment, and their execution time deteriorates. Therefore, the distribution of deteriorated functions will show a clear correlation with the distribution of experimental groups in that experiment.

[0105] If a function degrades, experiments that may affect the degradation of the function can be identified by examining the distribution of device identifiers that execute the degradation function.

[0106] Figure 7 is a schematic diagram of another scenario for determining distribution information provided by an embodiment of this disclosure. Referring to Figure 7, compared to Figure 6, the number of device identifiers corresponding to Experiment Group 3 is reduced. However, it is still greater than the set threshold. The experiment to which Experiment Group 3 belongs can be an experiment that affects the degradation function, and the configuration information of this experiment is used as the filtered configuration information.

[0107] In the process of filtering configuration information based on the number of device identifiers corresponding to the configuration information, this disclosure can be based on a set threshold or on the size of the quantity corresponding to each configuration information, such as filtering the configuration information with the largest corresponding quantity. The set quantity can be a pre-defined quantity and is not limited here.

[0108] This disclosure provides a method for filtering configuration information in a serial funnel manner during the screening process of configuration information set, thereby selecting experimental configurations that cause degradation of the degradation function.

[0109] Figure 8 is a schematic diagram of an information determination device provided in an embodiment of this disclosure. As shown in Figure 8, the device includes:

[0110] The first determining module 810 is used to determine the configuration information set of the application, the configuration information set including multiple configuration information, the configuration information indicating the configuration transmitted by the server for the application to use;

[0111] The second determining module 820 is used to determine the function attributes of the degraded functions in the application.

[0112] The filtering module 830 is used to filter the configuration information set at least twice based on the function attributes, and the configuration information set after each filtering is used as the input for the next filtering to obtain the configuration information that causes the degradation function to degrade.

[0113] The technical solution provided in this disclosure, by filtering the configuration information set multiple times based on the function attributes of the degradation function, can gradually narrow down the range of configuration information that leads to the degradation of the degradation function, locate the configuration information that leads to the degradation of the degradation function, and realize automatic attribution of the degradation function degradation.

[0114] In one embodiment, the function attribute includes the degradation time at which the degradation function degrades, and correspondingly, the filtering module includes:

[0115] The first determining unit is used to determine the configuration information that has been changed in the configuration information set within a set time period before the degradation time.

[0116] The second determining unit is used to determine the determined configuration information as the filtered configuration information set.

[0117] In one embodiment, the first determining unit is specifically used for:

[0118] Determine the configuration information of the application at the degradation time;

[0119] Determine the configuration information of the application at a target time, where the target time is earlier than the degradation time, and the time difference between the target time and the degradation time is the set duration;

[0120] Configuration information that has changed relative to the target time in the determined configuration information of the degradation time is identified as the configuration information that has changed in the configuration information set.

[0121] In one embodiment, the function attribute includes the execution time of the degraded function, and correspondingly, the filtering module includes:

[0122] The third determining unit is used to determine the configuration information corresponding to the configuration associated with the execution of the degradation function before the execution time;

[0123] The fourth determining unit is used to determine the determined configuration information as the filtered configuration information set.

[0124] In one embodiment, the third determining unit is configured to:

[0125] In the application's relationship graph, the modified configuration is searched forward towards the node corresponding to the degradation function. The relationship graph includes the timing relationship between function execution and configuration reading within the application.

[0126] The configuration information corresponding to the found changed configuration is determined to be the configuration information corresponding to the configuration associated with the execution of the degradation function before the degradation time.

[0127] In one embodiment, the function attribute includes a device identifier corresponding to the device executing the degradation function, and the filtering module is specifically used for:

[0128] Determine the distribution information of the device identifier within the configuration information included in the configuration information set;

[0129] Based on the distribution information, determine the number of device identifiers corresponding to each configuration information in the configuration information set;

[0130] The configuration information set is filtered to select configuration information whose corresponding quantity is greater than a set threshold.

[0131] In one embodiment, the configuration information includes information corresponding to the experiment configuration, which includes parameters set for executing the specified experiment.

[0132] The information determination apparatus provided in this disclosure can execute the information determination method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of executing the method.

[0133] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0134] Figure 9 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. Referring now to Figure 9, a schematic diagram of the structure of an electronic device (e.g., a terminal device or server in Figure 9) 500 suitable for implementing an embodiment of the present disclosure is shown.

[0135] Electronic equipment 500, including:

[0136] One or more processing devices 501;

[0137] Storage device 508, for storing one or more programs,

[0138] When the one or more programs are executed by the one or more processing devices 501, the one or more processing devices 501 implement any of the methods provided in this disclosure.

[0139] The terminal devices in this disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. The electronic device shown in Figure 9 is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this disclosure.

[0140] As shown in Figure 9, the electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An edit / output (I / O) interface 505 is also connected to the bus 504.

[0141] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although FIG9 shows electronic device 500 with various devices, it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0142] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0143] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0144] The electronic device provided in this embodiment and the information determination method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0145] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the information determination method provided in the above embodiments.

[0146] It should be noted that the computer-readable medium described above in this disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination thereof.

[0147] The computer storage medium may be a storage medium for computer-executable instructions, which, when executed by a computer processor, are used to perform the methods provided in this disclosure.

[0148] Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0149] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0150] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0151] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: determine a set of configuration information for an application, the set of configuration information including multiple configuration information, the configuration information indicating a configuration transmitted by a server for the application to be used by the application;

[0152] Determine the function properties of the degraded functions in the application;

[0153] Based on the function attributes, the configuration information set is filtered at least twice, and the configuration information set after each filtering is used as the input for the next filtering to obtain the configuration information that causes the degradation function to degrade.

[0154] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0155] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0156] The modules or units described in the embodiments of this disclosure can be implemented in software or hardware. The names of modules or units do not necessarily limit the specific unit; for example, the first determining module can also be described as a "configuration information set determining module".

[0157] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0158] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0159] A computer program product includes a computer program that, when executed by a processor, implements the information determination method provided in this disclosure. It possesses the corresponding functional modules and beneficial effects of the method.

[0160] According to one or more embodiments of this disclosure, [Example 1] provides an information determination method, including:

[0161] Determine the configuration information set of the application, the configuration information set including multiple configuration information, the configuration information indicating the configuration transmitted by the server for the application to use;

[0162] Determine the function properties of the degraded functions in the application;

[0163] Based on the function attributes, the configuration information set is filtered at least twice, and the configuration information set after each filtering is used as the input for the next filtering to obtain the configuration information that causes the degradation function to degrade.

[0164] According to one or more embodiments of this disclosure, [Example 2] provides the method described in Example 1, wherein the function attribute includes the degradation time at which the degradation function degrades, and correspondingly, filtering the configuration information set includes:

[0165] Determine the configuration information that was changed in the configuration information group within a set period of time before the degradation time;

[0166] The determined configuration information is selected as the filtered configuration information set.

[0167] According to one or more embodiments of this disclosure, [Example 3] provides the method of Example 2, wherein determining the configuration information that has been changed in the configuration information set within a set period of time prior to the degradation time includes:

[0168] Determine the configuration information of the application at the degradation time;

[0169] Determine the configuration information of the application at a target time, where the target time is earlier than the degradation time, and the time difference between the target time and the degradation time is the set duration;

[0170] Configuration information that has changed relative to the target time in the determined configuration information of the degradation time is identified as the configuration information that has changed in the configuration information set.

[0171] According to one or more embodiments of this disclosure, [Example 4] provides the method described in Example 1, wherein the function attribute includes the execution time of the degraded function, and correspondingly, filtering the configuration information set includes:

[0172] Configuration information corresponding to the configuration associated with the execution of the degradation function prior to determining the execution time;

[0173] The determined configuration information is selected as the filtered configuration information set.

[0174] According to one or more embodiments of this disclosure, [Example 5] provides the method described in Example 4, wherein the configuration information corresponding to the configuration associated with the execution of the degraded function before determining the execution time includes:

[0175] In the application's relationship graph, the modified configuration is searched forward towards the node corresponding to the degradation function. The relationship graph includes the timing relationship between function execution and configuration reading within the application.

[0176] The configuration information corresponding to the found changed configuration is determined to be the configuration information corresponding to the configuration associated with the execution of the degradation function before the degradation time.

[0177] According to one or more embodiments of this disclosure, [Example 6] provides the method described in Example 1, wherein the function attribute includes a device identifier corresponding to the device executing the degradation function, and correspondingly, filtering the configuration information set includes:

[0178] Determine the distribution information of the device identifier within the configuration information included in the configuration information set;

[0179] Based on the distribution information, determine the number of device identifiers corresponding to each configuration information in the configuration information set;

[0180] The configuration information set is filtered to select configuration information whose corresponding quantity is greater than a set threshold.

[0181] According to one or more embodiments of this disclosure, Example 7 provides the method described in Example 1, wherein the configuration information includes information corresponding to the experimental configuration, and the experimental configuration includes parameters set for executing the set experiment.

[0182] According to one or more embodiments of this disclosure, [Example 8] provides an information determining apparatus, including:

[0183] The first determining module is used to determine the configuration information set of the application, the configuration information set including multiple configuration information, the configuration information indicating the configuration transmitted by the server for the application to use;

[0184] The second determining module is used to determine the function attributes of the degraded functions in the application;

[0185] The filtering module is used to filter the configuration information set at least twice based on the function attributes, and the configuration information set after each filtering is used as the input for the next filtering to obtain the configuration information that causes the degradation function to degrade.

[0186] According to one or more embodiments of this disclosure, [Example 9] an electronic device is provided, the electronic device comprising:

[0187] One or more processing devices;

[0188] Storage device for storing one or more programs.

[0189] When the one or more programs are executed by the one or more processing devices, the one or more processing devices perform the method as described in any of Examples 1-7.

[0190] According to one or more embodiments of this disclosure, [Example 10] provides a storage medium containing computer-executable instructions that, when executed by a computer processor, are used to perform the method described in any of Examples 1-7.

[0191] According to one or more embodiments of this disclosure, [Example 11] provides a computer program product including a computer program that, when executed by a processor, implements the method according to any one of Examples 1-7.

[0192] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0193] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0194] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for determining information, characterized in that, include: Determine the configuration information set of the application, the configuration information set including multiple configuration information, the configuration information indicating the configuration transmitted by the server for the application to use; Determine the function properties of the degraded functions in the application; Based on the function attributes, the configuration information set is filtered at least twice, and the configuration information set after each filtering is used as the input for the next filtering to obtain the configuration information that causes the degradation function to degrade.

2. The method according to claim 1, characterized in that, The function attribute includes the degradation time when the degradation function degrades. Correspondingly, filtering the configuration information set includes: determining the configuration information that has changed in the configuration information set within a set time period before the degradation time; and determining the determined configuration information as the filtered configuration information set.

3. The method according to claim 2, characterized in that, The step of determining the configuration information that is centrally changed within a set time period before the degradation time includes: determining the configuration information of the application at the degradation time; determining the configuration information of the application at a target time, wherein the target time is earlier than the degradation time, and the time difference between the target time and the degradation time is the set time period; and determining the configuration information that has changed relative to the configuration information at the target time from the determined configuration information at the degradation time as the configuration information centrally changed.

4. The method according to claim 1, characterized in that, The function attribute includes the execution time of the degraded function. Correspondingly, filtering the configuration information set includes: determining the configuration information corresponding to the configuration associated with the execution of the degraded function before the execution time is determined; and determining the determined configuration information as the filtered configuration information set.

5. The method according to claim 4, characterized in that, The step of determining the configuration information corresponding to the configuration associated with the execution of the degradation function before the execution time includes: searching forward in the relationship graph of the application to find the configuration that has been changed, the relationship graph including the timing relationship between function execution and configuration reading in the application; and determining the configuration information corresponding to the changed configuration found as the configuration information corresponding to the configuration associated with the execution of the degradation function before the degradation time.

6. The method according to claim 1, characterized in that, The function attribute includes the device identifier corresponding to the device executing the degradation function. Accordingly, filtering the configuration information set includes: determining the distribution information of the device identifier in the configuration information included in the configuration information set; determining the number of device identifiers corresponding to each configuration information in the configuration information set based on the distribution information; and filtering configuration information from the configuration information set whose corresponding number is greater than a set threshold.

7. The method according to claim 1, characterized in that, The configuration information includes information corresponding to the experimental configuration, which includes the parameters set for executing the experiment.

8. An information determining device, characterized in that, include: The first determining module is used to determine the configuration information set of the application, the configuration information set including multiple configuration information, the configuration information indicating the configuration transmitted by the server for the application to use; The second determining module is used to determine the function attributes of the degraded functions in the application; The filtering module is used to filter the configuration information set at least twice based on the function attributes, and the configuration information set after each filtering is used as the input for the next filtering to obtain the configuration information that causes the degradation function to degrade.

9. An electronic device, characterized in that, The electronic device includes: one or more processing devices; and a storage device for storing one or more programs, which, when executed by the one or more processing devices, cause the one or more processing devices to perform the method as described in any one of claims 1-7.

10. A storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to perform the method as described in any one of claims 1-7.

11. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images