Grayscale test user screening method and device, test method and device and electronic equipment

Abstract

The invention provides a grayscale test user screening method and device, a test method and device and electronic equipment. A specific embodiment of the grayscale test user screening method comprises the following steps: determining candidate users meeting a grayscale subscription condition; sending subscription invitation information to the candidate users; the subscription invitation information is used for inviting the candidate user to subscribe gray scale test information; and if a gray scale test instruction is received, screening out a target user meeting a gray scale test condition from candidate users who confirm to subscribe to the gray scale test information. According to the method, a high-quality gray scale test user can be obtained, so that the accuracy of a gray scale test result is improved.

Description

Technical Field

[0001] The present application relates to the field of grayscale testing, and in particular to a grayscale testing user screening method, testing method, device and electronic equipment. Background Art

[0002] As user needs continue to change, publishers can continuously update applications based on user needs. In some application scenarios, publishers can directly invite users to install new versions of applications. In this way, if there are problems with the new version of the application, the version needs to be reviewed again, which reduces the timeliness of the new version of the application.

[0003] To improve the timeliness issue, grayscale testing can be performed on the new version of the application to promptly identify problems during the testing process so that problems can be dealt with in a timely manner and the risk of re-auditing the version can be reduced.

[0004] In the related art, the publisher usually invites a certain number of users in batches to conduct grayscale testing. In this way, the quality of the invited users is not high, resulting in low accuracy of the grayscale testing results. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a grayscale test user screening method, testing method, device and electronic device to obtain high-quality grayscale test users to improve the accuracy of grayscale test results.

[0006] In the first aspect, an embodiment of the present application provides a method for screening grayscale test users, the method comprising: determining candidate users who meet grayscale subscription conditions; sending subscription invitation information to the candidate users; the subscription invitation information is used to invite the candidate users to subscribe to grayscale test information; if a grayscale test instruction is received, screening out target users who meet the grayscale test conditions from the candidate users who have confirmed their subscription to the grayscale test information. In this way, high-quality grayscale test users can be obtained through layer-by-layer screening. Moreover, each time a grayscale test is performed, no or only a small number of users occupying grayscale test quotas may be added, thereby improving the utilization rate of grayscale test quotas and the accuracy of grayscale test results when the grayscale test quotas are limited or not limited.

[0007] Optionally, the step of selecting target users who meet the grayscale test conditions from the candidate users who have confirmed their subscription to the grayscale test information includes selecting target users whose usage frequency in a preset time period is greater than a preset frequency from the candidate users who have confirmed their subscription to the grayscale test information. In this way, since the activity of the target users is high from the beginning to the end, when these target users perform grayscale testing, the probability of being able to maintain high activity is high. Therefore, the user quality of the target users can be improved to a certain extent.

[0008] Optionally, the step of screening out target users who meet the grayscale test conditions from the candidate users who have confirmed their subscription to the grayscale test information further includes: screening out candidate users whose test participation duration is greater than a preset participation duration, and candidate users who have not participated in the test within the preset duration, from the candidate users who have confirmed their subscription to the grayscale test information, to obtain the target users. In this way, candidate users who have exceeded the deadline and have low activity levels can be screened out. Subsequently, these candidate users will not be determined as target users, and will not be used for grayscale testing, thereby not occupying grayscale test quotas. In the case where grayscale test quotas are limited, the utilization rate of grayscale test quotas is effectively improved.

[0009] Optionally, before determining the candidate users who meet the grayscale subscription conditions, the method further includes: determining the active type of the user according to the frequency and time of use of the target application by the user; determining the candidate users who meet the grayscale subscription conditions from the users whose active types meet the preset type requirements; wherein the activity of the users who meet the preset type requirements is greater than the preset activity. In this way, users can be preliminarily screened by the active type to which they belong, thereby improving the user quality of the candidate users to a certain extent.

[0010] Optionally, the determining of the candidate users who meet the grayscale subscription conditions includes: if the grayscale test meets the execution condition, determining the users whose user behaviors meet the preset behavior conditions as candidate users who meet the grayscale subscription conditions. In this way, since these candidate users can meet the preset behavior conditions during the normal grayscale test, the probability of still meeting the preset behavior conditions during the subsequent grayscale test process is relatively high, thereby improving the user quality of the determined candidate users to a certain extent.

[0011] In the second aspect, the embodiment of the present application provides a grayscale testing method, which includes: pushing grayscale testing invitation information to target users who meet the grayscale testing conditions; the grayscale testing invitation information is used to invite the target users to install the target application of the version to be tested; the target users are screened by any method provided in the first aspect; based on the usage information of the installation users who have installed the target application of the version to be tested, the target application of the version to be tested is tested. In this way, since the installation users are all target users screened out layer by layer, and the user quality of the target users is high, the user quality of the installation users is also high, so when the grayscale test is performed based on the usage information of the installation users, a more accurate grayscale test result can be obtained. In addition, when the grayscale test is performed each time, it is also possible to not increase or only increase a small number of users occupying the grayscale test quota, so that when the grayscale test quota is limited, the utilization rate of the grayscale test quota and the accuracy of the grayscale test result can be achieved.

[0012] Optionally, after testing the target application of the version to be tested based on the usage information of the installation user who has installed the target application of the version to be tested, the method further includes: obtaining performance information of the target application of the version to be tested during the use of the installation user; and updating the grayscale test condition according to the performance information. In this way, the user quality of the target user can be further improved.

[0013] Optionally, after testing the target application of the version to be tested based on the usage information of the installation user who has installed the target application of the version to be tested, the method further includes: adjusting the development plan of the new function according to the usage information of the new function in the target application of the version to be tested by the installation user. In this way, it is easy to develop new functions that users like, thereby meeting user needs and improving user experience.

[0014] In the third aspect, an embodiment of the present application provides a grayscale test user screening device, which includes: a determination module for determining candidate users who meet the grayscale subscription conditions; a subscription module for sending subscription invitation information to the candidate users; the subscription invitation information is used to invite the candidate users to subscribe to the grayscale test information; a screening module for, upon receiving a grayscale test instruction, screening out target users who meet the grayscale test conditions from the candidate users who have confirmed their subscription to the grayscale test information. In this way, high-quality grayscale test users can be obtained through layer-by-layer screening. Moreover, each time a grayscale test is performed, no or only a small number of users occupying the grayscale test quota need to be added, thereby improving the utilization rate of the grayscale test quota and the accuracy of the grayscale test results when the grayscale test quota is limited or not.

[0015] In the fourth aspect, the embodiment of the present application provides a grayscale test device, which includes: a push module, which is used to push grayscale test invitation information to target users who meet the grayscale test conditions; the grayscale test invitation information is used to invite the target user to install the target application of the version to be tested; the target user is obtained by screening through any method provided in the first aspect; a test module, which is used to test the target application of the version to be tested based on the usage information of the installation user who has installed the target application of the version to be tested. In this way, since the installation users are all target users screened out layer by layer, and the user quality of the target users is high, the user quality of the installation users is also high, so that when the grayscale test is performed based on the usage information of the installation users, a more accurate grayscale test result can be obtained. In addition, when the grayscale test is performed each time, it is also possible to not increase or only increase a small number of users occupying the grayscale test quota, so that when the grayscale test quota is limited, the utilization rate of the grayscale test quota and the accuracy of the grayscale test result can be achieved.

[0016] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method provided in the first aspect are executed.

[0017] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the method provided in the first aspect are performed.

[0018] In a seventh aspect, an embodiment of the present application provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the method described in the first aspect is performed.

[0019] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the embodiments of the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A flowchart of a grayscale test user screening method provided in an embodiment of the present application;

[0022] Figure 2 A flowchart of a grayscale testing method provided in an embodiment of the present application;

[0023] Figure 3 A structural block diagram of a grayscale test user screening device provided in an embodiment of the present application;

[0024] Figure 4 A structural block diagram of a grayscale testing device provided in an embodiment of the present application;

[0025] Figure 5 A structural schematic diagram of an electronic device for executing a grayscale test user screening method or a grayscale test method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0027] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0028] It should be noted that the embodiments in this application or the technical features in the embodiments may be combined if there is no conflict.

[0029] In the related art, there is a problem that the quality of grayscale test users is not high, which leads to low accuracy of grayscale test results. In order to solve this problem, the present application provides a grayscale test user screening method. Furthermore, by first screening out candidate users who are willing to participate in the grayscale test, and then screening out target users who meet the grayscale test conditions from the candidate users. In this way, since the target user is screened out from the candidate users who are willing to participate in the grayscale test, he is more motivated to participate in the grayscale test, and then he is a high-quality user. Therefore, by performing grayscale testing on the target user, the accuracy of the grayscale test result can be improved.

[0030] In some application scenarios, the grayscale test user screening method may be executed in a server, for example. The server may include a server, a server cluster or a cloud platform, etc. This application does not impose any restrictions on this.

[0031] The defects existing in the solutions in the above-mentioned related technologies are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above-mentioned problems and the solutions proposed in the embodiments of the present invention for the above-mentioned problems below should all be the contributions made by the inventor to the present invention during the process of the present invention.

[0032] Please refer to Figure 1 , which shows a flow chart of a grayscale test user screening method provided by an embodiment of the present application. Figure 1As shown, the grayscale test user screening method includes the following steps 101 to 103.

[0033] Step 101, determining candidate users that meet the grayscale subscription conditions;

[0034] In some application scenarios, the server can, for example, determine candidate users who meet the grayscale subscription conditions from among many historical users who use the target application. It should be noted that the target application may have multiple versions, and each version of the target application may be used by multiple users. Therefore, the many historical users who use the target application may include all or some of the users who use various versions of the target application, or all or some of the users who use some versions of the target application.

[0035] In other application scenarios, the server may also determine candidate users according to the type of the target application. For example, if the target application is an e-commerce application, candidate users who meet the grayscale subscription conditions may be determined in the existing e-commerce user pool.

[0036] It should be noted that the server can determine candidate users who meet the grayscale subscription conditions from different users based on the actual required adaptability, and this application does not limit this.

[0037] Furthermore, the above-mentioned grayscale subscription conditions can be regarded as conditions for preliminarily determining whether the user may have the intention to participate in the grayscale test, which may include, for example, determining whether the user has installed the grayscale test tool, whether the installed target application needs to be updated, and other conditions.

[0038] Step 102, sending subscription invitation information to the candidate user; the subscription invitation information is used to invite the candidate user to subscribe to the grayscale test information;

[0039] In some application scenarios, after receiving the subscription invitation information, the candidate user may confirm to subscribe to the grayscale test information, or confirm to refuse to subscribe to the grayscale test information.

[0040] Among them, if a candidate user confirms to subscribe to the grayscale test information, it can be determined that the candidate user has the intention to participate in the grayscale test. If a candidate user refuses to subscribe to the grayscale test information, it can be determined that the candidate user has no intention to participate in the grayscale test. Then, the server can push the grayscale test information only to the candidate users who are determined to subscribe to the grayscale test information, so as to reduce the disturbance to the candidate users who have no intention to participate in the grayscale test. In addition, the server can store the relevant information of the candidate users who confirm to subscribe to the grayscale test information, so as to gradually obtain the candidate user pool for grayscale testing.

[0041] In some application scenarios, the grayscale test information is also the relevant information of the grayscale test. It may include, for example, test rules, test operation instructions, etc. Alternatively, it may also include relevant information of the target application of the version to be tested (such as version number, application name, function introduction, etc.), test time, test duration, etc., which is not limited in this application.

[0042] Step 103: If a grayscale test instruction is received, target users meeting the grayscale test conditions are screened out from candidate users who have confirmed subscribing to the grayscale test information.

[0043] In some application scenarios, the above-mentioned grayscale testing instructions may include relevant information of the target application of the version to be tested. Then, the server can screen out target users from candidate users who are willing to participate in the grayscale testing, and send grayscale testing invitations for the target application of the version to be tested to these target users.

[0044] In other application scenarios, the grayscale test instruction may not include the relevant information of the target application of the version to be tested, but only the relevant information for instructing to conduct the grayscale test. In this way, the server can also filter out target users from the candidate users who are willing to participate in the grayscale test, so as to subsequently send grayscale test invitations to these target users for the target application of the version to be tested.

[0045] The above-mentioned grayscale test conditions are used to screen out target users who meet the current test requirements, which may include, for example, currently having installed a grayscale test tool (such as the testFlight tool), currently not unsubscribing from grayscale test information, and other conditions.

[0046] In some application scenarios, the server may, for example, filter out target users that meet the grayscale test conditions from the candidate user pool for grayscale testing.

[0047] It should be noted that although the candidate user has the willingness to participate in the grayscale test, if he participates in the grayscale test, his enthusiasm is usually high. However, if the candidate user cannot meet the current test requirements, the accuracy of the grayscale test results based on the candidate user will not be very high. For example, if a candidate user uninstalls the grayscale test tool after confirming the subscription to the grayscale test information, the candidate user obviously does not meet the current test requirements (that is, cannot participate in the grayscale test normally), which will affect the accuracy of the grayscale test results. Therefore, the server can continue to track the subsequent situation of the candidate user, thereby improving the accuracy of the grayscale test results.

[0048] In this implementation, candidate users who are willing to participate in the grayscale test can be screened out first, and then target users who meet the grayscale test conditions can be screened out from these candidate users. In this way, high-quality grayscale test users can be obtained through layer-by-layer screening, thereby improving the accuracy of the grayscale test results.

[0049] It should be noted that this application can improve the accuracy of the grayscale test results when the grayscale test quota is limited or the grayscale test quota is not limited. Taking the grayscale test quota as an example, if the grayscale test quota is limited to 10,000 (that is, during the test process, only 10,000 user information can be registered, and the registered 10,000 users can perform grayscale testing), the server can, for example, invite 3,000 target users for the target application of the current version to be tested, and these 3,000 target users can be screened out from the candidate users who have subscribed to the grayscale test information. If all 3,000 target users have performed grayscale testing on the target application of the version to be tested, 3,000 grayscale test quotas are occupied. For the target application of the next version to be tested, if the server allocates 2,000 test quotas, it can invite at least 2,000 target users to perform grayscale testing. These 2,000 test quotas can be allocated to target users screened out from candidate users, and most of these target users are users who have already occupied grayscale test quotas. If the number of target users who have occupied the grayscale test quota is less than 2,000, you can continue to screen out target users from the newly determined candidate users who have confirmed their subscription to the grayscale test information (that is, candidate users who have not occupied the test quota) for invitation to make up 2,000 test quotas. In this way, each time the grayscale test is conducted, no or only a small number of users occupying the grayscale test quota need to be added, thereby improving the utilization rate of the grayscale test quota and the accuracy of the grayscale test results when the grayscale test quota is limited.

[0050] In addition, in some application scenarios, the above steps 101 to 102 can be performed in parallel with step 103. That is, the server can determine candidate users and send subscription invitation information to the candidate users during the process of determining the target users, so that when the number of people for the current grayscale test is insufficient, the required target users can be screened out in time to promote the normal progress of the grayscale test.

[0051] In some optional implementations, before determining the candidate users who meet the grayscale subscription conditions in step 101 above, the method further includes:

[0052] First, determine the user's active type based on the user's usage frequency and usage time of the target application;

[0053] It should be noted that the activity type to which a user belongs may include, for example, a recent activity type, a newly added activity type, a stable activity type, a returning activity type, a low-frequency activity type, a lost activity type, etc.

[0054] In this way, the server can determine the active type of the user based on the frequency and duration of use of the target application by the user. For example, if the usage frequency of a user in a preset recent period of time is greater than the corresponding preset usage frequency upper limit, the user can be determined as a recent active type. If the usage frequency of a user for a long period of time continues to be greater than the corresponding preset usage frequency upper limit, the user can be determined as a stable active type. If the usage frequency of a user in a period of time is lower than the preset usage frequency lower limit, the user can be determined as a low-frequency active type.

[0055] Then, candidate users who meet the grayscale subscription conditions can be determined from users whose activity types meet the preset type requirements; wherein the activity of users who meet the preset type requirements is greater than the preset activity.

[0056] The preset type requirements may include, for example, stable active type, recent active type, etc., which indicate that the user is highly active. Then, if the active type to which the user belongs meets the preset type requirements, it can be determined that the user is highly active.

[0057] Then, the server can identify candidate users who meet the grayscale subscription conditions among the more active users.

[0058] In this implementation, users can be preliminarily screened based on their activity types, which improves the user quality of candidate users to a certain extent.

[0059] In some other optional implementations, the determination of candidate users who meet the grayscale subscription conditions in the above step 101 includes: if the grayscale test meets the execution condition, users whose user behavior meets the preset behavior condition are determined as candidate users who meet the grayscale subscription conditions.

[0060] The above execution conditions may include, for example, that the number of target users installing the target application of the version to be tested does not reach the number of allocated test quotas, that the grayscale test quotas are not exhausted, that the duration of the grayscale test does not exceed the preset test duration, and other conditions that can be used to determine that the grayscale test can be executed normally. Then, if the grayscale test meets the execution conditions, the grayscale test can be executed normally. If the execution conditions are not met, it is determined that the grayscale test is terminated.

[0061] The above-mentioned preset behavioral conditions may include, for example, whether the frequency of use of the target application is greater than the preset frequency, whether the usage time is greater than the preset usage time, whether the grayscale testing tool is installed, whether the number of update prompts is greater than a preset threshold, etc., which can essentially be used to determine whether the user may have the willingness to participate in the grayscale test.

[0062] Then, if the grayscale test can be carried out normally, users who meet the preset behavior conditions can be determined as candidate users who may have the intention to participate in the grayscale test. In this way, since these candidate users can meet the preset behavior conditions when the grayscale test is carried out normally, the probability of still meeting the preset behavior conditions in the subsequent grayscale test process is relatively high, thereby improving the user quality of the determined candidate users to a certain extent.

[0063] In some optional implementations, the above-mentioned step 103 of screening out target users who meet the grayscale test conditions from the candidate users who have confirmed their subscription to the grayscale test information includes: screening out target users whose usage frequency within a preset time period is greater than a preset frequency from the candidate users who have confirmed their subscription to the grayscale test information.

[0064] In this way, since the activity of the target users is high from the beginning to the end, when these target users are tested in grayscale, they are more likely to maintain high activity. Therefore, the user quality of the target users can be improved to a certain extent.

[0065] In other optional implementations, the above-mentioned step 103 of screening out target users who meet the grayscale test conditions from the candidate users who have confirmed their subscription to the grayscale test information also includes: screening out candidate users whose test participation duration is greater than a preset participation duration and candidate users who have not participated in the test within the preset duration from the candidate users who have confirmed their subscription to the grayscale test information, to obtain the target users.

[0066] That is to say, the server can clean up expired candidate users and candidate users who have not used the target application for a long time, thereby improving the user quality of the target users.

[0067] It should be noted that the above-mentioned preset participation duration and preset duration can be set according to actual needs, and this application does not impose any restrictions on this. For example, if it is clearly stipulated that the duration of a user's participation in the grayscale test cannot exceed 90 days, the preset participation duration can be set to 90 days. If the grayscale test duration is 45 days, the preset duration can include, for example, 15 days, 20 days, and other relatively long durations.

[0068] Furthermore, the server can, for example, regularly clean up the candidate users in a unified manner to simplify the screening process. In addition, the cleaning can also be performed irregularly, and when a user's test participation time is longer than a preset participation time, and / or the non-participation time is longer than a preset time, the user is screened out to achieve the effect of timely cleaning.

[0069] In this implementation, candidate users who have expired or have low activity can be screened out. In this way, these candidate users will not be identified as target users and will not be used for grayscale testing, thereby not occupying grayscale testing quotas. When grayscale testing quotas are limited, the utilization rate of grayscale testing quotas is effectively improved.

[0070] See also Figure 2 , which shows a flow chart of a grayscale testing method provided by an embodiment of the present application. Figure 2 As shown, the grayscale testing method includes the following steps 201 to 202.

[0071] Step 201: Push grayscale test invitation information to target users who meet the grayscale test conditions; the grayscale test invitation information is used to invite the target users to install the target application of the version to be tested; the target users Figure 1 Any of the implementation methods shown is selected;

[0072] If the target user accepts the invitation, he / she can install the target application of the version to be tested, and then grayscale testing can be performed. If the target user refuses the invitation, he / she may not install it, and then grayscale testing cannot be performed.

[0073] Step 202: testing the target application of the version to be tested based on the usage information of the installation user who has installed the target application of the version to be tested.

[0074] The above-mentioned usage information may include, for example, the waiting time for startup, startup frequency, frequency of using new functions, usage time, comment operations, browsing content and other information.

[0075] Thus, the server can obtain the problems or advantages of the target application of this version during use based on the usage information, thereby obtaining the grayscale test results.

[0076] In this implementation, since the installation users are all target users selected through layers of screening, and the target users have high user quality, the installation users also have high user quality, so when grayscale testing is performed based on the usage information of the installation users, a relatively accurate grayscale test result can be obtained. In addition, when grayscale testing is performed each time, it is also possible to not increase or only increase a small number of users occupying grayscale test quotas, so that when the grayscale test quota is limited, the effect of taking into account both the utilization rate of the grayscale test quota and the accuracy of the grayscale test results can be achieved.

[0077] In some application scenarios, the server can obtain the performance information of the target application of the version to be tested during the use of the installed user, so as to determine whether the version to be tested has performance defects based on the performance information. The above performance information may include, for example, the startup time, response time, memory usage, crash rate, exception rate, etc. of the target application of the version to be tested.

[0078] Furthermore, the server can also update the gray test conditions according to the performance information, thereby further improving the user quality of the target user. For example, for a user terminal pre-installed with plug-in A, it takes a long time to start the target application of the version to be tested, so a restriction condition that plug-in A is not installed can be added to the initial gray test conditions. Or, in the evening, the response time is long, so a time limit constraint can be added to the initial gray test conditions to determine the cause of the long response time from the usage information of the installed users in the evening, and solve the performance defect in a timely manner.

[0079] In addition, after using the target application of the tested version, the installed user can also provide feedback on the usage, make optimization suggestions, etc., so that after receiving the feedback information, the server can evaluate the quality of the feedback information provided and the participation of the installed user who provided the feedback information, so as to screen out the installed users who provide high-quality feedback information and high participation. Then, additional restrictions on feedback information and participation can be added to the initial grayscale test conditions to further improve the user quality of the target users.

[0080] In other application scenarios, the server may also adjust the development plan of the new function according to the usage information of the new function in the target application of the version to be tested by the installation user.

[0081] For example, the server can determine the installation user's preference for the new function based on the frequency of use of the new function and the duration of each use, thereby determining whether to continue developing the new function. In addition, if there are multiple new functions, the installation user's preference for each new function can be further determined, thereby determining the development priority of multiple new functions. In this way, it is easy to develop new functions that users like, thereby meeting user needs and improving user experience.

[0082] In other application scenarios, the server can also adjust the release frequency of the test version according to performance data and feedback from installation users, so as to balance the test coverage and development resources.

[0083] Those skilled in the art will appreciate that, in the above method of a specific embodiment, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0084] Please refer to Figure 3 , which shows a structural block diagram of a grayscale test user screening device provided by an embodiment of the present application, and the grayscale test user screening device can be a module, program segment or code on an electronic device. It should be understood that the device is similar to the above Figure 1 The method embodiment corresponds to and can be executed Figure 1 The method embodiment involves various steps.

[0085] Optionally, the grayscale test user screening device includes a determination module 301, a subscription module 302, and a screening module 303. The determination module 301 is used to determine candidate users who meet the grayscale subscription conditions; the subscription module 302 is used to send subscription invitation information to the candidate users; the subscription invitation information is used to invite the candidate users to subscribe to the grayscale test information; the screening module 303 is used to screen out target users who meet the grayscale test conditions from the candidate users who confirm to subscribe to the grayscale test information when receiving the grayscale test instruction.

[0086] Optionally, the screening module 303 is further used to screen out target users whose usage frequency in a preset time period is greater than a preset frequency from candidate users who have confirmed to subscribe to the grayscale test information.

[0087] Optionally, the screening module 303 is further used to: screen out candidate users whose test participation duration is greater than a preset participation duration and candidate users who have not participated in the test within the preset duration from candidate users who have confirmed to subscribe to the grayscale test information, to obtain the target user.

[0088] Optionally, the device also includes an active type determination module, which is used to: before determining the candidate users who meet the grayscale subscription conditions, determine the active type of the user based on the frequency and time of use of the target application by the user; determine the candidate users who meet the grayscale subscription conditions from the users whose active types meet the preset type requirements; wherein the activity level of the users who meet the preset type requirements is greater than the preset activity level.

[0089] Optionally, the determination module 301 is further configured to: if the grayscale test meets the execution condition, determine the user whose user behavior meets the preset behavior condition as a candidate user who meets the grayscale subscription condition.

[0090] Please refer to Figure 4 , which shows a structural block diagram of a grayscale testing device provided in an embodiment of the present application, and the grayscale testing device can be a module, program segment or code on an electronic device. It should be understood that the device is similar to the above Figure 2 The method embodiment corresponds to and can be executed Figure 2 The method embodiment involves various steps.

[0091] Optionally, the grayscale test device may include a push module 401 and a test module 402. The push module 401 is used to push grayscale test invitation information to target users who meet the grayscale test conditions; the grayscale test invitation information is used to invite the target users to install the target application of the version to be tested; the target users can Figure 1 Any one of the implementation methods shown is screened out; a testing module 402 is used to test the target application of the version to be tested based on the usage information of the installation user who has installed the target application of the version to be tested.

[0092] Optionally, the device includes an update module, which is used to: after testing the target application of the version to be tested based on usage information of the installation user who has installed the target application of the version to be tested, obtain performance information of the target application of the version to be tested during the use of the installation user; and update the grayscale test conditions according to the performance information.

[0093] Optionally, the device also includes an adjustment module, which is used to: after testing the target application of the version to be tested based on the usage information of the installation user who has installed the target application of the version to be tested, adjust the development plan of the new function according to the usage information of the new function in the target application of the version to be tested by the installation user.

[0094] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0095] Please refer to Figure 5 , Figure 5 A structural diagram of an electronic device for executing a grayscale test user screening method or a grayscale test method provided in an embodiment of the present application, the electronic device may include: at least one processor 501, such as a CPU, at least one communication interface 502, at least one memory 503 and at least one communication bus 504. Among them, the communication bus 504 is used to realize direct connection and communication between these components. Among them, the communication interface 502 of the device in the embodiment of the present application is used to communicate signaling or data with other node devices. The memory 503 can be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 503 can optionally also be at least one storage device located away from the aforementioned processor. Computer-readable instructions are stored in the memory 503. When the computer-readable instructions are executed by the processor 501, the electronic device can execute the above Figure 1 or Figure 2 The method process is shown.

[0096] Understandably, Figure 5 The structure shown is for illustration only, and the electronic device may also include Figure 5 More or fewer components as shown, or with Figure 5 Different configurations shown. Figure 5 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0097] The present application embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the following can be performed: Figure 1 or Figure 2 Any implementation method in the method embodiment shown.

[0098] An embodiment of the present application provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments.

[0099] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0100] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0101] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0102] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0103] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A grayscale test user screening method, characterized in that: include: Determine candidate users who meet the grayscale subscription conditions; Sending subscription invitation information to the candidate user; The subscription invitation information is used to invite the candidate user to subscribe to the grayscale test information; If a grayscale test instruction is received, target users meeting the grayscale test conditions are screened out from candidate users who have confirmed subscribing to the grayscale test information.

2. The method according to claim 1, characterized in that The step of screening out target users who meet the grayscale test conditions from the candidate users who have confirmed to subscribe to the grayscale test information includes: From the candidate users who have confirmed to subscribe to the grayscale test information, target users whose usage frequency within a preset time period is greater than a preset frequency are screened out.

3. The method according to claim 1 or 2, characterized in that: The step of screening out target users who meet the grayscale test conditions from the candidate users who have confirmed to subscribe to the grayscale test information further includes: From the candidate users who have confirmed to subscribe to the grayscale test information, the candidate users whose test participation time is longer than the preset participation time and those who have not participated in the test within the preset time are screened out to obtain the target user.

4. The method according to claim 1, characterized in that: Before determining the candidate users that meet the grayscale subscription conditions, the method further includes: Determine the user's active type based on the user's usage frequency and usage time of the target application; Determine candidate users who meet the grayscale subscription condition from users whose activity types meet the preset type requirements; wherein the activity of users who meet the preset type requirements is greater than the preset activity.

5. The method according to claim 1 or 4, characterized in that: The determining of candidate users meeting the grayscale subscription conditions includes: If the grayscale test meets the execution condition, users whose user behaviors meet the preset behavior conditions are determined as candidate users who meet the grayscale subscription conditions.

6. A grayscale testing method, characterized in that: include: Pushing grayscale test invitation information to target users who meet the grayscale test conditions; the grayscale test invitation information is used to invite the target users to install the target application of the version to be tested; The target user is obtained by screening by the method according to any one of claims 1 to 5; The target application of the version to be tested is tested based on the usage information of the installation user who has installed the target application of the version to be tested.

7. The method according to claim 6, characterized in that After testing the target application of the version to be tested based on the usage information of the installation user who has installed the target application of the version to be tested, the method further includes: Obtaining performance information of the target application of the version to be tested during use by the installing user; The grayscale test condition is updated according to the performance information.

8. The method according to claim 6 or 7, characterized in that: After testing the target application of the version to be tested based on the usage information of the installation user who has installed the target application of the version to be tested, the method further includes: According to the usage information of the new function in the target application of the version to be tested by the installation user, the development plan of the new function is adjusted.

9. A grayscale test user screening device, characterized in that: include: A determination module, used to determine candidate users who meet the grayscale subscription conditions; A subscription module, used to send subscription invitation information to the candidate user; The subscription invitation information is used to invite the candidate user to subscribe to the grayscale test information; The screening module is used to screen out target users who meet the grayscale test conditions from candidate users who have confirmed to subscribe to the grayscale test information when receiving the grayscale test instruction.

10. A grayscale testing device, characterized in that: include: A push module, used for pushing grayscale test invitation information to target users who meet the grayscale test conditions; the grayscale test invitation information is used to invite the target users to install the target application of the version to be tested; the target users are obtained by screening according to the method described in any one of claims 1 to 5; The testing module is used to test the target application of the version to be tested based on the usage information of the installation user who has installed the target application of the version to be tested.

11. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 5 or 6 to 8 is executed.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 or 6 to 8 is performed.

13. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the method according to any one of claims 1-5 or 6-8 is performed.

Images