Product gray release method, device, equipment and medium

Abstract

The invention provides a product gray release method and device, equipment and a medium, which can be applied to the technical field of computers, the field of intelligent operation and maintenance, the field of big data and the field of finance. The method comprises the following steps: acquiring a gray version of a product; based on the gray version of the product, a gray strategy is created, and the gray strategy comprises a gray scale and a user group; based on a gray scale strategy, selecting a pilot user in the ith range to perform pilot function verification on the gray scale version of the product; collecting user behavior information of using the gray version of the product by the pilot user in the ith range; when the verification result of the gray version of the product accords with a gray strategy, analyzing the user behavior information by using a user behavior analysis model to obtain the ith satisfaction degree of the pilot user in the ith range to the gray version of the product; when the ith satisfaction degree exceeds the ith preset threshold value, gradually expanding the gray scale proportion; and when the gray version of the product reaches the gray-to-positive standard, deploying the gray version of the product to all target servers.

Description

Technical Field

[0001] The present invention relates to the fields of computer technology, intelligent operation and maintenance, and big data, and in particular to a product grayscale release method, device, equipment, and medium. Background Art

[0002] As the iteration cycle of software products accelerates, the time cycle from project establishment to delivery and production of new businesses is also required to be shorter and shorter. In order to avoid the system risks brought by rapid version iteration and reduce the scope of impact of product iteration and upgrade, you can set a gray release strategy to release the version to a specified server at a certain ratio. Through the gray strategy, the new version will be piloted for some users. If the pilot effect is good, you can consider full coverage, otherwise the version will be rolled back.

[0003] The existing grayscale release method usually adopts a fixed user group to conduct function pilots, and the collection of user behavior information within the pilot is also relatively simple, which may lead to certain deviations in the result analysis and fail to effectively obtain result feedback from different user groups, which may lead to reduced user satisfaction with the new version. Summary of the invention

[0004] In view of the above problems, embodiments of the present invention provide a method, apparatus, device and medium for grayscale release of a product.

[0005] According to a first aspect of the present invention, a method for grayscale release of a product is provided, comprising: step S01: obtaining a grayscale version of a product; step S02: creating a grayscale strategy based on the grayscale version of the product, wherein the grayscale strategy includes a grayscale ratio and a user group; step S03: based on the grayscale strategy, selecting a pilot user within the i-th range to perform a pilot function verification on the grayscale version of the product, wherein the pilot user within the i-th range is determined based on the grayscale ratio and the user group, and i is a positive integer greater than or equal to 1; step S04: collecting user behavior information of the pilot users within the i-th range using the grayscale version of the product; step S05 : When the verification result of the grayscale version of the product complies with the grayscale strategy, the user behavior analysis model is used to analyze the user behavior information to obtain the i-th satisfaction of the pilot users in the i-th range with the grayscale version of the product, wherein the user behavior analysis model is pre-trained; Step S06: When the i-th satisfaction exceeds the i-th preset threshold, gradually expand the grayscale ratio, and based on the expanded grayscale ratio, repeat the steps S03, S04 and S05; and Step S07: When the grayscale version of the product meets the standard for gray-to-positive conversion, deploy the grayscale version of the product to all target servers.

[0006] According to an embodiment of the present invention, the step of gradually expanding the grayscale ratio and repeating the step S03, the step S04 and the step S05 based on the expanded grayscale ratio specifically include: step S03: based on the grayscale strategy, selecting a pilot user within the i+1th range to perform a pilot function verification on the grayscale version of the product, wherein the pilot users within the i+1th range are determined based on the expanded grayscale ratio and the user group; step S04: collecting user behavior information of the pilot users within the i+1th range using the grayscale version of the product; step S05: when the verification result of the grayscale version of the product complies with the grayscale strategy, analyzing the user behavior information using a user behavior analysis model to obtain the i+1th satisfaction of the pilot users within the i+1th range with the grayscale version of the product.

[0007] According to an embodiment of the present invention, the S04 step specifically includes: obtaining the pilot function transaction log and system tracking log generated in the process of the pilot user using the gray version of the product; and obtaining the satisfaction evaluation data given by the pilot user after using the gray version of the product.

[0008] According to an embodiment of the present invention, the S04 step further specifically includes: based on the obtained pilot function transaction log and the system tracking log, collecting one or more information including user usage frequency, usage time, page dwell time, user page behavior events, page click control statistics, user behavior path analysis and funnel model analysis data.

[0009] According to an embodiment of the present invention, the method also includes: obtaining k+1 satisfaction levels, the k+1 satisfaction levels including the i-th satisfaction level to the i+k-th satisfaction level, wherein k is a positive integer greater than or equal to 3; and based on the k+1 satisfaction levels, using a production determination model to determine whether the grayscale version of the product meets the standard for gray-to-positive conversion, wherein the production determination model is pre-trained.

[0010] According to an embodiment of the present invention, the method further includes: when the verification result of the grayscale version of the product does not comply with the grayscale policy, recreating the grayscale policy.

[0011] According to an embodiment of the present invention, the method further includes: when the i-th satisfaction level does not reach the i-th preset threshold, modifying the product grayscale version.

[0012] According to an embodiment of the present invention, pre-training the user behavior analysis model includes: using a decision tree model to train the user behavior analysis model; and / or using a decision tree model to train the production determination model.

[0013] According to an embodiment of the present invention, the use of a decision tree model to train the user behavior analysis model specifically includes: obtaining historical user behavior information and historical user satisfaction to obtain an overall data set; dividing the overall data set into a training data set and a test data set; inputting sample data of the training data set into the decision tree model for training, and obtaining a decision tree model according to node classification; and verifying the decision tree model using sample data of the test data set. If the verification passes, determining the threshold of each node in the decision tree model to obtain the user behavior analysis model.

[0014] According to an embodiment of the present invention, the use of a decision tree model to train the production determination model specifically includes: obtaining historical user satisfaction and historical regularization effect evaluation data in each stage to obtain an overall data set; dividing the overall data set into a training data set and a test data set; inputting sample data of the training data set into the decision tree model for training, and obtaining a decision tree model according to node classification; and verifying the decision tree model using sample data of the test data set. If the verification passes, determining the threshold of each node in the decision tree model to obtain the production determination model.

[0015] According to an embodiment of the present invention, the method also includes: after the grayscale version of the product is converted to formal, continuing to collect user behavior information of users using the grayscale version of the product; using the decision tree model to analyze the user behavior information to obtain user satisfaction; and adjusting relevant indicators of the user behavior information in the decision tree model based on the user satisfaction.

[0016] The second aspect of the present invention provides a product grayscale release device, including: a product acquisition module, used to obtain a product grayscale version; a grayscale strategy creation module, used to create a grayscale strategy based on the product grayscale version, wherein the grayscale strategy includes a grayscale ratio and a user group; a function verification module, used to select a pilot user within the i-th range based on the grayscale strategy to perform a pilot function verification on the product grayscale version, wherein the pilot users within the i-th range are determined based on the grayscale ratio and the user group, and i is a positive integer greater than or equal to 1; a user behavior collection module, used to collect user behavior information of the pilot users within the i-th range using the product grayscale version; a user behavior analysis module , which is used to analyze the user behavior information using a user behavior analysis model to obtain the i-th satisfaction of the pilot users in the i-th range with the product grayscale version when the verification result of the product grayscale version conforms to the grayscale strategy, wherein the user behavior analysis model is pre-trained; a grayscale ratio expansion module, which is used to gradually expand the grayscale ratio when the i-th satisfaction exceeds the i-th preset threshold, and repeatedly execute the function verification module, the user behavior collection module and the user behavior analysis module based on the expanded grayscale ratio; and a gray-to-positive module, which is used to deploy the product grayscale version to all target servers when the product grayscale version reaches the gray-to-positive standard.

[0017] The third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned product grayscale release method.

[0018] The fourth aspect of the present invention also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above-mentioned product phase release method.

[0019] According to the embodiments of the present invention, the product grayscale release method, device, equipment and medium provided by the present invention have at least the following beneficial effects: through the pilot verification of the function of the product grayscale version, the user behaviors of users in different pilot ranges on multiple dimensions of the product grayscale version are collected, and relevant user feedback information in the product grayscale version verification process can be obtained. Combined with the relevant user feedback information, corresponding adjustment measures such as optimizing the product grayscale version, adjusting the pilot strategy or expanding the grayscale ratio can be taken, which is conducive to shortening the iteration cycle of the software version, while improving user satisfaction with the product grayscale version, avoiding system risks caused by rapid version iteration, and reducing the scope of product iteration upgrades. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0021] Figure 1 A schematic diagram showing an application scenario of a product phased release according to an embodiment of the present invention;

[0022] Figure 2 A flowchart of a method for grayscale release of a product according to an embodiment of the present invention is schematically shown;

[0023] Figure 3 A schematic diagram showing a distribution diagram of a pilot range for a grayscale release of a product according to an embodiment of the present invention is shown;

[0024] Figure 4 A flowchart schematically shows steps for collecting user behavior according to an embodiment of the present invention;

[0025] Figure 5 A flowchart schematically shows the grayscale ratio adjustment steps according to an embodiment of the present invention;

[0026] Figure 6 Schematically shows a flow chart of obtaining a user behavior analysis model according to an embodiment of the present invention;

[0027] Figure 7 Schematically shows a flow chart of obtaining a production determination model according to an embodiment of the present invention;

[0028] Figure 8 A schematic diagram shows a structural block diagram of a device for phased release of a product according to an embodiment of the present invention; and

[0029] Fig. 9 A block diagram of an electronic device suitable for implementing a grayscale release of a product according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0030] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0031] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "include", "comprises", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0032] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0033] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0034] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with the relevant laws, regulations and standards of relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0035] The following is a brief description of the technical terms involved in the present invention so that relevant personnel can better understand the present solution.

[0036] Grayscale testing: Before a product or application is officially released, a specific group of people are selected for trial use, and the number of trial users is gradually expanded to enable problems to be discovered and corrected in a timely manner.

[0037] Grayscale version: When a product is undergoing grayscale testing, a new version of the application will be launched for trial use in a specific range, and testers will conduct online testing on the new version. At this time, the new version of the application is the grayscale version.

[0038] Grayscale ratio: During the grayscale testing process, the ratio of opening the new version of the application to the target user group is the grayscale ratio.

[0039] Gray to formal: Opening the new version of the application to all target user groups is called gray to formal.

[0040] As the iteration cycle of program products accelerates, the time cycle from project establishment to delivery and production of new businesses is also required to be shorter and shorter. In order to avoid the system risks brought by rapid version iteration and reduce the scope of impact of product iteration and upgrade, you can set a gray release strategy to release the version to a specified server at a certain ratio. Through the gray strategy, the new version will be piloted for some users. If the pilot effect is good, you can consider full coverage, otherwise the version will be rolled back.

[0041] Through the functional pilot verification of the product grayscale version, the user behaviors of users in different pilot ranges on multiple dimensions of the product grayscale version can be collected, and relevant user feedback information in the product grayscale version verification process can be obtained. Combined with the relevant user feedback information, corresponding adjustment measures such as version optimization of the product grayscale version, adjustment of the pilot strategy or expansion of the grayscale ratio can be taken, which is conducive to shortening the iteration cycle of the software version, while improving user satisfaction with the product grayscale version, avoiding system risks caused by rapid version iteration, and reducing the scope of impact of product iteration upgrades.

[0042] Figure 1 The following schematically shows an application scenario diagram of a product phase release according to an embodiment of the present invention.

[0043] like Figure 1 As shown, according to the application scenario of this embodiment, all target servers 100 may include service terminal devices 101, 102, 103 and service servers 105. Network 104 is used to provide a medium for communication links between service terminal devices 101, 102, 103 and service servers 105. Network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc. Users may use service terminal devices 101, 102, 103 to interact with service servers 105 through network 104 to receive or send messages, etc.

[0044] The products in this embodiment may include various communication client applications installed on the business terminal devices 101, 102, and 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0045] The service terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0046] The product in this embodiment may also include various background programs installed in the business server 105. The business server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the business terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0047] Figure 2 The flowchart of the method for phased release of a product according to an embodiment of the present invention is schematically shown.

[0048] like Figure 2 As shown, the product grayscale release method of this embodiment includes operations S110 to S190. This method can be applied to a computer device, which can be the above-mentioned business server or a business terminal with certain data processing capabilities, etc., depending on the situation. The embodiment of the present invention does not limit the product type of the computer device.

[0049] In operation S110, the product acquisition step in step S01 is performed: a grayscale version of the product is acquired.

[0050] In some embodiments, the program product needs to be upgraded due to requirements such as function upgrade or device version matching. The program product version to be upgraded may include a first grayscale version, a second grayscale version, or an nth grayscale version, where n is a positive integer greater than or equal to 3. Multiple grayscale versions can be continuously optimized according to the results of the pilot verification to ensure that the final grayscale version released meets user needs. The grayscale version of the product can be obtained according to the current verification stage and the pilot verification results. For example, the first grayscale version can be obtained in the initial stage, and the product grayscale version can be optimized accordingly after analyzing user satisfaction based on user behavior information, so that in the subsequent verification stage, the second grayscale version or the nth grayscale version can be obtained.

[0051] In operation S120, the grayscale strategy creation step in step S02 is performed: based on the grayscale version of the product, a grayscale strategy is created, wherein the grayscale strategy includes a grayscale ratio and a user group.

[0052] In some embodiments, the grayscale strategy for grayscale release of a product may include a grayscale ratio and a user group. For example, the grayscale ratio during the i-th grayscale version verification of the product may be designed to be 10%, 20%, 30%, or other ratios. The grayscale ratios of different verification stages may be flexibly set according to actual needs, and the embodiments of the present invention do not limit this.

[0053] The user group can be selected based on the region, for example, different ranges of pilot users can be selected based on different urban areas, different provinces or different countries, etc., or based on user tags, for example, based on gender, age, occupation, activity level and other tags. The user group can also be selected in other ways, such as by replacing the whitelist.

[0054] Adopting different strategies for different types of user groups can avoid deviations in pilot results due to factors such as region and user preference, which can lead to unreliable grayscale version releases of the product and poor user experience.

[0055] In operation S130, the function verification step in step S03 is performed: based on the grayscale strategy, a pilot user within the i-th range is selected to perform a pilot function verification on the grayscale version of the product, wherein the pilot user within the i-th range is determined based on the grayscale ratio and the user group, and i is a positive integer greater than or equal to 1. If the function verification effect meets expectations, operation S140 is performed, otherwise, operation S180 is performed. The function verification effect meets expectations may include that the pilot user has a high usage rate of the product grayscale version and can obtain sufficient and effective user behavior information.

[0056] In operation S140, the user behavior collection step in step S04 is performed: user behavior information of the pilot users in the i-th range using the grayscale version of the product is collected.

[0057] In an embodiment of the present invention, before obtaining the user's information, the user's consent or authorization may be obtained. For example, before operation S140, a request to obtain the user's information may be issued to the user. When the user agrees or authorizes that the user's information may be obtained, the operation S140 is performed.

[0058] In an embodiment of the present invention, user behavior information may include common indicators such as usage frequency, usage time, page dwell time, etc., and may also include one or more of user page behavior events, page click control statistics, user behavior path analysis, and funnel model analysis.

[0059] User page behavior events are behaviors generated by users on the product. To analyze user behaviors, we need to define them as various events. For example, user search is an event. The time, platform, ID used, and search content of the user's search constitute the elements of a complete event and can be considered as the definition of user behavior.

[0060] Page click control statistics: Count the number of times each button is pressed, the number of times the left and right mouse buttons are pressed, the number of mouse wheel button presses, and the number of scrolls. By counting the user's page click actions, it is convenient to analyze customer preferences later.

[0061] User behavior path analysis: User behavior path is the user's access behavior path in the APP or website. In order to measure the effect of website optimization or marketing promotion, and to understand user behavior preferences, it is often necessary to analyze the conversion data of the access path. After in-depth analysis in conjunction with other analysis models, user motivations can be quickly found, thereby leading users to the optimal path or the expected path.

[0062] The funnel analysis model is used in scenarios such as website user behavior analysis and APP user behavior analysis, and is widely used in daily data operations and data analysis work such as traffic monitoring and product target conversion. The funnel analysis model can obtain user conversion rate and churn rate indicators, which can help developers find problems more intuitively.

[0063] By collecting more real-dimensional user behavior information, developers can analyze the pilot verification results of the current grayscale version based on the user behavior information, optimize the grayscale version based on the pilot verification results, or further expand the grayscale version ratio for verification.

[0064] In operation S150, the user behavior analysis step in step S05 is performed: when the verification result of the grayscale version of the product meets the grayscale strategy, the user behavior information is analyzed using the user behavior analysis model to obtain the i-th satisfaction of the pilot users in the i-th range with the product grayscale version, wherein the user behavior analysis model is pre-trained. According to the pilot user's satisfaction, it is judged whether the current product grayscale version meets the user's needs. When the pilot user's satisfaction is poor, operation S190 is performed; when the i-th satisfaction exceeds the i-th preset threshold, operation S160 is performed.

[0065] In an embodiment of the present invention, by establishing a correlation analysis model between user behavior information and user satisfaction, developers can quickly and accurately understand user satisfaction with the grayscale version, thereby facilitating developers to perform subsequent operations based on user satisfaction test results, such as optimizing the grayscale version of the product or expanding the grayscale ratio.

[0066] In operation S160, the grayscale ratio enlargement step in step S06 is performed: when the i-th satisfaction level exceeds the i-th preset threshold, the grayscale ratio is gradually enlarged, and based on the enlarged grayscale ratio, the function verification step, the user behavior collection step, and the user behavior analysis step are repeatedly performed;

[0067] In an embodiment of the present invention, using user satisfaction as a criterion for determining the grayscale ratio expansion can effectively evaluate the actual acceptance of the grayscale version by pilot users in each verification phase, thereby reducing the adverse effects of product iterations.

[0068] In operation S170, the grayscale version of the product is deployed to all target servers when the grayscale version of the product meets the grayscale version standard.

[0069] In an embodiment of the present invention, after multiple pilot verifications in multiple pilot ranges, the historical user satisfaction and historical evaluation data in each stage can be obtained, and based on the historical satisfaction and historical evaluation data, it can be judged whether the gray version of the product meets the gray-to-positive standard, thereby avoiding system risks caused by rapid version iterations, which is conducive to improving user satisfaction with the gray version of the product.

[0070] Figure 3 A distribution diagram of a pilot range for a grayscale release of a product according to an embodiment of the present invention is schematically shown.

[0071] like Figure 3 As shown, all target servers 100 that need to be deployed according to the gray version of the product of this embodiment may include the i-th pilot range 210, the i+1-th pilot range 220, the i+2-th pilot range 230, and the i+k-th pilot range, where k is a positive integer greater than or equal to 3. The server 100 may be a server that provides various services, such as a background management server that provides support for websites browsed by users using terminal devices (only as an example). The background management server may analyze and process the received data such as user requests, and feed back the processing results to the terminal device.

[0072] It should be noted that the pilot scope of the latter scope pilot may include the entire pilot scope of the former scope pilot, may include part of the pilot scope of the former scope pilot, or may not include the pilot scope of the former scope pilot, and the embodiments of the present invention are not limited to this.

[0073] Exemplarily, the selection of the pilot scope of the grayscale version may be selected using different strategies. For example, the selection of the pilot scope of the grayscale version may be based on region, by selecting servers in different regions as the pilot scope, and then gradually expanding the grayscale ratio.

[0074] Exemplarily, the selection of the pilot scope of the grayscale version can also be selected based on user tags combined with the pilot ratio. For example, when 10% of all target servers 100 need to be deployed for pilot verification during the i-th pilot, the users corresponding to all target servers that need to be deployed include multiple tags. According to the multiple tags, the corresponding users can be divided into multiple user groups, and 10% of the users in each of the multiple user groups are selected to form the i-th range of pilot users to conduct pilot verification of the grayscale version of the product. When 20% of all target servers 100 need to be deployed for pilot verification during the i+1-th pilot, 20% of the users in each of the multiple user groups can be selected to form the i+1-th range of pilot users to conduct pilot verification of the grayscale version of the product.

[0075] By gradually expanding the pilot scope of the grayscale version of the product and replacing different user groups, the pilot results can be ensured to be more reliable, thereby increasing the stability of the grayscale version release of the product and customer satisfaction, which is conducive to the promotion of the grayscale version of the product.

[0076] In an embodiment of the present invention, the gradually enlarging the grayscale ratio in operation S160 and repeating the step S03, the step S04 and the step S05 based on the enlarged grayscale ratio specifically include:

[0077] In operation S130, the function verification step in step S03 is performed: based on the grayscale strategy, pilot users within the i+1th range are selected to perform pilot function verification on the grayscale version of the product, wherein the pilot users within the i+1th range are determined based on the expanded grayscale ratio and the user group;

[0078] In operation S140, the user behavior collection step in step S04 is performed: user behavior information of pilot users in the i+1th range using the grayscale version of the product is collected; and

[0079] In operation S150, the user behavior analysis step in step S05 is executed: when the verification result of the grayscale version of the product meets the grayscale strategy, the user behavior information is analyzed using a user behavior analysis model to obtain the i+1th satisfaction of the pilot users within the i+1th range with the product grayscale version.

[0080] By using user satisfaction as the criterion for gradually expanding the grayscale ratio, user satisfaction can be guaranteed while increasing the verification scope, thereby reducing the risk of product iteration.

[0081] In an embodiment of the present invention, the user behavior collection step in step S04 specifically includes: obtaining the pilot function transaction log and system tracking log generated in the process of the pilot user using the gray version of the product; and obtaining the satisfaction evaluation data given by the pilot user after using the gray version of the product. Based on the obtained pilot function transaction log and system tracking log, collect one or more information including user usage frequency, usage time, page dwell time, user page behavior events, page click control statistics, user behavior path analysis and funnel model analysis data.

[0082] Figure 4 The flowchart schematically shows the steps of collecting user behavior according to an embodiment of the present invention.

[0083] like Figure 4 As shown, collecting user behaviors includes operations S310 to S330.

[0084] In operation S310, user pilot function transaction logs and system tracking logs are collected daily.

[0085] In an embodiment of the present invention, the pilot function transaction log and the system tracking log can be screened out according to the system transaction log table.

[0086] In operation S320, statistics are collected on user pilot function usage on a daily basis.

[0087] In an embodiment of the present invention, based on the obtained pilot function transaction log and the system embedding log, one or more information including the user's usage frequency, usage time, page dwell time, user page behavior events, page click control statistics, user behavior path analysis and funnel model analysis data is collected, so as to more effectively obtain the user's true preference for grayscale products. Among them, through the daily transaction log data entry operation, all system transaction logs are uniformly collected into the data warehouse for unified storage. At the same time, the log analysis thread analyzes the transaction log information according to the functional dimension every day and stores it in the data warehouse table for subsequent data analysis. The system embedding log is similar to the system transaction log, and the embedding log is loaded into the data warehouse every day. Indicators such as page usage frequency, page browsing time, page click events, page behavior path analysis, etc. are counted based on the function dimension.

[0088] By collecting more real-dimensional user behavior information, developers can analyze the pilot verification results of the current grayscale version based on the user behavior information, optimize the grayscale version based on the pilot verification results, or further expand the grayscale version ratio for verification.

[0089] In operation S330, user feedback on the pilot function is counted daily. By regularly conducting product satisfaction surveys, users' preferences for the launched functions are counted. For example, the feedback may include options such as excellent experience, good experience, average experience, poor experience, and extremely poor experience. In order to make the pilot users more representative, different user groups can be gradually added to count the feedback from a wider range of users.

[0090] like Figure 2 As shown, in the embodiment of the present invention, the product grayscale method further includes operations S180-S190.

[0091] In operation S180, when the verification result of the grayscale version of the product does not comply with the grayscale policy, a grayscale policy is recreated.

[0092] In the embodiments of the present invention, the grayscale strategy can be adaptively adjusted according to the validity of the collected user behavior information. For example, when the current grayscale strategy shows that the group preferences of users are too consistent or the grayscale ratio is too small or too large, the grayscale strategy can be adaptively adjusted to ensure that the collected pilot users are more representative and the reliability of the pilot verification is guaranteed.

[0093] In operation S190, when the i-th satisfaction level does not reach the i-th preset threshold, the product grayscale version is modified.

[0094] In an embodiment of the present invention, for example, if the current product grayscale version is the first grayscale version, when the user satisfaction feedback of the first grayscale version during the pilot verification process is poor and lower than a preset threshold, the product grayscale version can be upgraded and optimized to form the second grayscale version. Similarly, when the user satisfaction feedback of the mth grayscale version during the pilot verification process is poor and lower than a preset threshold, the product grayscale version can be upgraded and optimized to form the m+1th grayscale version, where m is a positive integer greater than or equal to 2.

[0095] By combining user behavior analysis and user satisfaction evaluation data, we can timely adjust the grayscale strategy and optimize the product grayscale version. This can improve the iteration efficiency of the product grayscale version and increase user satisfaction, which is beneficial to the promotion of the product grayscale version.

[0096] In an embodiment of the present invention, the product grayscale release method further includes: constructing a user behavior analysis model based on historically collected user behavior information and historical user satisfaction evaluation. Based on the constructed user behavior analysis model, user satisfaction can be obtained on the basis of acquiring new user behavior information, so that user satisfaction is used as the basis for expanding the product grayscale ratio, thereby achieving the expansion of the product grayscale ratio.

[0097] The product grayscale release method also includes: constructing a production determination model based on historical user satisfaction and historical conversion effect evaluation data in each stage. Based on the production determination model, determining whether the product grayscale version meets the grayscale conversion requirement.

[0098] Figure 5 The flowchart schematically shows the grayscale ratio adjustment steps according to an embodiment of the present invention.

[0099] like Figure 5 As shown, in the embodiment of the present invention, the grayscale ratio adjustment step may include steps S410 to S460.

[0100] In step S410, the results of the user behavior analysis model and the results of the production determination model are collected regularly.

[0101] In step S420, it is determined whether the grayscale version of the product has been converted to regular version in the current stage. If the grayscale version of the product has not been converted to regular version, step S430 is executed; if the grayscale version of the product has been converted to regular version, step S460 is executed.

[0102] In step S430, it is further determined whether the grayscale version of the product meets the grayscale ratio expansion requirement. If it meets the grayscale ratio expansion requirement, the grayscale ratio is expanded for pilot verification and step S440 is executed; otherwise, the process returns to step S410.

[0103] In step S440, it is determined whether the gray-scale conversion standard is met. When the overall gray-scale product pilot effect reaches the expected result and meets the gray-scale conversion standard in the production determination model, step S450 is executed.

[0104] In step S450, the program deployment interface is called to deploy the grayscale version of the product to all target servers 100, and the process returns to step S410.

[0105] In step S460, the user satisfaction after production is collected, and the index screening and threshold configuration of user behavior information are guided according to the satisfaction of all users with the new function after production, and then the process returns to step S410.

[0106] In an embodiment of the present invention, the method further includes: after the grayscale version of the product is completed from gray to formal, continuing to collect user behavior information of users using the grayscale version of the product; using the decision tree model to analyze the user behavior information to obtain user satisfaction; and adjusting and configuring thresholds for relevant indicators of the user behavior information in the decision tree model based on the user satisfaction. For example, if the overall user satisfaction after production is greater than the pilot function satisfaction, the overall user behavior analysis model is maintained; if the overall user satisfaction after production is less than the pilot function satisfaction, the user behavior analysis model can be continuously optimized to continuously improve the accuracy of the user behavior analysis model in guiding the grayscale version.

[0107] In an embodiment of the present invention, a decision tree model can be used to train the user behavior analysis model. The decision tree model may include a LightGBM (Light Gradient Boosting Machine) algorithm, wherein the LightGBM algorithm is a gradient boosting framework based on a decision tree, which can be used to analyze user behavior information and improve training efficiency and model accuracy. The LightGBM algorithm has the characteristics of faster training speed, less memory consumption, and the ability to process large-scale data, and is suitable for large-scale user behavior information analysis.

[0108] Figure 6 A flowchart for obtaining a user behavior analysis model according to an embodiment of the present invention is schematically shown.

[0109] like Figure 6 As shown, in an embodiment of the present invention, the process of obtaining a user behavior analysis model includes steps S510 to S530.

[0110] In step S510, historical user behavior information and historical user satisfaction are obtained to obtain an overall data set; the overall data set is divided into a training data set and a test data set.

[0111] In step S520, sample data of the training data set is input into the decision tree model for training, and a decision tree model is obtained according to node classification.

[0112] In step S530, the decision tree model is verified using sample data of the test data set. If the verification passes, the threshold of each node in the decision tree model is determined to obtain the user behavior analysis model.

[0113] After obtaining the user behavior analysis model, the user behavior analysis model is used to analyze user behavior information within the pilot scope to determine whether the user is satisfied with the new function.

[0114] For example, some historical pilot user behavior information and user satisfaction data can be found in Table 1.

[0115] Table 1 Relationship between user behavior and user satisfaction

[0116]

[0117] By using historical user behavior information and historical user satisfaction, an overall data set is obtained; the overall data set is divided into a training data set and a test data set, which can be used to train a user behavior analysis model, thereby analyzing user satisfaction in subsequent pilots to determine whether users are satisfied with the new function.

[0118] Regularly collect user satisfaction results from the user behavior analysis model. If the requirements for expanding the grayscale pilot ratio are met, adjust the grayscale pilot uniformly and continue to monitor the results of the grayscale product pilot.

[0119] In an embodiment of the present invention, the method further comprises using a decision tree model to train a production determination model.

[0120] Figure 7 The flowchart for obtaining the production determination model according to an embodiment of the present invention is schematically shown.

[0121] like Figure 7 As shown, in an embodiment of the present invention, the process of obtaining the production determination model includes steps S610 to S630.

[0122] In step S610, historical user satisfaction and historical regularization effect evaluation data in each stage are obtained to obtain an overall data set; the overall data set is divided into a training data set and a test data set.

[0123] In step S620, the sample data of the training data set is input into the decision tree model for training, and a decision tree model is obtained according to node classification;

[0124] In step S630, the decision tree model is verified using sample data of the test data set. If the verification is successful, the threshold of each node in the decision tree model is determined to obtain the production decision model.

[0125] In an embodiment of the present invention, the method further includes: obtaining k+1 satisfaction levels, the k+1 satisfaction levels including the i-th satisfaction level to the i+k-th satisfaction level, wherein k is a positive integer greater than or equal to 3; and based on the k+1 satisfaction levels, using a production determination model to determine whether the grayscale version of the product meets the standard for gray-to-positive conversion, wherein the production determination model is pre-trained. For example, it is possible to determine whether the grayscale version meets the standard for gray-to-positive conversion based on the trend of customer satisfaction changes for k consecutive times. For example, the standard for gray-to-positive conversion can be set to multiple consecutive increases in user satisfaction, such as four consecutive increases in user satisfaction.

[0126] In an embodiment of the present invention, the standard for gray-to-positive conversion can also be set to that the i+kth satisfaction reaches a set threshold, for example, the threshold can be set to 60%. The i+kth satisfaction corresponds to the i+kth range pilot verification, where k is a positive integer greater than or equal to 3. The proportion of target servers in the pilot within the i+kth range to all target servers 100 can be set according to actual needs, for example, the proportion of target servers in the pilot within the i+kth range to all target servers 100 is greater than or equal to 50%.

[0127] The overall effect of the new function after gray-to-full conversion under the satisfaction ratio of each stage can be seen in Table 2. When the satisfaction of each stage is continuously improved for many times or the satisfaction of the i+kth pilot exceeds the threshold, the overall effect after gray-to-full conversion can meet the requirements.

[0128] Table 2 Relationship between the satisfaction rate at each stage and the overall effect evaluation after the gray-to-positive transition

[0129]

[0130] Based on the above-mentioned product grayscale release method, the present invention also provides a product grayscale release device. Figure 8 The device is described in detail.

[0131] Figure 8 The structure block diagram of the device for phased release of products according to an embodiment of the present invention is schematically shown.

[0132] like Figure 8 As shown, the product grayscale release device 1000 of this embodiment includes a product acquisition module 1010, a grayscale strategy creation module 1020, a function verification module 1030, a user behavior collection module 1040, a user behavior analysis module 1050, a grayscale ratio expansion module 1060 and a grayscale conversion module 1070.

[0133] The product acquisition module 1010 is used to acquire a grayscale version of a product. In one embodiment, the product acquisition module 1010 may be used to perform the operation S110 described above, which will not be described in detail herein.

[0134] The grayscale strategy creation module 1020 is used to create a grayscale strategy based on the grayscale version of the product, wherein the grayscale strategy includes a grayscale ratio and a user group. In one embodiment, the grayscale strategy creation module 1020 can be used to perform the operation S120 described above, which will not be repeated here.

[0135] The function verification module 1030 is used to select pilot users within the i-th range to perform pilot function verification on the product gray version based on the gray strategy, wherein the pilot users within the i-th range are determined based on the gray ratio and the user group, and i is a positive integer greater than or equal to 1. In one embodiment, the function verification module 1030 can be used to perform the operation S130 described above, which will not be repeated here.

[0136] The user behavior collection module 1040 is used to collect user behavior information of the pilot users in the i-th range using the gray version of the product. In one embodiment, the user behavior collection module 1040 can be used to perform the operation S140 described above, which will not be described in detail here.

[0137] The user behavior analysis module 1050 is used to analyze the user behavior information using a user behavior analysis model when the verification result of the grayscale version of the product meets the grayscale strategy, and obtain the i-th satisfaction of the pilot users in the i-th range with the grayscale version of the product, wherein the user behavior analysis model is pre-trained. In one embodiment, the user behavior analysis module 1050 can be used to perform the operation S150 described above, which will not be repeated here.

[0138] The grayscale ratio expansion module 1060 is used to gradually expand the grayscale ratio when the i-th satisfaction exceeds the i-th preset threshold, and based on the expanded grayscale ratio, repeatedly execute the function verification module, the user behavior collection module and the user behavior analysis module. In one embodiment, the grayscale ratio expansion module 1060 can be used to perform the operation S160 described above, which will not be repeated here.

[0139] The gray-to-regularization module 1070 is used to deploy the gray-to-regularization version of the product to all target servers when the gray-to-regularization version of the product meets the gray-to-regularization standard. In one embodiment, the gray-to-regularization module 1070 can be used to perform the operation S170 described above, which will not be repeated here.

[0140] After multiple pilot verifications in multiple pilot ranges, we can obtain the historical user satisfaction and historical conversion evaluation data in each stage, and judge whether the gray version of the product meets the gray conversion standards based on the historical satisfaction and historical conversion evaluation data, thereby avoiding system risks caused by rapid version iteration and improving user satisfaction with the gray version of the product.

[0141] According to an embodiment of the present invention, any multiple modules among the product acquisition module 1010, the grayscale strategy creation module 1020, the function verification module 1030, the user behavior collection module 1040, the user behavior analysis module 1050, the grayscale ratio expansion module 1060 and the gray conversion module 1070 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the product acquisition module 1010, the grayscale strategy creation module 1020, the function verification module 1030, the user behavior collection module 1040, the user behavior analysis module 1050, the grayscale ratio expansion module 1060 and the gray conversion module 1070 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware and firmware or by an appropriate combination of any of them. Alternatively, at least one of the product acquisition module 1010, the grayscale strategy creation module 1020, the function verification module 1030, the user behavior collection module 1040, the user behavior analysis module 1050, the grayscale ratio expansion module 1060 and the gray conversion module 1070 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0142] Fig. 9 A block diagram of an electronic device suitable for implementing a grayscale release of a product according to an embodiment of the present invention is schematically shown.

[0143] like Fig. 9As shown, the electronic device 800 according to an embodiment of the present invention includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include an onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0144] In RAM 803, various programs and data required for the operation of electronic device 800 are stored. Processor 801, ROM 802 and RAM 803 are connected to each other via bus 804. Processor 801 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 802 and / or RAM 803. It should be noted that the program can also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in one or more memories.

[0145] According to an embodiment of the present invention, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device 800 may further include one or more of the following components connected to the I / O interface 805: an input portion 806 including a keyboard, a mouse, etc.; an output portion 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 808 including a hard disk, etc.; and a communication portion 809 including a network interface card such as a LAN card, a modem, etc. The communication portion 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage portion 808 as needed.

[0146] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0147] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 802 and / or RAM 803 described above and / or one or more memories other than ROM 802 and RAM 803.

[0148] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the product phased release method provided by the embodiment of the present invention.

[0149] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when the processor 801 executes the computer program. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0150] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 809, and / or installed from a removable medium 811. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0151] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0152] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages, specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, such as Java, C++, python, "C" language or similar programming languages. The program code can be executed completely on the user computing device, partially on the user device, partially on the remote computing device, or completely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0153] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0154] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present invention may be combined and / or coupled in various ways, even if such combinations and / or couplings are not explicitly described in the present invention. In particular, the features described in the various embodiments and / or claims of the present invention may be combined and / or coupled in various ways without departing from the spirit and teachings of the present invention. All of these combinations and / or couplings fall within the scope of the present invention.

[0155] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination. The scope of the present invention is defined by the attached claims and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A product phased release method, wherein: The method comprises: Step S01: Get the grayscale version of the product; Step S02: creating a grayscale strategy based on the grayscale version of the product, wherein the grayscale strategy includes a grayscale ratio and a user group; Step S03: Based on the grayscale strategy, select pilot users within the i-th range to perform pilot function verification on the grayscale version of the product, wherein the pilot users within the i-th range are determined based on the grayscale ratio and the user group, and i is a positive integer greater than or equal to 1; Step S04: collecting user behavior information of pilot users within the i-th range using the gray version of the product; Step S05: when the verification result of the grayscale version of the product meets the grayscale strategy, the user behavior information is analyzed using a user behavior analysis model to obtain the i-th satisfaction of the pilot users in the i-th range with the grayscale version of the product, wherein the user behavior analysis model is pre-trained; Step S06: when the i-th satisfaction level exceeds the i-th preset threshold, gradually enlarging the grayscale ratio, and based on the enlarged grayscale ratio, repeatedly executing steps S03, S04 and S05; and Step S07: When the grayscale version of the product reaches the standard of grayscale conversion to formal, the grayscale version of the product is deployed to all target servers.

2. The method according to claim 1, wherein: The gradually enlarging the grayscale ratio and repeating the step S03, the step S04 and the step S05 based on the enlarged grayscale ratio specifically includes: Step S03: Based on the grayscale strategy, select pilot users within the i+1th range to perform pilot function verification on the grayscale version of the product, wherein the pilot users within the i+1th range are determined based on the expanded grayscale ratio and the user group; Step S04: collecting user behavior information of pilot users in the i+1th range using the gray version of the product; Step S05: When the verification result of the grayscale version of the product complies with the grayscale strategy, the user behavior information is analyzed using a user behavior analysis model to obtain the i+1th satisfaction of the pilot users within the i+1th range with the grayscale version of the product.

3. The method according to claim 1, wherein: The S04 step specifically includes: Obtaining the pilot function transaction logs and system tracking logs generated during the pilot user's use of the gray version of the product; and Obtain satisfaction evaluation data given by the pilot users after using the grayscale version of the product.

4. The method according to claim 3, wherein: The S04 step further specifically includes: Based on the obtained pilot function transaction log and the system tracking log, one or more information including user usage frequency, usage time, page dwell time, user page behavior events, page click control statistics, user behavior path analysis and funnel model analysis data is collected.

5. The method according to claim 1, 2 or 4, wherein: The method further comprises: Obtain k+1 satisfaction levels, where the k+1 satisfaction levels include the i-th satisfaction level to the i+k-th satisfaction level, where k is a positive integer greater than or equal to 3; and Based on the k+1 satisfaction levels, a production determination model is used to determine whether the grayscale version of the product meets the grayscale conversion standard, wherein the production determination model is pre-trained.

6. The method according to claim 1, 2 or 4, wherein: The method further comprises: When the verification result of the grayscale version of the product does not comply with the grayscale policy, a grayscale policy is recreated.

7. The method according to claim 1, 2 or 4, wherein: The method further comprises: When the i-th satisfaction level does not reach the i-th preset threshold, the product grayscale version is modified.

8. The method according to claim 5, wherein: Pre-training the user behavior analysis model includes: using a decision tree model to train the user behavior analysis model; and / or, The production determination model is trained using a decision tree model.

9. The method according to claim 8, wherein: The method of training the user behavior analysis model by using a decision tree model specifically includes: Obtain historical user behavior information and historical user satisfaction to obtain an overall data set; Dividing the overall data set into a training data set and a test data set; Inputting sample data of the training data set into the decision tree model for training, and obtaining a decision tree model according to node classification; and The decision tree model is verified using sample data of the test data set. If the verification passes, the threshold of each node in the decision tree model is determined to obtain the user behavior analysis model.

10. The method according to claim 8, wherein: The use of the decision tree model to train the production determination model specifically includes: Obtain historical user satisfaction and historical conversion effect evaluation data in each stage to obtain an overall data set; Dividing the overall data set into a training data set and a test data set; Inputting sample data of the training data set into the decision tree model for training, and obtaining a decision tree model according to node classification; and The decision tree model is verified using sample data of the test data set. If the verification is successful, the threshold of each node in the decision tree model is determined to obtain the production decision model.

11. The method according to claim 10, wherein: The method further comprises: After the gray version of the product is converted to official version, continue to collect user behavior information of users using the gray version of the product; Analyzing the user behavior information using the decision tree model to obtain user satisfaction; and The relevant indicators of the user behavior information in the decision tree model are adjusted according to the user satisfaction.

12. A product phased release device, characterized in that: The device comprises: Product acquisition module, used to obtain the grayscale version of the product; A grayscale strategy creation module, used to create a grayscale strategy based on the grayscale version of the product, wherein the grayscale strategy includes a grayscale ratio and a user group; A function verification module, configured to select, based on the grayscale strategy, a pilot user within the i-th range to perform a pilot function verification on the grayscale version of the product, wherein the pilot user within the i-th range is determined based on the grayscale ratio and the user group, and i is a positive integer greater than or equal to 1; A user behavior collection module, used to collect user behavior information of pilot users within the i-th range using the gray version of the product; A user behavior analysis module is used to analyze the user behavior information using a user behavior analysis model to obtain an i-th satisfaction of the pilot user within the i-th range with the product grayscale version when the verification result of the product grayscale version meets the grayscale strategy, wherein the user behavior analysis model is pre-trained; a grayscale ratio expansion module, configured to gradually expand the grayscale ratio when the i-th satisfaction level exceeds the i-th preset threshold, and repeatedly execute the function verification module, the user behavior collection module and the user behavior analysis module based on the expanded grayscale ratio; and The gray-to-regularization module is used to deploy the gray-to-regularization version of the product to all target servers when the gray-to-regularization version of the product meets the gray-to-regularization standard.

13. An electronic device, comprising: one or more processors; as well as a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 11.

14. A computer-readable storage medium storing computer-executable instructions, wherein the instructions are used to implement the method according to any one of claims 1 to 11 when executed.

Images