Gray release method and device, electronic equipment and storage medium

Abstract

The invention provides a gray release method and device, electronic equipment and a storage medium. According to the method, system index data are monitored in real time in the process of performing gray release on a target project, and the system index data comprise dimension index data of a plurality of dimensions; obtaining first target data corresponding to a current publishing period from the system index data every first preset time interval; determining corresponding strategy adjustment information according to the first target data; according to the strategy adjustment information, adjusting the gray release strategy of the current release period to obtain a target gray release strategy; and in the next release period, performing gray release on the target project according to the target gray release strategy. According to the scheme, the system index data can be monitored in real time to realize intelligent dynamic adjustment of the gray release strategy, so that the decision-making efficiency and accuracy are improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a grayscale publishing method, device, electronic device and storage medium. Background Art

[0002] Grayscale releases can usually be divided into two types: front-end (client and web) and back-end. Front-end grayscale releases mainly involve functional updates of client applications (such as mobile applications, desktop applications) and web applications. By gradually pushing new features to users, monitor their performance and user feedback. Application scenarios include: testing of new features (such as new interface design, functional modules), gradual promotion of features (such as new payment methods, social sharing functions), and so on. Back-end grayscale releases involve functional updates on the server side and changes to the API (Application Programming Interface). By gradually pushing the new version of the service to some users, monitor system performance and user feedback. Application scenarios include: online testing of new APIs (such as data structure changes, interface performance optimization), version iteration of back-end services (such as database migration, service architecture adjustment), and so on.

[0003] Currently, grayscale releases mostly rely on rules and policies set by administrators, with limited scalability and automation, and lack of real-time analysis capabilities, resulting in a complex and delayed decision-making process. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a grayscale release method, device, electronic device and storage medium to solve the problem that grayscale release currently mostly relies on rules and policies set by administrators, has limited scalability and automation, and lacks real-time analysis capabilities, resulting in a complex and delayed decision-making process. The specific technical solution is as follows:

[0005] In a first aspect, the present application provides a grayscale release method, comprising:

[0006] During the grayscale release of the target project, real-time monitoring of system indicator data, wherein the system indicator data includes dimensional indicator data of several dimensions;

[0007] At every first preset time interval, obtaining first target data corresponding to the current release cycle from the system indicator data;

[0008] Determining corresponding strategy adjustment information according to the first target data;

[0009] Adjust the grayscale release strategy of the current release cycle according to the strategy adjustment information to obtain a target grayscale release strategy;

[0010] In the next release cycle, the target project is released in grayscale according to the target grayscale release strategy.

[0011] In a possible implementation manner, determining corresponding strategy adjustment information according to the first target data includes:

[0012] The first target data is input into a pre-trained large model, so that the large model can perform problem analysis and prediction on the grayscale release strategy of the current release cycle based on the first target data, obtain the problem analysis results corresponding to the grayscale release strategy of the current release cycle, and determine the corresponding strategy adjustment information based on the problem analysis results.

[0013] In a possible implementation, the method further includes:

[0014] Determine the first performance data corresponding to the large model in the current evaluation period at every second preset time interval;

[0015] When the first performance data does not meet the preset conditions, obtaining second target data corresponding to the current evaluation period from the system indicator data;

[0016] The large model is updated using the second target data to obtain an updated model, and the large model used in the system is replaced with the updated model.

[0017] In a possible implementation, the method further includes:

[0018] Performing a performance test on the updated model to obtain corresponding second performance data;

[0019] In the case where the second performance data is better than the first performance data, a step of replacing the large model applied in the system with the updated model is performed.

[0020] In a possible implementation manner, determining corresponding strategy adjustment information according to the problem analysis result includes:

[0021] When the problem analysis result is that the target project performance is at the first performance level, and the user satisfaction is at the first satisfaction level, the corresponding strategy adjustment information is to expand the grayscale range;

[0022] When the problem analysis result is that the target project performance is at the second performance level, and the user satisfaction is at the second satisfaction level, the corresponding strategy adjustment information is to maintain the existing grayscale range;

[0023] When the problem analysis result is that the target project performance is at the third performance level, or the user satisfaction is at the third satisfaction level, the corresponding strategy adjustment information is to reduce the grayscale range;

[0024] When the problem analysis result is that the performance of the target project is at the fourth performance level, or the user satisfaction is at the fourth satisfaction level, the corresponding policy adjustment information is to roll back and update to the old version corresponding to the target project;

[0025] Among them, the first performance level is better than the second performance level, the second performance level is better than the third performance level, and the third performance level is better than the fourth performance level; the first satisfaction level is better than the second satisfaction level, the second satisfaction level is better than the third satisfaction level, and the third satisfaction level is better than the fourth satisfaction level.

[0026] In a possible implementation, the method further includes:

[0027] For each dimension indicator data, obtain the dimension indicator range corresponding to the dimension indicator data;

[0028] When the dimension indicator data exceeds the dimension indicator range, an alarm mechanism is triggered to handle the abnormal situation indicated by the dimension indicator data.

[0029] In a possible implementation, the real-time monitoring system indicator data includes:

[0030] Monitor user experience index data, performance index data, usage index data, conversion rate index data and feedback index data in real time, and use the user experience index data, performance index data, usage index data, conversion rate index data and feedback index data as the system index data.

[0031] In a second aspect, the present application provides a grayscale publishing device, including:

[0032] A monitoring module is used to monitor system indicator data in real time during the grayscale release of the target project, wherein the system indicator data includes dimensional indicator data of several dimensions;

[0033] An acquisition module, configured to acquire first target data corresponding to a current release cycle from the system indicator data at every first preset time interval;

[0034] A determination module, configured to determine corresponding strategy adjustment information according to the first target data;

[0035] An adjustment module, used to adjust the grayscale release strategy of the current release cycle according to the strategy adjustment information to obtain a target grayscale release strategy;

[0036] The release module is used to perform a grayscale release on the target project according to the target grayscale release strategy in the next release cycle.

[0037] In a possible implementation manner, the determining module is specifically configured to:

[0038] The first target data is input into a pre-trained large model, so that the large model can perform problem analysis and prediction on the grayscale release strategy of the current release cycle based on the first target data, obtain the problem analysis results corresponding to the grayscale release strategy of the current release cycle, and determine the corresponding strategy adjustment information based on the problem analysis results.

[0039] In a possible implementation manner, the device further includes an updating module, configured to:

[0040] Determine the first performance data corresponding to the large model in the current evaluation period at every second preset time interval;

[0041] When the first performance data does not meet the preset conditions, obtaining second target data corresponding to the current evaluation period from the system indicator data;

[0042] The large model is updated using the second target data to obtain an updated model, and the large model used in the system is replaced with the updated model.

[0043] In a possible implementation manner, the update module is further configured to:

[0044] Performing a performance test on the updated model to obtain corresponding second performance data;

[0045] In the case where the second performance data is better than the first performance data, a step of replacing the large model applied in the system with the updated model is performed.

[0046] In a possible implementation manner, the determining module is further configured to:

[0047] When the problem analysis result is that the target project performance is at the first performance level, and the user satisfaction is at the first satisfaction level, the corresponding strategy adjustment information is to expand the grayscale range;

[0048] When the problem analysis result is that the target project performance is at the second performance level, and the user satisfaction is at the second satisfaction level, the corresponding strategy adjustment information is to maintain the existing grayscale range;

[0049] When the problem analysis result is that the target project performance is at the third performance level, or the user satisfaction is at the third satisfaction level, the corresponding strategy adjustment information is to reduce the grayscale range;

[0050] When the problem analysis result is that the performance of the target project is at the fourth performance level, or the user satisfaction is at the fourth satisfaction level, the corresponding policy adjustment information is to roll back and update to the old version corresponding to the target project;

[0051] Among them, the first performance level is better than the second performance level, the second performance level is better than the third performance level, and the third performance level is better than the fourth performance level; the first satisfaction level is better than the second satisfaction level, the second satisfaction level is better than the third satisfaction level, and the third satisfaction level is better than the fourth satisfaction level.

[0052] In a possible implementation manner, the device further includes an alarm module, which is used to:

[0053] For each dimension indicator data, obtain the dimension indicator range corresponding to the dimension indicator data;

[0054] When the dimension indicator data exceeds the dimension indicator range, an alarm mechanism is triggered to handle the abnormal situation indicated by the dimension indicator data.

[0055] In a possible implementation manner, the monitoring module is specifically used to:

[0056] Monitor user experience index data, performance index data, usage index data, conversion rate index data and feedback index data in real time, and use the user experience index data, performance index data, usage index data, conversion rate index data and feedback index data as the system index data.

[0057] In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0058] Memory, used to store computer programs;

[0059] The processor is used to implement any method step described in the first aspect when executing a program stored in the memory.

[0060] In a fourth aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any method step described in the first aspect is implemented.

[0061] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute any of the grayscale release methods described above.

[0062] Beneficial effects of the embodiments of the present application:

[0063] The embodiments of the present application provide a grayscale release method, device, electronic device and storage medium. In the embodiments of the present application, first, in the process of grayscale release of a target project, the system indicator data is monitored in real time, wherein the system indicator data includes dimensional indicator data of several dimensions, and at every first preset time interval, the first target data corresponding to the current release cycle is obtained from the system indicator data, and the corresponding policy adjustment information is determined according to the first target data, and then, the grayscale release policy of the current release cycle is adjusted according to the policy adjustment information to obtain the target grayscale release policy, and in the next release cycle, the target project is grayscale released according to the target grayscale release policy. This scheme can realize intelligent dynamic adjustment of the grayscale release policy by real-time monitoring of the system indicator data, thereby improving decision-making efficiency and accuracy.

[0064] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0067] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0068] Figure 1 A flowchart of a grayscale publishing method provided in an embodiment of the present application;

[0069] Figure 2 A flowchart of another grayscale publishing method provided in an embodiment of the present application;

[0070] Figure 3 A flowchart of another grayscale publishing method provided in an embodiment of the present application;

[0071] Figure 4 An overall processing flow chart of a grayscale release provided in an embodiment of the present application;

[0072] Figure 5 A schematic diagram of the structure of a grayscale publishing device provided in an embodiment of the present application;

[0073] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0075] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0076] Figure 1 A flow chart of a grayscale publishing method provided for an embodiment of the present application. This method can be applied to one or more electronic devices such as smart phones, laptops, desktop computers, portable computers, servers, etc. In addition, the execution subject of this method can be hardware or software. When the above-mentioned execution subject is hardware, the execution subject can be one or more of the above-mentioned electronic devices. For example, a single electronic device can execute this method, or multiple electronic devices can cooperate with each other to execute this method. When the above-mentioned execution subject is software, this method can be implemented as multiple software or software modules, or as a single software or software module. It is not specifically limited here.

[0077] like Figure 1 As shown, the method specifically includes:

[0078] Step 101: During the grayscale release of a target project, real-time monitoring of system indicator data is performed, wherein the system indicator data includes dimensional indicator data of several dimensions.

[0079] The above-mentioned target projects can be front-end grayscale release projects, such as functional updates of client applications (such as mobile applications, desktop applications) and Web applications; they can also be back-end grayscale release projects, such as server-side functional updates and API (Application Programming Interface) changes.

[0080] The above-mentioned system indicator data includes dimensional indicator data of several dimensions, for example, performance indicator data of system performance dimension, user experience indicator data of user experience dimension, usage indicator data of user usage dimension, etc.

[0081] In the application, you can deploy data collection scripts to the application client and server to capture user interaction logs and system performance logs, use message queue middleware (such as Kafka) to transmit the data in real time to the data storage layer for storage, and then obtain system indicator data from the user interaction logs and system performance logs.

[0082] In one embodiment, real-time monitoring of system indicator data may include the following steps:

[0083] Monitor user experience index data, performance index data, usage index data, conversion rate index data and feedback index data in real time, and use the user experience index data, performance index data, usage index data, conversion rate index data and feedback index data as the system index data.

[0084] Specifically, user experience indicator data may include response time, error rate, user satisfaction, etc., where response time refers to the system's response speed to user requests; error rate refers to the proportion of errors encountered by users when using new functions; user satisfaction refers to the user satisfaction collected through surveys or feedback. Performance indicator data may include system load and request processing time, where system load refers to the CPU, memory and network bandwidth usage of the server; request processing time refers to the time required for the system to process a single request. Usage indicator data may include the number of active users and function usage rate, where the number of active users refers to the number of users using new functions; function usage rate refers to the frequency with which new functions are used by users. Conversion rate indicator data may include conversion rate and churn rate, where conversion rate refers to the proportion of users completing specific operations (such as purchases, registrations, etc.); churn rate refers to the proportion of users who stop using a new function after using it. Feedback indicator data may include the number of user feedback and the number of problem reports, where the number of user feedback refers to the number and quality of collected user feedback; the number of problem reports refers to the number of bugs or problems reported by users.

[0085] Step 102: Obtain first target data corresponding to a current release cycle from the system indicator data at every first preset time interval.

[0086] In actual applications, a period (ie, a release period) for adjusting the grayscale release strategy may be preset, for example, ten minutes, one hour, etc. Correspondingly, the first preset time interval is the duration corresponding to the period.

[0087] Based on this, in the embodiment of the present application, at every first preset time interval, data corresponding to the collection time in the current release cycle is obtained from the system indicator data as the data to be processed, and then the data to be processed is cleaned (such as deleting missing values ​​and invalid values), sorted (such as formatting, that is, unifying the data format), denoised (applying filters to remove noise data) and other pre-processing to obtain the first target data. Thus, the quality and accuracy of the first target data are ensured.

[0088] Step 103: Determine corresponding strategy adjustment information according to the first target data.

[0089] Step 104: Adjust the grayscale release strategy of the current release cycle according to the strategy adjustment information to obtain a target grayscale release strategy.

[0090] Step 105: In the next release cycle, the target project is released in a grayscale manner according to the target grayscale release strategy.

[0091] For ease of understanding, steps 103 to 105 are described in a unified manner as follows:

[0092] Strategy adjustment information is used to represent the recommended content for adjusting the grayscale release strategy of the current release cycle.

[0093] In one embodiment, step 103 may specifically include the following steps:

[0094] The first target data is input into a pre-trained large model, so that the large model can perform problem analysis and prediction on the grayscale release strategy of the current release cycle based on the first target data, obtain the problem analysis results corresponding to the grayscale release strategy of the current release cycle, and determine the corresponding strategy adjustment information based on the problem analysis results.

[0095] The above-mentioned large model is an important model in the field of AI, usually referring to a deep learning model with a very large number of parameters and a very large amount of data. In the application, you can use a deep learning framework such as TensorFlow or PyTorch, select a suitable model (such as GPT3.5), and train it using historical system data. The target output is a predicted or recommended grayscale release decision, that is, strategy adjustment information, until the model converges to obtain the large model used in this solution.

[0096] Specifically, determining corresponding strategy adjustment information according to the problem analysis result may include the following steps:

[0097] When the problem analysis result is that the performance of the target project is the first performance level, and the user satisfaction is the first satisfaction level, the corresponding policy adjustment information is to expand the grayscale range; when the problem analysis result is that the performance of the target project is the second performance level, and the user satisfaction is the second satisfaction level, the corresponding policy adjustment information is to maintain the existing grayscale range; when the problem analysis result is that the performance of the target project is the third performance level, or the user satisfaction is the third satisfaction level, the corresponding policy adjustment information is to reduce the grayscale range; when the problem analysis result is that the performance of the target project is the fourth performance level, or the user satisfaction is the fourth satisfaction level, the corresponding policy adjustment information is to roll back and update to the old version corresponding to the target project; wherein, the first performance level is better than the second performance level, the second performance level is better than the third performance level, and the third performance level is better than the fourth performance level; the first satisfaction level is better than the second satisfaction level, the second satisfaction level is better than the third satisfaction level, and the third satisfaction level is better than the fourth satisfaction level.

[0098] That is, if the indicators show that the new function performs well and user satisfaction is high, the model may recommend expanding the grayscale range; if the indicators show that the current status is acceptable, the model may recommend maintaining the existing grayscale range; if the indicators show that there are performance problems or poor user experience, the model may recommend reducing the grayscale range; if the indicators show that the new function causes serious problems, the model may recommend rolling back the update to the old version.

[0099] Among them, the grayscale range can refer to the size of the user group. For example, the strategy may be initially tested on only 1% of users. If the strategy is successful, it may be expanded to 5%, 10% or even 100% of users. It can also refer to geographic location. For example, the strategy may be first implemented in a small geographic area (such as a city or country), and then gradually expanded to more regions. It can also be device type. For example, the strategy may initially target only a specific type of device (such as iOS devices), and then expand to Android devices, and eventually cover all device types. It can also refer to user behavior. For example, the strategy may initially target only specific user behaviors or activities, such as frequent shoppers or newly registered users, and then expand to other users. It can also refer to version control. For example, the strategy may initially target only the new version of the software, and as the version stability improves, it is gradually promoted to the old version.

[0100] After obtaining the policy adjustment information through the above scheme, the grayscale release strategy of the current release cycle is adjusted according to the policy adjustment information to obtain the target grayscale release strategy, and in the next release cycle, the target project is grayscale released according to the target grayscale release strategy. In this way, real-time dynamic adjustment of the grayscale release strategy is achieved.

[0101] In the application, the grayscale release strategy of the initial release cycle can be a manually set release strategy, or it can be a strategy that analyzes and outputs historical system indicator data through a large model.

[0102] In addition, in another embodiment of the present application, the problem analysis results may also include analysis content of the system and the target project. Based on this, the big model may also output optimization suggestions for the system and the target project according to the problem analysis results.

[0103] For example, if the problem analysis result is that the system is highly loaded, the model may recommend increasing server resources to cope with the high load. If the problem analysis result is that users have poor feedback on a certain function in the project, the model may recommend specific optimization of the function based on user feedback. If the problem analysis result is that there are a large number of bugs or problems, the model may recommend prioritizing specific problems based on the number and severity of problem reports.

[0104] Through this solution, the big model can be used to timely discover problems in the system and target projects, so that users can optimize the functions of the system and target projects according to the results output by the big model and improve the user experience.

[0105] In an embodiment of the present application, first, in the process of grayscale release of the target project, the system indicator data is monitored in real time, wherein the system indicator data includes dimensional indicator data of several dimensions, and at every first preset time interval, the first target data corresponding to the current release cycle is obtained from the system indicator data, and the corresponding policy adjustment information is determined according to the first target data, and then, the grayscale release policy of the current release cycle is adjusted according to the policy adjustment information to obtain the target grayscale release policy, and in the next release cycle, the target project is grayscale released according to the target grayscale release policy. This solution can realize intelligent dynamic adjustment of the grayscale release policy by real-time monitoring of the system indicator data, thereby improving decision-making efficiency and accuracy.

[0106] See also Figure 2 , is a flow chart of another embodiment of a grayscale publishing method provided in an embodiment of the present application. Figure 2 As shown, the process may include the following steps:

[0107] Step 201: Determine first performance data corresponding to the large model in the current evaluation period at every second preset time interval.

[0108] Step 202: When the first performance data does not meet the preset conditions, obtain second target data corresponding to the current evaluation cycle from the system indicator data.

[0109] Step 203: Use the second target data to update the large model to obtain an updated model, and replace the large model used in the system with the updated model.

[0110] For ease of understanding, steps 201 to 203 are described in a unified manner as follows:

[0111] In practical applications, a period (ie, evaluation period) for performance evaluation of a large model may be preset, for example, one week, one month, etc. Correspondingly, the second preset time interval is the duration corresponding to the period.

[0112] The first performance data is used to characterize the performance of the large model. In the application, the performance of the model can be determined by the difference between the output of the model and the actual result, as well as the performance of the model on key indicators. When the first performance data meets the preset conditions, it means that the performance of the current large model meets expectations and does not need to be updated; when the first performance data does not meet the preset conditions, it means that the performance of the current large model does not meet expectations and needs to be updated.

[0113] Taking the evaluation of the performance of the current big model through A / B testing as an example, first, users are randomly assigned to the experimental group (using artificial grayscale release) and the control group (using the big model as an aid). Then, key performance indicators (such as click-through rate, conversion rate, response time, etc.) are determined and the corresponding data is collected. Finally, the performance of the big model is determined by comparing the key indicators of the experimental group and the control group to see whether they are close to or better than artificial grayscale release. In other words, when the key indicators corresponding to the big model are close to or better than artificial grayscale release, the first performance data meets the preset conditions. Otherwise, the first performance data does not meet the preset conditions.

[0114] Based on this, in the embodiment of the present application, the performance of the large model is evaluated at every second preset time interval, and when its performance does not meet expectations, the data corresponding to the collection time in the current evaluation period is obtained from the system indicator data as the data to be processed, and then the data to be processed is cleaned (such as deleting missing values ​​and invalid values), sorted (such as formatting, that is, unifying the data format), denoised (applying filters to remove noise data) and other pre-processing to obtain the second target data. Thereby ensuring the quality and accuracy of the second target data.

[0115] Then, the model is retrained or fine-tuned through the second target data to update the model's weights and parameters, and, based on the model's performance feedback, hyperparameters such as learning rate and batch size are adjusted to optimize the training process, thereby obtaining an updated model, and without affecting the user experience, the updated model is used to gradually replace the old model to achieve a smooth transition of the model. In this way, the large model used in the system can be made to conform to the real-time situation of the system and the accuracy of the large model output can be improved.

[0116] Furthermore, to ensure the effect of the updated model, in another embodiment of the present application, the method may further include the following steps: performing a performance test on the updated model to obtain corresponding second performance data; when the second performance data is better than the first performance data, performing a step of replacing the large model used in the system with the updated model. In this way, it can be ensured that when the performance of the updated model is better than the large model in the current evaluation cycle, the model replacement step is performed, thereby ensuring that the model version with the best performance is applied in the system.

[0117] Furthermore, in another embodiment of the present application, a feedback mechanism can be added to the system, specifically: the results of model optimization and user feedback are incorporated into the next round of optimization cycle to form a continuous learning and improvement process. In the application, the process and results of each optimization can also be recorded to facilitate tracking the evolution of the model and diagnose problems when necessary.

[0118] For easier understanding, the model update process is described in detail through the following steps:

[0119] Step 1: Prepare the input data (second target data) and target output (grayscale release strategy decision) and divide them into training set and validation set.

[0120] Step 2: Choose a suitable pre-trained model, such as GPT-3.5 or other large language models, and make necessary modifications to adapt to the input and output formats.

[0121] Step 3: Define a suitable loss function to quantify the difference between the model prediction and the actual decision.

[0122] Step 4: Use the training set to fine-tune the model and optimize the model parameters to minimize the loss function.

[0123] Step 5: Evaluate the performance of the updated model on an independent validation set to ensure the model's generalization ability on new data, and perform iterative optimization as needed.

[0124] This solution enables the large model to make more accurate grayscale release decisions based on real-time and historical data.

[0125] Specifically, during fine-tuning, the model will use the input data to predict the target output and adjust it by minimizing the loss function. The loss function will be defined according to the target output of the model. For example, if the target output is a classification problem (such as expansion, reduction, maintenance or rollback), the cross entropy loss function can be used. In actual operation, a composite loss function can be designed that combines the losses of multiple tasks to reflect the importance and priority of different decisions in the system.

[0126] In a large model-based intelligent grayscale deployment system, determining the true situation (i.e., whether to expand, reduce, maintain, or roll back the grayscale release) usually relies on system-defined business rules, performance indicator thresholds, and user feedback. Since it is impossible to know the optimal result of each decision in advance in actual operation, the design of the loss function must rely on historical data, preset business logic, and expert experience.

[0127] In the application, the implementation method of the loss function may include the following steps:

[0128] 1. Set business rules and thresholds: (1) Define indicator thresholds: Set thresholds for each key indicator based on historical data and business needs, such as response time not exceeding a certain value, error rate below a certain percentage, etc. (2) Define business logic: Establish business rules to guide decision-making. For example, if the error rate exceeds the threshold, it triggers a reduction in release or rollback; if user satisfaction is high and the system load is stable, consider expanding the release.

[0129] 2. Use historical data and simulation: (1) Historical data: Analyze historical grayscale release data to learn which indicator changes are associated with successful release strategies. (2) Simulation testing: Simulate grayscale releases in a controlled environment and use simulation results to train and verify model decisions.

[0130] 3. Expert experience and feedback loop: (1) Expert knowledge: The loss function is defined in combination with the knowledge of domain experts. Experts can use their experience to guide how to weigh the importance of different indicators. (2) Feedback loop: The actual published results and user feedback are used as learning signals to continuously adjust the model's loss function and decision logic.

[0131] 4. Design a composite loss function: (1) Multi-task learning: Design a composite loss function that combines the losses of multiple tasks (such as prediction indicators, classification decisions). (2) Weight assignment: Assign different weights to different decision results to reflect the business impact of different decisions. For example, the cost of rolling back may be higher than the cost of expanding the release.

[0132] 5. Simulated labels and soft labels: (1) Simulated labels: In the absence of clear real labels, simulated labels can be generated according to business rules as training targets. (2) Soft labels: Use probability distribution instead of hard classification to express the confidence level of different decisions. The loss function can minimize the difference between the predicted probability distribution and the soft label distribution.

[0133] 6. Model validation and evaluation: (1) Offline validation: Use historical data sets for cross-validation to evaluate the performance of the model on known results. (2) Online evaluation: In the actual grayscale release, use A / B testing or gradual rollout methods to evaluate the decision-making effect of the model.

[0134] Through the above method, even without knowing the true situation, an effective loss function can be designed to guide the training and optimization of the model. Over time, the system can improve the accuracy of its decisions through continuous learning and feedback loops.

[0135] See also Figure 3 , is a flow chart of another embodiment of a grayscale publishing method provided in the present application. Figure 3 As shown, the process may include the following steps:

[0136] Step 301: For each dimensional indicator data, obtain the dimensional indicator range corresponding to the dimensional indicator data.

[0137] Step 302: When the dimension indicator data exceeds the dimension indicator range, trigger an alarm mechanism to handle the abnormal situation indicated by the dimension indicator data.

[0138] From the above description, it can be seen that in the embodiment of the present application, each dimension indicator data can be monitored in real time. Once a dimension indicator data exceeds the preset range (i.e., the dimension indicator range), the system automatically alarms and prompts that manual intervention or model adjustment is required. Thus, problems can be quickly discovered and corresponding optimization measures can be taken to improve user experience and system stability.

[0139] Optionally, the embodiment of the present application also provides an overall processing flow for grayscale release, such as Figure 4 As shown, the specific steps are as follows.

[0140] 1. Data Collection Module

[0141] Implementation method: Deploy data collection scripts to the application client and server to capture user interaction logs and system performance logs; use message queue middleware, such as Kafka, to transmit data to the data storage layer in real time.

[0142] 2. Data preprocessing module

[0143] Implementation method: Cleaning: delete missing values ​​and invalid values; Denoising: apply filters to remove noisy data; Formatting: unify data format

[0144] 3. Large model training module

[0145] Implementation method: Use deep learning frameworks such as TensorFlow or PyTorch; select a suitable model (such as GPT3.5) for fine-tuning.

[0146] 4. Strategy generation and adjustment module

[0147] Implementation method: Generate an initial grayscale release strategy based on model prediction results; adjust the strategy range in real time; include expansion and rollback mechanisms.

[0148] 5. Grayscale adjustment API module

[0149] Implementation method: Adjust the strategies in the system in real time according to the generated grayscale strategy.

[0150] 6. Result evaluation and feedback module

[0151] Implementation method: Use A / B testing method to evaluate the effect; monitor key indicators; collect user feedback to optimize models and strategies.

[0152] This solution can realize intelligent dynamic adjustment of grayscale release strategy through real-time monitoring of system indicator data, thereby improving decision-making efficiency and accuracy.

[0153] Based on the same technical concept, the embodiment of the present application also provides a grayscale publishing device, such as Figure 5 As shown, the device comprises:

[0154] The monitoring module 51 is used to monitor the system indicator data in real time during the grayscale release of the target project, wherein the system indicator data includes dimensional indicator data of several dimensions;

[0155] An acquisition module 52 is used to acquire first target data corresponding to a current release cycle from the system indicator data at every first preset time interval;

[0156] A determination module 53, configured to determine corresponding strategy adjustment information according to the first target data;

[0157] An adjustment module 54, configured to adjust the grayscale release strategy of the current release cycle according to the strategy adjustment information to obtain a target grayscale release strategy;

[0158] The release module 55 is used to perform a grayscale release on the target project according to the target grayscale release strategy in the next release cycle.

[0159] In a possible implementation manner, the determining module is specifically configured to:

[0160] The first target data is input into a pre-trained large model, so that the large model can perform problem analysis and prediction on the grayscale release strategy of the current release cycle based on the first target data, obtain the problem analysis results corresponding to the grayscale release strategy of the current release cycle, and determine the corresponding strategy adjustment information based on the problem analysis results.

[0161] In a possible implementation manner, the device further includes an updating module, configured to:

[0162] Determine the first performance data corresponding to the large model in the current evaluation period at every second preset time interval;

[0163] When the first performance data does not meet the preset conditions, obtaining second target data corresponding to the current evaluation period from the system indicator data;

[0164] The large model is updated using the second target data to obtain an updated model, and the large model used in the system is replaced with the updated model.

[0165] In a possible implementation manner, the update module is further configured to:

[0166] Performing a performance test on the updated model to obtain corresponding second performance data;

[0167] In the case where the second performance data is better than the first performance data, a step of replacing the large model applied in the system with the updated model is performed.

[0168] In a possible implementation manner, the determining module is further configured to:

[0169] When the problem analysis result is that the target project performance is at the first performance level, and the user satisfaction is at the first satisfaction level, the corresponding strategy adjustment information is to expand the grayscale range;

[0170] When the problem analysis result is that the target project performance is at the second performance level, and the user satisfaction is at the second satisfaction level, the corresponding strategy adjustment information is to maintain the existing grayscale range;

[0171] When the problem analysis result is that the target project performance is at the third performance level, or the user satisfaction is at the third satisfaction level, the corresponding strategy adjustment information is to reduce the grayscale range;

[0172] When the problem analysis result is that the performance of the target project is at the fourth performance level, or the user satisfaction is at the fourth satisfaction level, the corresponding policy adjustment information is to roll back and update to the old version corresponding to the target project;

[0173] Among them, the first performance level is better than the second performance level, the second performance level is better than the third performance level, and the third performance level is better than the fourth performance level; the first satisfaction level is better than the second satisfaction level, the second satisfaction level is better than the third satisfaction level, and the third satisfaction level is better than the fourth satisfaction level.

[0174] In a possible implementation manner, the device further includes an alarm module, which is used to:

[0175] For each dimension indicator data, obtain the dimension indicator range corresponding to the dimension indicator data;

[0176] When the dimension indicator data exceeds the dimension indicator range, an alarm mechanism is triggered to handle the abnormal situation indicated by the dimension indicator data.

[0177] In a possible implementation manner, the monitoring module is specifically used to:

[0178] Monitor user experience index data, performance index data, usage index data, conversion rate index data and feedback index data in real time, and use the user experience index data, performance index data, usage index data, conversion rate index data and feedback index data as the system index data.

[0179] Based on the same technical concept, the embodiment of the present application also provides an electronic device, such as Figure 6As shown, it includes a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0180] Memory 113, used for storing computer programs;

[0181] The processor 111 is used to execute the program stored in the memory 113 to implement the following steps:

[0182] During the grayscale release of the target project, real-time monitoring of system indicator data, wherein the system indicator data includes dimensional indicator data of several dimensions;

[0183] At every first preset time interval, obtaining first target data corresponding to the current release cycle from the system indicator data;

[0184] Determining corresponding strategy adjustment information according to the first target data;

[0185] Adjust the grayscale release strategy of the current release cycle according to the strategy adjustment information to obtain a target grayscale release strategy;

[0186] In the next release cycle, the target project is released in grayscale according to the target grayscale release strategy.

[0187] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0188] The communication interface is used for communication between the above electronic device and other devices.

[0189] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0190] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0191] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned grayscale release methods are implemented.

[0192] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any of the grayscale publishing methods in the above embodiments.

[0193] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0194] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0195] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0196] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A grayscale release method, characterized in that: The method comprises: During the grayscale release of the target project, real-time monitoring of system indicator data, wherein the system indicator data includes dimensional indicator data of several dimensions; At every first preset time interval, obtaining first target data corresponding to the current release cycle from the system indicator data; Determining corresponding strategy adjustment information according to the first target data; Adjust the grayscale release strategy of the current release cycle according to the strategy adjustment information to obtain a target grayscale release strategy; In the next release cycle, the target project is released in grayscale according to the target grayscale release strategy.

2. The method according to claim 1, characterized in that: The determining corresponding strategy adjustment information according to the first target data includes: The first target data is input into a pre-trained large model, so that the large model can perform problem analysis and prediction on the grayscale release strategy of the current release cycle based on the first target data, obtain the problem analysis results corresponding to the grayscale release strategy of the current release cycle, and determine the corresponding strategy adjustment information based on the problem analysis results.

3. The method according to claim 2, characterized in that The method further comprises: Determine the first performance data corresponding to the large model in the current evaluation period at every second preset time interval; When the first performance data does not meet the preset conditions, obtaining second target data corresponding to the current evaluation period from the system indicator data; The large model is updated using the second target data to obtain an updated model, and the large model used in the system is replaced with the updated model.

4. The method according to claim 3, characterized in that The method further comprises: Performing a performance test on the updated model to obtain corresponding second performance data; In the case where the second performance data is better than the first performance data, a step of replacing the large model applied in the system with the updated model is performed.

5. The method according to claim 2, characterized in that: The determining corresponding strategy adjustment information according to the problem analysis result includes: When the problem analysis result is that the target project performance is at the first performance level, and the user satisfaction is at the first satisfaction level, the corresponding strategy adjustment information is to expand the grayscale range; When the problem analysis result is that the target project performance is at the second performance level, and the user satisfaction is at the second satisfaction level, the corresponding strategy adjustment information is to maintain the existing grayscale range; When the problem analysis result is that the target project performance is at the third performance level, or the user satisfaction is at the third satisfaction level, the corresponding strategy adjustment information is to reduce the grayscale range; When the problem analysis result is that the performance of the target project is at the fourth performance level, or the user satisfaction is at the fourth satisfaction level, the corresponding policy adjustment information is to roll back and update to the old version corresponding to the target project; Among them, the first performance level is better than the second performance level, the second performance level is better than the third performance level, and the third performance level is better than the fourth performance level; the first satisfaction level is better than the second satisfaction level, the second satisfaction level is better than the third satisfaction level, and the third satisfaction level is better than the fourth satisfaction level.

6. The method according to claim 1, characterized in that The method further comprises: For each dimension indicator data, obtain the dimension indicator range corresponding to the dimension indicator data; When the dimension indicator data exceeds the dimension indicator range, an alarm mechanism is triggered to process the abnormal situation indicated by the dimension indicator data.

7. The method according to claim 1, characterized in that The real-time monitoring system indicator data includes: Monitor user experience index data, performance index data, usage index data, conversion rate index data and feedback index data in real time, and use the user experience index data, performance index data, usage index data, conversion rate index data and feedback index data as the system index data.

8. A grayscale publishing device, characterized in that: The device comprises: A monitoring module is used to monitor system indicator data in real time during the grayscale release of the target project, wherein the system indicator data includes dimensional indicator data of several dimensions; An acquisition module, configured to acquire first target data corresponding to a current release cycle from the system indicator data at every first preset time interval; A determination module, configured to determine corresponding strategy adjustment information according to the first target data; An adjustment module, used to adjust the grayscale release strategy of the current release cycle according to the strategy adjustment information to obtain a target grayscale release strategy; The release module is used to perform a grayscale release on the target project according to the target grayscale release strategy in the next release cycle.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the grayscale release method described in any one of claims 1 to 7 when executing the program stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the grayscale release method according to any one of claims 1 to 7 is implemented.

Images