Deployment effect evaluation method and device, electronic equipment and readable storage medium

Abstract

The disclosure provides a deployment effect evaluation method and device, electronic equipment and readable storage medium, relates to the technical field of computers, and particularly to the technical field of continuous deployment. The specific implementation scheme is: obtaining change related information corresponding to a current code change operation of a to-be-evaluated code module in a current deployment stage, and / or current alarm information of the to-be-evaluated code module in the current deployment stage, the to-be-evaluated code module being a code module whose current alarm information satisfies a preset alarm condition; and determining the deployment effect of the to-be-evaluated code module based on the change related information and / or the current alarm information. Based on the scheme, the deployment effect of the code module can be automatically and effectively evaluated, which helps to reduce the degree of dependence on manpower and improve processing efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to the field of continuous deployment technology. Specifically, this disclosure relates to a deployment effectiveness evaluation method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] Continuous deployment is the process of automatically deploying code after integration testing to the production environment based on continuous integration technology. Continuous deployment usually breaks down the deployment process into multiple deployment phases, and the deployment effect is verified after the deployment of each phase is completed.

[0003] In existing solutions, manual assistance is generally required to assess the deployment effectiveness. Summary of the Invention

[0004] To address at least one of the aforementioned deficiencies, this disclosure provides a deployment effectiveness evaluation method, apparatus, electronic device, and readable storage medium.

[0005] According to a first aspect of this disclosure, a method for evaluating deployment effectiveness is provided, the method comprising:

[0006] Get the change-related information corresponding to the current code change operation performed by the code module to be evaluated in the current deployment phase, and / or the current alarm information of the code module to be evaluated in the current deployment phase. The code module to be evaluated is the code module whose current alarm information meets the preset alarm conditions.

[0007] Based on change-related information and / or current alarm information, determine the deployment effect of the code module to be evaluated.

[0008] According to a second aspect of this disclosure, a deployment effect evaluation apparatus is provided, the apparatus comprising:

[0009] The information acquisition module is used to acquire change-related information corresponding to the current code change operation performed by the code module to be evaluated in the current deployment phase, and / or the current alarm information of the code module to be evaluated in the current deployment phase. The code module to be evaluated is a code module whose current alarm information meets the preset alarm conditions.

[0010] The deployment effectiveness evaluation module is used to determine the deployment effectiveness of the code module to be evaluated based on change-related information and / or current alarm information.

[0011] According to a third aspect of this disclosure, an electronic device is provided, the electronic device comprising:

[0012] At least one processor; and

[0013] A memory communicatively connected to at least one of the aforementioned processors; wherein,

[0014] The memory stores instructions that can be executed by at least one processor, which enables the at least one processor to perform the deployment effect evaluation method.

[0015] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to perform the above-described deployment effect evaluation method.

[0016] According to a fifth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described deployment effect evaluation method.

[0017] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0018] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0019] Figure 1 This is a flowchart illustrating a deployment effect evaluation method provided in an embodiment of this disclosure;

[0020] Figure 2 This is an architecture diagram of a deployment effect evaluation system provided in an embodiment of this disclosure;

[0021] Figure 3 , Figure 4 This is a trend chart showing the changes in relevant indicators of the deployment effect evaluation system provided in this embodiment during operation;

[0022] Figure 5 This is a schematic diagram of the structure of a deployment effect evaluation device provided in an embodiment of this disclosure;

[0023] Figure 6 This is a block diagram of an electronic device used to implement the deployment effect evaluation method of the embodiments of this disclosure. Detailed Implementation

[0024] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0025] In related technologies, after the deployment of the code module is completed, it is generally necessary to test the deployed code module (e.g., using A / B testing). Developers then use the test results to help judge the deployment effect. This method is highly dependent on manpower and has low processing efficiency.

[0026] The deployment effect evaluation method, apparatus, electronic device, and readable storage medium provided in this disclosure are intended to solve at least one of the above-mentioned technical problems of the prior art.

[0027] Figure 1 This illustration shows a flowchart of a deployment effect evaluation method provided in an embodiment of the present disclosure, such as... Figure 1 As shown, the method can mainly include:

[0028] Step S110: Obtain change-related information corresponding to the current code change operation performed by the code module to be evaluated in the current deployment phase, and / or the current alarm information of the code module to be evaluated in the current deployment phase, wherein the code module to be evaluated is a code module whose current alarm information meets the preset alarm conditions.

[0029] Step S120: Based on change-related information and / or current alarm information, determine the deployment effect of the code module to be evaluated.

[0030] In continuous deployment, the deployment process is typically broken down into multiple deployment phases, and the deployment effect is verified after each deployment phase is completed.

[0031] Code change operations involve deploying the modified code. Code change operations can include modifications to program code, configuration information, and dictionaries.

[0032] Once code changes to the code modules are completed within the current deployment phase, each modified code module can be tested, and the test results can be obtained. The test results may include alert information generated by the code modules during the testing process.

[0033] For ease of description, the code changes performed on the code module during the current deployment phase will be referred to as "current code change operations" and the alarm information generated by the code module during the current deployment phase will be referred to as "current alarm information".

[0034] In this embodiment of the disclosure, the current alarm information can reflect to some extent the poor deployment effect of the code module in the current deployment stage. Alarm conditions can be set, and the current alarm information of the code module can be statistically analyzed to obtain the statistical results of the alarm information. When the statistical results of the alarm information meet the alarm conditions, it indicates that the code module may have a poor deployment effect in the current deployment stage, and it can be regarded as a module to be evaluated, and its deployment effect can be evaluated in the future.

[0035] As an example, alarm information can include alarm logs. When the alarm log contains a fatal error, it indicates that a serious error may have occurred in the code module, which may lead to poor deployment of the code module. The alarm condition can be set to include fatal-level logs in the alarm information.

[0036] In this embodiment of the disclosure, among all code modules that have undergone code change operations during the current deployment phase, code modules that meet the alarm conditions in the current alarm information can be selected, that is, code modules that may have poor deployment effects during the current deployment phase can be selected as modules to be evaluated, and the deployment effect of the modules to be evaluated during the current deployment phase can be evaluated.

[0037] In this embodiment of the disclosure, the deployment effect can reflect the running effect of the code module after the code change operation in the current deployment stage is completed.

[0038] In this embodiment, the change-related information refers to information related to code change operations in the module to be evaluated. By accessing the change-related information, the specific details of the code changes occurring in the module to be evaluated can be obtained. Similarly, by accessing the current alarm information, the specific details of the alarms that have occurred can be obtained. Therefore, based on the change-related information and / or the current alarm information, the deployment effect of the code module to be evaluated can be accurately and effectively assessed.

[0039] The method provided in this disclosure determines the deployment effect of a code module to be evaluated by identifying code modules whose current alarm information meets preset alarm conditions, based on change-related information corresponding to the current code change operations performed by the code module to be evaluated during the current deployment phase, and / or the current alarm information of the code module to be evaluated during the current deployment phase. Based on this solution, the deployment effect of code modules can be automatically and effectively evaluated, helping to reduce reliance on human labor and improve processing efficiency.

[0040] In this embodiment, if the deployment effectiveness of the code modules to be evaluated is directly assessed manually, it will result in significant manpower costs due to the large number of modules typically being evaluated. While increasing the alert level can improve the accuracy of the evaluated code modules and reduce their number, thus improving processing efficiency, this approach may lead to a decrease in recall. This solution achieves improved processing efficiency by automatically and effectively evaluating the deployment effectiveness of the modules to be evaluated.

[0041] As an example, deployment results can include good and poor deployment results. When the deployment result is good, it means that the code module to be evaluated is performing well and can proceed to the next deployment stage. When the deployment result is poor, it means that the code module to be evaluated is not performing well, and in this case, the code module that has undergone code change operations in the current deployment stage should be rolled back.

[0042] In this embodiment, the current alarm information can be obtained by crawling alarm logs from the device where the code module to be evaluated is deployed. The advantage of this method is its wide applicability. When the central control system and the system where the code module to be evaluated is deployed are on the same platform, the alarm logs can be crawled from the system deployed by the code module to be evaluated through the application programming interface (API) of the central control system. However, this method suffers from a severe dependence on the stability of the API; when the system where the code module to be evaluated is deployed is under heavy load, logging into that system has an extremely high failure rate.

[0043] In this embodiment of the disclosure, the current alarm information can also be obtained by subscribing to the alarm logs reported by the code module to be evaluated. Subscribing to the relevant logs of the code module to be evaluated online in real time, such as using a real-time streaming data subscription like Bigpipe, offers the advantage of high timeliness, allowing for rapid acquisition of alarm information after an alarm is generated. However, this method has the disadvantage of consuming a lot of resources, and therefore is generally used in evaluation modules that cannot undergo high-frequency iterations.

[0044] In practice, the two methods mentioned above can be used in combination to obtain alarm information. In code modules that cannot be changed frequently, the coverage of alarm information can be improved by subscribing to online alarm logs. In other code modules, alarm logs can be crawled from the devices where the code modules to be evaluated are deployed.

[0045] In one optional approach of this disclosure, the deployment effect of the code module to be evaluated is determined based on change-related information and / or current alarm information, including:

[0046] Based on change-related information and / or current alarm information, the target evaluation strategy is matched from the preset evaluation strategies. The evaluation strategy is pre-configured with corresponding sub-effect evaluation indicators.

[0047] Based on the target sub-effect evaluation indicators corresponding to the target evaluation strategy, the effect evaluation indicators of the code module to be evaluated are determined. The effect evaluation indicators are used to characterize the deployment effect of the code module to be evaluated.

[0048] In this embodiment of the disclosure, the evaluation strategy can be formulated based on historical experience in manually evaluating deployment effectiveness. The evaluation strategy can be set from multiple data dimensions to ensure the effectiveness of the deployment effectiveness evaluation.

[0049] Evaluation strategies can be pre-configured with corresponding sub-effect evaluation metrics, which characterize the degree of impact of the evaluation strategy on the deployment effect. As an example, the higher the value of the sub-effect evaluation metric, the greater its impact on the deployment effect.

[0050] The sub-effect evaluation metrics can have either a positive or a negative impact on deployment effectiveness. A positive impact can lead to better deployment results, while a negative impact can lead to poor deployment results.

[0051] In this embodiment of the disclosure, when evaluating the deployment effect of a certain code module to be evaluated, only a portion of the evaluation strategies may be applicable to the code module to be evaluated. These partial evaluation strategies can be used as the target evaluation strategies, and the effect evaluation indicators of the code module to be evaluated can be determined according to the target sub-effect evaluation indicators corresponding to the target evaluation strategies.

[0052] In this embodiment of the disclosure, the effect evaluation index is used to characterize the deployment effect of the code module to be evaluated.

[0053] As an example, the performance evaluation metric can be a numerical value, where a higher value indicates a better deployment effect, and a lower value indicates a worse deployment effect.

[0054] As an example, each sub-effect evaluation index can be pre-configured with different weights based on the degree of impact of the evaluation strategy. The target sub-effect evaluation indexes are then weighted to obtain the effect evaluation index.

[0055] In one optional embodiment of this disclosure, the evaluation strategy includes at least two items, and the evaluation strategy corresponds to a preset matching order. Based on change-related information and / or current alarm information, a target evaluation strategy is matched from the preset evaluation strategies, including:

[0056] According to the matching order, based on the change-related information and / or the current alarm information, the system sequentially matches each preset evaluation strategy until the target sub-effect evaluation index corresponding to the matched target evaluation strategy can be used to determine the effect evaluation index that meets the preset indicator conditions.

[0057] In this embodiment of the disclosure, the matching order corresponding to each evaluation strategy can be pre-configured. The matching order indicates the order in which each evaluation strategy is matched when the target evaluation strategy is obtained.

[0058] In this embodiment of the disclosure, the matching order can be set according to the time spent matching each evaluation strategy, the amount of processing resources consumed in matching, and the degree of impact of the evaluation strategy on the deployment effect.

[0059] As an example, for evaluation strategies that consume less time and resources during matching and have a greater impact on deployment effectiveness, a higher matching order can be configured.

[0060] In this embodiment of the disclosure, during the process of matching each evaluation strategy sequentially according to the matching order, the effect evaluation index can be determined in real time based on the target sub-effect evaluation index corresponding to the matched target evaluation strategy. Specifically, whenever a target evaluation strategy is determined, the effect evaluation index can be recalculated based on its corresponding target sub-effect evaluation index.

[0061] During the process of updating the performance evaluation metrics as they are matched with the target evaluation strategies, when a metric meets the preset criteria, it indicates that the deployment effect can be determined based on the matched target evaluation strategies, and there is no need to continue matching evaluation strategies or to match all target evaluation strategies. At this point, the current deployment performance evaluation process can be terminated to save computing power, reduce processing pressure, and improve processing efficiency.

[0062] As an example, there are 10 evaluation strategies, numbered 1, 2, ..., 10 according to their matching order. Starting with strategy 1, each strategy is matched sequentially, and the performance evaluation metrics are recalculated based on the target sub-performance evaluation metrics corresponding to the matched strategies. When strategy 5 is matched, strategies 1, 2, and 5 are identified as target evaluation strategies. If the performance evaluation metrics calculated based on the target sub-performance evaluation metrics corresponding to strategies 1, 2, and 5 meet the preset indicator conditions, the current deployment performance evaluation process can end, and matching strategies 6-10 can cease.

[0063] As an example, two preset indicator conditions can be set: one is that when the effect evaluation indicator is greater than A, the deployment effect is judged to be good; the other is that when the effect evaluation indicator is less than B, the deployment effect is judged to be poor.

[0064] In one alternative embodiment of this disclosure, changing the relevant information includes:

[0065] The start and end times of the current code change operation;

[0066] The current equipment room corresponding to the current deployment phase;

[0067] The changes made in the current code modification operation;

[0068] The current device that is displaying the current alarm message.

[0069] For ease of description, the equipment room corresponding to the current deployment phase will be referred to as the current equipment room, and the device that is currently displaying the alarm information will be referred to as the current device.

[0070] In this embodiment of the disclosure, the change-related information may also include, but is not limited to, the platform that initiated the code change operation; the type of code change, the current deployment stage, the deployment stage before the current deployment stage, and the deployment stage after the current deployment stage, etc., to meet various possible usage requirements in the subsequent processing.

[0071] In one optional embodiment of this disclosure, the evaluation strategy includes a first evaluation strategy, wherein the current alarm information or alarm information similar to the current alarm information does not appear after the current code change operation is completed, and the sub-effect evaluation index corresponding to the first evaluation strategy is the first sub-effect evaluation index, which has a positive impact on the deployment effect of the code module to be evaluated.

[0072] In this embodiment of the disclosure, if the current alarm message or an alarm message similar to the current alarm message does not appear after the completion of the current code change operation, it indicates that the appearance of the current alarm message is highly likely unrelated to the code change operation within the current deployment phase. The first evaluation strategy can be configured such that the current alarm message or an alarm message similar to the current alarm message does not appear after the completion of the current code change operation. The first sub-effect evaluation index corresponding to the first evaluation strategy has a positive impact on the deployment effect of the code module to be evaluated; that is, the first sub-effect evaluation index can make the deployment effect tend to be better.

[0073] Optionally, when matching for the first evaluation strategy, the embodiments of this disclosure may specifically adopt the following methods:

[0074] Based on change-related information and / or current alarm information, a target evaluation strategy is matched from the preset evaluation strategies, including at least one of the following:

[0075] If, based on the start and end times of the current code change operation, the current equipment room corresponding to the current deployment stage, and the current alarm information, it is determined that the current alarm information in the current equipment room appeared before the current code change operation, and the current alarm information appeared multiple times after the current code change operation was completed, then the first evaluation strategy will be determined as the target evaluation strategy.

[0076] If, based on the start and end times of the current code change operation, the current equipment room corresponding to the current deployment stage, and the current alarm information, it is determined that a first alarm information similar to the current alarm information appeared in the current equipment room within a preset time period before the current code change operation, then the first evaluation strategy will be determined as the target evaluation strategy.

[0077] If, based on the current equipment room and current alarm information corresponding to the current deployment phase, it is determined that a second alarm information similar to the current alarm information has appeared in an equipment room other than the current equipment room, then the first evaluation strategy will be determined as the target evaluation strategy.

[0078] In this embodiment of the disclosure, a first evaluation strategy can be matched based on the start and end time of the current code change operation, the current equipment room corresponding to the current deployment stage, and the current alarm information.

[0079] Specifically, if the current alarm message appears in the current equipment room before the current code change operation, and the current alarm message appears multiple times after the current code change operation is completed, it can be considered that the appearance of the current alarm message is unrelated to the code change operation in the current deployment phase. In this case, the first evaluation strategy can be determined as the target evaluation strategy.

[0080] Specifically, if a first alarm message similar to the current alarm message appears in the current equipment room within a preset time period before the current code change operation, it can be considered that the appearance of the current alarm message is unrelated to the code change operation in the current deployment phase. In this case, the first evaluation strategy can be determined as the target evaluation strategy.

[0081] In this embodiment of the disclosure, since each deployment stage corresponds to a different equipment room, and the deployment time of different deployment stages is generally different, the first evaluation strategy can be matched according to the current equipment room corresponding to the current deployment stage and the current alarm information.

[0082] Specifically, if a second alarm message similar to the current alarm message appears in a data center other than the current data center, it is considered that the second alarm message similar to the current alarm message did not appear after the current code change operation was completed during the current deployment phase. In this case, the appearance of the second alarm message can be considered to be unrelated to the current code change operation, and the first evaluation strategy can be determined as the target evaluation strategy.

[0083] As an example, the similarity of alarm messages can be determined based on the proportion of similar strings in the alarm messages, such as if the proportion of similar strings is greater than a certain predetermined proportion.

[0084] In one optional embodiment of this disclosure, the evaluation strategy includes a second evaluation strategy, wherein the impact of the current alarm information meets preset impact conditions, and the sub-effect evaluation index corresponding to the second evaluation strategy is the second sub-effect evaluation index, which has a positive impact on the deployment effect of the code module to be evaluated.

[0085] In this embodiment, when the current alarm information has a minor impact on the operation of each code module within the current deployment phase, it can still be considered to have a positive impact on the deployment effect within the current deployment phase in order to improve processing efficiency. A preset impact condition can be configured; when the current alarm information meets the preset impact condition, it indicates that the current alarm information has a minor impact on the operation of each code module within the current deployment phase. The satisfaction of the preset impact condition with the impact of the current alarm information is used as the second evaluation strategy. The second sub-effect evaluation index corresponding to the second evaluation strategy has a positive impact on the deployment effect of the code module to be evaluated, that is, the second sub-effect evaluation index can make the deployment effect tend to be better.

[0086] Optionally, when matching for the second evaluation strategy, the following methods may be specifically adopted in this embodiment of the disclosure.

[0087] Based on change-related information and / or current alarm information, a target evaluation strategy is matched from the preset evaluation strategies, including at least one of the following:

[0088] If, based on the current equipment room and current alarm information corresponding to the current deployment phase, it is determined that the number of devices in the current equipment room that have the current alarm information meets the first preset quantity condition, then the second evaluation strategy will be determined as the target evaluation strategy.

[0089] If, based on the current equipment room and current alarm information corresponding to the current deployment phase, it is determined that the number of current equipment rooms that have previously issued current alarm information does not meet the second preset quantity condition, or the number of devices in the current equipment room that have issued current alarm information does not meet the third preset quantity condition, then the second evaluation strategy will be determined as the target evaluation strategy.

[0090] In this embodiment of the disclosure, a second evaluation strategy can be matched based on the current equipment room and current alarm information corresponding to the current deployment stage.

[0091] Specifically, a first quantity condition can be pre-configured for the number of devices in the current equipment room that have the current alarm information. When the number of devices in the current equipment room that have the current alarm information meets the first preset quantity condition, it means that the current alarm information has little impact on the operation of each code module in the current deployment phase. At this time, the second evaluation strategy can be determined as the target evaluation strategy.

[0092] As an example, the first preset quantity condition can be that the number of devices in the current equipment room that have the current alarm information is no more than 5% of the total number of devices in the current equipment room.

[0093] Specifically, a second preset quantity condition can be configured for the number of current equipment rooms that have previously issued the current alarm message, and a third preset quantity condition can be configured for the number of devices in the current equipment room that have issued the current alarm message. If the number of current equipment rooms that have previously issued the current alarm message meets the second preset quantity condition, it indicates that the proportion of devices issuing the current alarm message in the current equipment room is relatively high. If the number of devices in the current equipment room that have issued the current alarm message meets the third preset quantity condition, it indicates that there are significant differences in the proportion of devices issuing the current alarm message in different current equipment rooms.

[0094] When the number of current equipment rooms that have previously issued current alarm information meets the second preset quantity condition, and the number of devices in the current equipment room that have issued current alarm information meets the third preset quantity condition, it indicates that the proportion of current alarm information occurring in the current equipment room is relatively high, and there are significant differences in the proportion of devices with current alarm information occurring in different current equipment rooms. In this case, it can be considered that the current alarm information has a significant impact on the operation of each code module in the current deployment phase. Correspondingly, when the number of current equipment rooms that have previously issued current alarm information does not meet the second preset quantity condition, or the number of devices in the current equipment room that have issued current alarm information does not meet the third preset quantity condition, it can be considered that the current alarm information has a relatively small impact on the operation of each code module in the current deployment phase. In this case, the second evaluation strategy can be determined as the target evaluation strategy.

[0095] As an example, the second preset quantity condition can be that the number of current equipment rooms that have previously received the current alarm information accounts for no less than 80% of the total number of current equipment rooms. The third preset quantity condition can be that the proportion of the number of devices receiving the current alarm information in each current equipment room to the total number of devices in that room is different.

[0096] In one optional embodiment of this disclosure, the evaluation strategy includes a third evaluation strategy. The third evaluation strategy is that the similarity between the current alarm information and the alarm information in the historical evaluation record of the first target meets a first preset similarity condition, or the similarity between the change content corresponding to the current code change operation and the change content in the historical evaluation record of the first target meets a second preset similarity condition. The historical evaluation record of the first target is a historical evaluation record in which the deployment effect of the corresponding code module meets a preset positive effect condition. The sub-effect evaluation index corresponding to the third evaluation strategy is a third sub-effect evaluation index, and the third sub-effect evaluation index has a positive impact on the deployment effect of the code module to be evaluated.

[0097] In this embodiment of the disclosure, the historical evaluation records generally contain manual evaluation results of the deployment effect. Referring to these manual evaluation results in the historical evaluation records allows for an effective evaluation of the deployment effect of the code module to be evaluated. Therefore, a historical knowledge base can be constructed based on the historical evaluation records, and the deployment effect of the code module can be evaluated by accessing the historical database.

[0098] In this embodiment of the disclosure, the manual evaluation results in the historical evaluation records may have a positive impact on the deployment effect or a negative impact on the deployment effect.

[0099] For example, the results of a manual evaluation can include a manually marked "passable," indicating a positive impact on the deployment effect, or a manually marked "failable," indicating a negative impact on the deployment effect.

[0100] For example, the results of manual evaluation may include manually labeled alarm reasons. When the manually labeled alarm reasons are low-risk reasons, such as equipment reasons, traffic reasons, or occasional reasons, since these low-risk reasons have little impact on the operation of the deployed code modules, it can still be considered that the alarm reasons are low-risk reasons and have a positive impact on the deployment effect.

[0101] In this embodiment, historical evaluation records that have a positive impact on deployment effectiveness can be used as first target historical evaluation records, and a third evaluation strategy can be matched based on these first target historical evaluation records. When the current code change operation has a certain similarity to the first target historical evaluation record, the deployment effectiveness of the code module to be evaluated is evaluated by referring to the first target historical evaluation record.

[0102] Specifically, a first preset similarity condition can be configured for alarm information, and a second preset similarity condition can be configured for changed content. If the similarity between the current alarm information and the alarm information in the first target's historical evaluation record meets the first preset similarity condition, or if the similarity between the changed content corresponding to the current code change operation and the changed content in the first target's historical evaluation record meets the second preset similarity condition, it indicates that the current code change operation and the first target's historical evaluation record have a certain degree of similarity. The deployment effect of the code module to be evaluated can be evaluated by referring to the first target's historical evaluation record. The third sub-effect evaluation index corresponding to the third evaluation strategy is set to have a positive impact on the deployment effect of the code module to be evaluated, that is, the third sub-effect evaluation index can make the deployment effect tend to be better.

[0103] As an example, alarm messages and changes typically conform to specific data formats. These formats can be used to split alarm messages and changes into strings. A first preset similarity condition is that the proportion of identical strings in the alarm messages is greater than a first preset value. A second preset similarity condition is that the proportion of identical strings in the changes is greater than a second preset value.

[0104] As an example, to ensure the validity of historical evaluation records, the deployment effects of the code module to be evaluated can be assessed by referring to historical evaluation records from a nearby time period. For example, the nearby time period could be the 24 hours prior to the current code change operation.

[0105] In one optional embodiment of this disclosure, the evaluation strategy includes a fourth evaluation strategy. The fourth evaluation strategy is that the current alarm information or the device identifier of the current device belongs to a preset whitelist. The sub-effect evaluation index corresponding to the fourth evaluation strategy is the fourth sub-effect evaluation index, which has a positive impact on the deployment effect of the code module to be evaluated.

[0106] In this embodiment of the disclosure, the whitelist can be configured manually so that the information contained in the whitelist has a positive impact on the deployment effect.

[0107] Specifically, the information included in the whitelist can include the name and content of the alarm information, device identifier, etc. The fourth evaluation strategy can be configured so that the current alarm information or the device identifier of the current device is within the preset whitelist. The fourth sub-effect evaluation index corresponding to the fourth evaluation strategy has a positive impact on the deployment effect of the code module to be evaluated; that is, the fourth sub-effect evaluation index can make the deployment effect tend to be better.

[0108] As an example, if at least one of the following conditions is met, it can be determined that the fourth evaluation strategy has been matched as the target evaluation strategy: the name of the current alarm information is in the whitelist, part of the alarm content contained in the current alarm information is in the whitelist, or the device identifier of the current device is in the whitelist.

[0109] As an example, to ensure the effectiveness of the whitelist, an expiration period can be set for the whitelist, such as setting the expiration period to 24 hours.

[0110] In one optional embodiment of this disclosure, the evaluation strategy includes a fifth evaluation strategy. The fifth evaluation strategy is that the current alarm information contains specified alarm content, or the code change operation corresponding to the code module to be evaluated contains specified change content. Both the specified alarm content and the specified change content are determined based on the historical evaluation records of the second target. The historical evaluation records of the second target are historical evaluation records in which the deployment effect of the corresponding code module meets the preset negative effect conditions. The sub-effect evaluation index corresponding to the fifth evaluation strategy is the fifth sub-effect evaluation index, and the fifth sub-effect evaluation index has a negative impact on the deployment effect of the code module to be evaluated.

[0111] In this embodiment of the disclosure, the manual evaluation results in the historical evaluation records may have a negative impact on the deployment effect. The historical evaluation records with a negative impact on the deployment effect can be used as the second target historical evaluation records. Based on the analysis of the second target historical evaluation records, the specified alarm content or specified change content can be determined. The specified alarm content or specified change content can reflect the poor deployment effect of its corresponding code module.

[0112] Specifically, alarm information corresponding to multiple historical evaluation records of the second objective can be analyzed. When alarm content appears multiple times in these alarm messages, it can be considered that these alarm contents reflect the poor deployment effect of the corresponding code module, and these alarm contents can be designated as alarm contents.

[0113] For example, alarm information includes alarm logs. If certain page identifiers or A / B test identifiers appear multiple times in the alarm logs, these page identifiers or A / B test identifiers can be used as specified alarm content.

[0114] In this embodiment of the disclosure, the changes corresponding to the historical evaluation records of the second target can be analyzed.

[0115] As an example, the static code corresponding to the configuration information in the changed content can be parsed to obtain the call relationship tree, the nesting relationship of the configuration in the call relationship tree can be obtained, and the changed content can be bound to the alarm information according to the nesting relationship, that is, the changed content that triggers the alarm can be determined, and this part of the bound changed content can be used as the specified changed content.

[0116] As another example, the terminology in the changed content can be analyzed. Inside the code, the terminology is generally used as a variable to pass parameters. Call chain analysis can be performed, and alarm information can be bound to the changed parameters based on the call chain analysis results. The terms in the terminology corresponding to these bound parameters can be used as the specified changed content.

[0117] In this embodiment of the disclosure, the fifth evaluation strategy can be configured such that the current alarm information includes specified alarm content, or that the change content corresponding to the code change operation of the code module to be evaluated includes specified change content. The fifth sub-effect evaluation index corresponding to the fifth evaluation strategy has a negative impact on the deployment effect of the code module to be evaluated, that is, the fifth sub-effect evaluation index can make the deployment effect tend to be poor.

[0118] In this embodiment of the disclosure, the data accumulated in the historical knowledge base can be reviewed manually on a regular basis. For some alarm information that has little impact on the running effect of the code module, the importance of these alarm information in the alarm conditions can be reduced by adjusting the alarm conditions. This avoids including code modules that only generate these alarm information in the modules to be evaluated, thus avoiding the evaluation of the deployment effect of these code modules, reducing the workload of evaluation and improving processing efficiency.

[0119] In this embodiment of the disclosure, when manually annotating historical evaluation records, annotations of alarm reasons can be added, which helps to enrich subsequent evaluation strategies and improve the accuracy and effectiveness of the evaluation.

[0120] In this embodiment of the disclosure, online code changes may be initiated on multiple change platforms. The historical knowledge base obtained by manually annotating a change platform can be extended to multiple change platforms to improve the accuracy and effectiveness of evaluating deployment effects.

[0121] As an example, Figure 2 The diagram shows an architecture diagram of a deployment effect evaluation system provided in an embodiment of this disclosure.

[0122] like Figure 2As shown, experimental deployment refers to deploying code modules within an experimental environment. Lightweight deployment, or lightweight application deployment, is one available deployment method in this system, but it's not used in this example. Module rollout completes the system by adding all modules. Data release involves releasing the code corresponding to the system to be deployed. Canary release (also known as gray-scale release) is a deployment method that allows for a smooth transition between black and white phases. BP stands for Business Process. Single-datacenter and multi-datacenter deployments refer to the business process where code modules are first deployed in a single datacenter and then in multiple datacenters. After the gray-scale release is completed, the code is deployed to the business retrieval system, which serves as an example of the system to be deployed.

[0123] CORE and FATAL log information, with CORE and FATAL logs being alarm logs generated in the commercial retrieval system or during the canary release phase after deployment. The code modules that generate CORE and FATAL logs are considered as modules to be evaluated, and their deployment effectiveness needs to be assessed.

[0124] Alarm interruption means that after checking alarm logs such as CORE logs and FATAL logs, the deployment process of the corresponding code module can be interrupted, and the deployment effect in the current deployment phase can be evaluated.

[0125] The intelligent diagnostic knowledge base, which contains the evaluation strategies in this example, is used to evaluate the deployment effectiveness of the modules to be evaluated.

[0126] Automatic recovery means that when the deployment effect of the module to be evaluated is found to be good, it can be restored to a normal running state, allowing changes to be made in subsequent deployment phases.

[0127] A labeling platform is a platform used for manual labeling of deployment effects. Administrators (OPERATOR, OP) and engineers can perform manual labeling operations.

[0128] The deployment effect evaluation system provided in this disclosure can automatically evaluate deployment effects, reducing reliance on manual evaluation and helping to improve processing efficiency. As an example, Figure 3 , Figure 4 The figure shows a trend graph of the changes in relevant indicators during the operation of the deployment effect evaluation system provided in the embodiments of this disclosure.

[0129] like Figure 3As shown, the invocation volume refers to the number of target evaluation strategies matched. The exemption rate is the percentage of code modules evaluated as having good deployment performance out of all code modules to be evaluated. The exemption accuracy rate is the percentage of code modules evaluated as having good deployment performance that actually still have good deployment performance. Actual deployment performance can be obtained through manual annotation.

[0130] Figure 3 In the bar chart, the bars represent the number of calls, located at... Figure 3 The topmost curve represents the exemption accuracy rate, while the curve below it represents the exemption rate.

[0131] Figure 3 The values ​​0, 20, 40, 60, 80, 100, and 120 on the left side represent the exemption accuracy rate and the exemption rate coordinates. Figure 3 The values ​​1000, 600, 3000, 4000, 5000, and 6000 on the right side of the middle section are the coordinates of the call volume. Figure 3 The time coordinates of the deployment effect evaluation system on the lower middle side (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12) represent the months in operation.

[0132] Depend on Figure 3 As shown in the dynamic relationship between call volume, exemption rate, and exemption accuracy, the exemption rate gradually increases as the deployment effect evaluation system runs for longer, while the exemption accuracy remains at a high level.

[0133] like Figure 4 As shown, the number of alarms handled manually refers to the number of code modules to be evaluated that were manually assessed. The proportion of alarms handled manually is the percentage of code modules to be evaluated that were manually assessed out of the total number of code modules to be evaluated.

[0134] Figure 4 In the bar chart, the bars represent the number of alarms that were manually handled. Figure 4 The curve in the figure represents the proportion of alarms handled manually.

[0135] Figure 4 The values ​​0.1, 0.2, 0.3, 0.4, 0.5, and 0.6 on the left side represent the coordinates for the proportion of alarms handled manually. Figure 4 The values ​​10, 20, 30, 40, 50, 60, 70, 80, and 90 on the right side represent the coordinates of the number of alarms that need to be manually processed. Figure 4 The time coordinates of the deployment effect evaluation system during operation in the lower middle part of the 13th week of 2021, the 14th week of 2021, the 15th week of 2021, ..., the 33rd week of 2021, the 34th week of 2021, and the 35th week of 2021.

[0136] Depend on Figure 4 The dynamic relationship between the number of alarms handled manually and the proportion of alarms handled manually, as shown in the figure, indicates that the proportion of alarms handled manually is relatively high in the early stage of operation, and then decreases as the deployment effect evaluation system runs for a longer period of time, and remains stable at a low value.

[0137] Based on and Figure 1 The method shown follows the same principle. Figure 5 A schematic diagram of a deployment effect evaluation device provided in an embodiment of this disclosure is shown, such as... Figure 5 As shown, the deployment effect evaluation device 50 may include:

[0138] The information acquisition module 510 is used to acquire change-related information corresponding to the current code change operation performed by the code module to be evaluated in the current deployment phase, and / or the current alarm information of the code module to be evaluated in the current deployment phase, wherein the code module to be evaluated is a code module whose current alarm information meets the preset alarm conditions.

[0139] The deployment effect evaluation module 520 is used to determine the deployment effect of the code module to be evaluated based on change-related information and / or current alarm information.

[0140] The apparatus provided in this disclosure determines the deployment effect of code modules to be evaluated by identifying code modules whose current alarm information meets preset alarm conditions, based on change-related information corresponding to the current code change operations performed by the code modules to be evaluated during the current deployment phase, and / or the current alarm information of the code modules to be evaluated during the current deployment phase. Based on this solution, the deployment effect of code modules can be automatically and effectively evaluated, helping to reduce reliance on human labor and improve processing efficiency.

[0141] Optionally, the deployment effect evaluation module is specifically used for:

[0142] Based on change-related information and / or current alarm information, the target evaluation strategy is matched from the preset evaluation strategies. The evaluation strategy is pre-configured with corresponding sub-effect evaluation indicators.

[0143] Based on the target sub-effect evaluation indicators corresponding to the target evaluation strategy, the effect evaluation indicators of the code module to be evaluated are determined. The effect evaluation indicators are used to characterize the deployment effect of the code module to be evaluated.

[0144] Optionally, the evaluation strategy includes at least two items, and the evaluation strategy corresponds to a preset matching order. When the deployment effect evaluation module matches the target evaluation strategy from the preset evaluation strategies based on change-related information and / or current alarm information, it is specifically used for:

[0145] According to the matching order, based on the change-related information and / or the current alarm information, the system sequentially matches each preset evaluation strategy until the target sub-effect evaluation index corresponding to the matched target evaluation strategy can be used to determine the effect evaluation index that meets the preset indicator conditions.

[0146] Optionally, changes to related information include:

[0147] The start and end times of the current code change operation;

[0148] The current equipment room corresponding to the current deployment phase;

[0149] The changes made in the current code modification operation;

[0150] The current device that is displaying the current alarm message.

[0151] Optionally, the evaluation strategy includes a first evaluation strategy, which is that the current alarm information or alarm information similar to the current alarm information does not appear after the current code change operation is completed. The sub-effect evaluation index corresponding to the first evaluation strategy is the first sub-effect evaluation index, which has a positive impact on the deployment effect of the code module to be evaluated.

[0152] Optionally, when the deployment effect evaluation module matches the target evaluation strategy from the preset evaluation strategies based on change-related information and / or current alarm information, it is specifically used for at least one of the following:

[0153] If, based on the start and end times of the current code change operation, the current equipment room corresponding to the current deployment stage, and the current alarm information, it is determined that the current alarm information in the current equipment room appeared before the current code change operation, and the current alarm information appeared multiple times after the current code change operation was completed, then the first evaluation strategy will be determined as the target evaluation strategy.

[0154] If, based on the start and end times of the current code change operation, the current equipment room corresponding to the current deployment stage, and the current alarm information, it is determined that a first alarm information similar to the current alarm information appeared in the current equipment room within a preset time period before the current code change operation, then the first evaluation strategy will be determined as the target evaluation strategy.

[0155] If, based on the current equipment room and current alarm information corresponding to the current deployment phase, it is determined that a second alarm information similar to the current alarm information has appeared in an equipment room other than the current equipment room, then the first evaluation strategy will be determined as the target evaluation strategy.

[0156] Optionally, the evaluation strategy includes a second evaluation strategy, which assumes that the impact of the current alarm information meets preset impact conditions. The sub-effect evaluation index corresponding to the second evaluation strategy is the second sub-effect evaluation index, which has a positive impact on the deployment effect of the code module to be evaluated.

[0157] Optionally, when the deployment effect evaluation module matches the target evaluation strategy from the preset evaluation strategies based on change-related information and / or current alarm information, it is specifically used for at least one of the following:

[0158] If, based on the current equipment room and current alarm information corresponding to the current deployment phase, it is determined that the number of devices in the current equipment room that have the current alarm information meets the first preset quantity condition, then the second evaluation strategy will be determined as the target evaluation strategy.

[0159] If, based on the current equipment room and current alarm information corresponding to the current deployment phase, it is determined that the number of current equipment rooms that have previously issued current alarm information does not meet the second preset quantity condition, or the number of devices in the current equipment room that have issued current alarm information does not meet the third preset quantity condition, then the second evaluation strategy will be determined as the target evaluation strategy.

[0160] Optionally, the evaluation strategy includes a third evaluation strategy, which is based on the similarity between the current alarm information and the alarm information in the historical evaluation record of the first target meeting a first preset similarity condition, or the similarity between the change content corresponding to the current code change operation and the change content in the historical evaluation record of the first target meeting a second preset similarity condition. The historical evaluation record of the first target is a historical evaluation record in which the deployment effect of the corresponding code module meets a preset positive effect condition. The sub-effect evaluation index corresponding to the third evaluation strategy is the third sub-effect evaluation index, which has a positive impact on the deployment effect of the code module to be evaluated.

[0161] Optionally, the evaluation strategy includes a fourth evaluation strategy, which is based on the current alarm information or the device identifier of the current device being in a preset whitelist. The sub-effect evaluation index corresponding to the fourth evaluation strategy is the fourth sub-effect evaluation index, which has a positive impact on the deployment effect of the code module to be evaluated.

[0162] Optionally, the evaluation strategy includes a fifth evaluation strategy, which is that the current alarm information contains specified alarm content, or that the code change operation of the code module to be evaluated contains specified change content. Both the specified alarm content and the specified change content are determined based on the historical evaluation records of the second target. The historical evaluation records of the second target are historical evaluation records in which the deployment effect of the corresponding code module meets the preset negative effect conditions. The sub-effect evaluation index corresponding to the fifth evaluation strategy is the fifth sub-effect evaluation index, which has a negative impact on the deployment effect of the code module to be evaluated.

[0163] It is understood that the above-mentioned modules of the deployment effect evaluation device in the embodiments of this disclosure have the ability to implement... Figure 1 The embodiments shown illustrate the functionality of corresponding steps in the deployment effect evaluation method. This functionality can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-described functions. These modules can be software and / or hardware, and each module can be implemented individually or integrated from multiple modules. For a detailed description of the functions of each module in the above deployment effect evaluation device, please refer to [link to relevant documentation]. Figure 1 The corresponding description of the deployment effect evaluation method in the illustrated embodiments will not be repeated here.

[0164] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0165] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0166] The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the deployment effect evaluation method provided in the embodiments of this disclosure.

[0167] Compared with existing technologies, this electronic device determines the deployment effectiveness of code modules to be evaluated by identifying code modules whose current alarm information meets preset alarm conditions, based on change-related information corresponding to the current code change operations performed by the code modules to be evaluated during the current deployment phase, and / or the current alarm information of the code modules to be evaluated during the current deployment phase. Based on this solution, the deployment effectiveness of code modules can be automatically and effectively evaluated, helping to reduce reliance on human labor and improve processing efficiency.

[0168] The readable storage medium is a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the deployment effect evaluation method provided in the embodiments of this disclosure.

[0169] Compared with existing technologies, this readable storage medium determines the deployment effectiveness of code modules to be evaluated by identifying code modules whose current alarm information meets preset alarm conditions, based on change-related information corresponding to the current code change operations performed by the code modules to be evaluated during the current deployment phase, and / or the current alarm information of the code modules to be evaluated during the current deployment phase. Based on this solution, the deployment effectiveness of code modules can be automatically and effectively evaluated, helping to reduce reliance on human intervention and improve processing efficiency.

[0170] The computer program product includes a computer program that, when executed by a processor, implements the deployment effect evaluation method provided in the embodiments of this disclosure.

[0171] Compared with existing technologies, this computer program product determines the deployment effectiveness of code modules to be evaluated by identifying code modules whose current alarm information meets preset alarm conditions, based on change-related information corresponding to the current code change operations performed by the code modules to be evaluated during the current deployment phase, and / or the current alarm information of the code modules to be evaluated during the current deployment phase. Based on this solution, the deployment effectiveness of code modules can be automatically and effectively evaluated, helping to reduce reliance on human labor and improve processing efficiency.

[0172] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0173] like Figure 6As shown, the electronic device 600 includes a computing unit 610, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 620 or a computer program loaded from a storage unit 680 into a random access memory (RAM) 630. The RAM 630 may also store various programs and data required for the operation of the device 600. The computing unit 610, ROM 620, and RAM 630 are interconnected via a bus 640. An input / output (I / O) interface 650 is also connected to the bus 640.

[0174] Multiple components in device 600 are connected to I / O interface 650, including: input unit 660, such as keyboard, mouse, etc.; output unit 670, such as various types of monitors, speakers, etc.; storage unit 680, such as disk, optical disk, etc.; and communication unit 690, such as network card, modem, wireless transceiver, etc. Communication unit 690 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0175] The computing unit 610 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 610 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 610 executes the deployment effect evaluation method provided in the embodiments of this disclosure. For example, in some embodiments, executing the deployment effect evaluation method provided in the embodiments of this disclosure can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 680. In some embodiments, part or all of the computer program can be loaded and / or installed on device 600 via ROM 620 and / or communication unit 690. When the computer program is loaded into RAM 630 and executed by computing unit 610, one or more steps of the deployment effect evaluation method provided in the embodiments of this disclosure can be performed. Alternatively, in other embodiments, the computing unit 610 can be configured to execute the deployment effect evaluation method provided in the embodiments of this disclosure by any other suitable means (e.g., by means of firmware).

[0176] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0177] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0178] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0179] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0180] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0181] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0182] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0183] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for evaluating deployment effect, comprising: obtaining change-related information corresponding to a current code change operation performed by a code module to be evaluated in a current deployment stage, and / or current alarm information of the code module to be evaluated in the current deployment stage, the code module to be evaluated being a code module whose current alarm information satisfies a preset alarm condition; based on the change-related information and / or the current alarm information, matching a target evaluation strategy from preset evaluation strategies, the evaluation strategies being preconfigured with corresponding sub-effect evaluation indexes; based on a target sub-effect evaluation index corresponding to the target evaluation strategy, determining an effect evaluation index of the code module to be evaluated, the effect evaluation index being used to represent a deployment effect of the code module to be evaluated.

2. The method of claim 1, wherein, The evaluation strategies include at least two, the evaluation strategies corresponding to a preset matching order, the matching of the target evaluation strategy from the preset evaluation strategies based on the change-related information and / or the current alarm information including: based on the change-related information and / or the current alarm information, sequentially matching each of the preset evaluation strategies in accordance with the matching order, until the target sub-effect evaluation index corresponding to the target evaluation strategy matched based on the matching is able to determine the effect evaluation index satisfying a preset index condition.

3. The method of claim 1, wherein, The change-related information includes: start and end times of the current code change operation; a current device room corresponding to the current deployment stage; corresponding change content of the current code change operation; a current device where the current alarm information occurs.

4. The method of claim 3, wherein, The evaluation strategies include a first evaluation strategy, the first evaluation strategy being that the current alarm information or alarm information similar to the current alarm information does not occur after completion of the current code change operation, a sub-effect evaluation index corresponding to the first evaluation strategy being a first sub-effect evaluation index, the first sub-effect evaluation index having a positive effect on the deployment effect of the code module to be evaluated.

5. The method of claim 4, wherein, The matching of the target evaluation strategy from the preset evaluation strategies based on the change-related information and / or the current alarm information includes at least one of the following: if, based on the start and end times of the current code change operation, the current device room corresponding to the current deployment stage, and the current alarm information, it is determined that the current alarm information in the current device room occurs before the current code change operation and the current alarm information occurs multiple times after completion of the current code change operation, the first evaluation strategy is determined as the target evaluation strategy; if, based on the start and end times of the current code change operation, the current device room corresponding to the current deployment stage, and the current alarm information, it is determined that first alarm information similar to the current alarm information occurs in a preset time period before the current code change operation in the current device room, the first evaluation strategy is determined as the target evaluation strategy; If it is determined that second alarm information similar to the current alarm information has occurred in a device room other than the current device room based on the current device room corresponding to the current deployment stage and the current alarm information, the first evaluation strategy is determined as the target evaluation strategy.

6. The method of claim 3, wherein, The evaluation strategy includes a second evaluation strategy, the second evaluation strategy is that the influence condition of the current alarm information satisfies a preset influence condition, and a second sub-effect evaluation index corresponding to the second evaluation strategy has a positive influence on the deployment effect of the code module to be evaluated.

7. The method of claim 6, wherein, The matching of the target evaluation strategy from the preset evaluation strategy based on the change-related information and / or the current alarm information includes at least one of the following: If it is determined that the number of devices in the current device room that have the current alarm information satisfies a first preset number condition based on the current device room corresponding to the current deployment stage and the current alarm information, the second evaluation strategy is determined as the target evaluation strategy. If it is determined that the number of current device rooms that have had the current alarm information does not satisfy a second preset number condition, or the number of devices in the current device room that have the current alarm information does not satisfy a third preset number condition based on the current device room corresponding to the current deployment stage and the current alarm information, the second evaluation strategy is determined as the target evaluation strategy.

8. The method of claim 3, wherein, The evaluation strategy includes a third evaluation strategy, the third evaluation strategy is that the similarity between the current alarm information and the alarm information in a first target historical evaluation record satisfies a first preset similarity condition, or the similarity between the change content corresponding to the current code change operation and the change content in the first target historical evaluation record satisfies a second preset similarity condition, the first target historical evaluation record is a historical evaluation record corresponding to a deployment effect of a code module that satisfies a preset positive effect condition, and a third sub-effect evaluation index corresponding to the third evaluation strategy has a positive influence on the deployment effect of the code module to be evaluated.

9. The method of claim 3, wherein, The evaluation strategy includes a fourth evaluation strategy, the fourth evaluation strategy is that the current alarm information or the device identifier of the device that has the current alarm information belongs to a preset whitelist, and a fourth sub-effect evaluation index corresponding to the fourth evaluation strategy has a positive influence on the deployment effect of the code module to be evaluated.

10. The method of claim 3, wherein, The evaluation strategy includes a fifth evaluation strategy, the fifth evaluation strategy is that the current alarm information contains specified alarm content, or the code change operation of the to-be-evaluated code module corresponds to specified change content, and the specified alarm content and the specified change content are determined based on a second target historical evaluation record, the second target historical evaluation record is a historical evaluation record corresponding to a deployment effect of a code module satisfying a preset negative effect condition, and a sub-effect evaluation index corresponding to the fifth evaluation strategy is a fifth sub-effect evaluation index, and the fifth sub-effect evaluation index has a negative influence on the deployment effect of the to-be-evaluated code module.

11. A deployment effect evaluation apparatus, comprising: An information acquisition module configured to acquire change-related information corresponding to a current code change operation performed by a to-be-evaluated code module in a current deployment stage, and / or current alarm information of the to-be-evaluated code module in the current deployment stage, the to-be-evaluated code module being a code module whose current alarm information satisfies a preset alarm condition; A deployment effect evaluation module configured to: Match a target evaluation strategy from preset evaluation strategies based on the change-related information and / or the current alarm information, the evaluation strategies being preconfigured with corresponding sub-effect evaluation indexes; Determine an effect evaluation index of the to-be-evaluated code module based on a target sub-effect evaluation index corresponding to the target evaluation strategy, the effect evaluation index being used to represent a deployment effect of the to-be-evaluated code module.

12. The apparatus of claim 11, wherein, The evaluation strategies include at least two items, the evaluation strategies corresponding to a preset matching order, and when the deployment effect evaluation module matches a target evaluation strategy from preset evaluation strategies based on the change-related information and / or the current alarm information, the deployment effect evaluation module is specifically configured to: Match the change-related information and / or the current alarm information with each of the preset evaluation strategies in turn according to the matching order, until a target sub-effect evaluation index corresponding to a target evaluation strategy obtained through matching can determine the effect evaluation index satisfying a preset index condition.

13. The apparatus of claim 11, wherein, The change-related information includes: A start and end time of the current code change operation; A current equipment room corresponding to the current deployment stage; A change content corresponding to the current code change operation; A current equipment corresponding to the current alarm information.

14. The apparatus of claim 13, wherein, The evaluation strategy includes a first evaluation strategy, the first evaluation strategy is that the current alarm information or alarm information similar to the current alarm information does not occur after the completion of the current code change operation, and a sub-effect evaluation index corresponding to the first evaluation strategy is a first sub-effect evaluation index, and the first sub-effect evaluation index has a positive influence on the deployment effect of the to-be-evaluated code module.

15. The apparatus of claim 14, wherein, When the deployment effect evaluation module matches a target evaluation strategy from preset evaluation strategies based on the change-related information and / or the current alarm information, the deployment effect evaluation module is specifically configured to at least one of: If it is determined, based on the start and end time of the current code change operation, the current device room corresponding to the current deployment stage, and the current alarm information, that the current alarm information occurred in the current device room before the current code change operation and occurred multiple times after the completion of the current code change operation, the first evaluation strategy is determined as the target evaluation strategy. If it is determined, based on the start and end time of the current code change operation, the current device room corresponding to the current deployment stage, and the current alarm information, that the first alarm information similar to the current alarm information occurred in the current device room within a preset time period before the current code change operation, the first evaluation strategy is determined as the target evaluation strategy. If it is determined, based on the current device room corresponding to the current deployment stage and the current alarm information, that the second alarm information similar to the current alarm information occurred in a device room other than the current device room, the first evaluation strategy is determined as the target evaluation strategy.

16. An electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-10.

17. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method of any one of claims 1-10.

18. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1-10.

Images