System performance index determination method and device, storage medium and electronic equipment

Abstract

The invention discloses a system performance index determination method and device, a storage medium and electronic equipment. The method comprises the following steps: acquiring log information of a system; inputting the log information into a defect perception model for analysis to obtain a word set corresponding to the log information; weight data corresponding to at least one initial sub-object in the word set is determined, and the weight data is used for representing the importance degree of the initial sub-object to the performance index of the system; based on the weight data, converting the initial sub-object to obtain a target sub-object; and determining a performance index of the system based on the target sub-object. Through the method and the device, the technical problem of low accuracy of system performance judgment in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular, to a method, device, storage medium and electronic device for determining performance indicators of a system. Background Art

[0002] At present, in the current software development and operation (DevOps) environment, with the increasing complexity of application systems and the accelerated development iteration speed, ensuring software quality and system stability faces major challenges.

[0003] In related technologies, traditional performance management tools usually rely on manual collection and analysis of performance data in various tool systems, such as build time, test coverage, deployment success rate, etc. However, the above content requires developers or operation and maintenance personnel to make judgments based on experience, and there is a technical problem of low accuracy in system performance judgment.

[0004] Currently, no effective solution has been proposed to address the problem of low accuracy in judging system performance in related technologies. Summary of the invention

[0005] The main purpose of the present application is to provide a method, device, storage medium and electronic device for determining a performance indicator of a system, so as to solve the problem of low accuracy of system performance judgment in related technologies.

[0006] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for determining the performance index of a system is provided. The method may include: obtaining the log information of the system; inputting the log information into a defect perception model for analysis to obtain a word set corresponding to the log information, wherein the defect perception model is trained based on the log information sample of the system and the word set sample corresponding to the log information sample, the word set is used to characterize the abnormal data in the log information, and the word set sample is used to characterize the abnormal data sample in the log information sample; determining the weight data corresponding to at least one initial sub-object in the word set, wherein the weight data is used to characterize the importance of the initial sub-object to the performance index of the system; based on the weight data, converting the initial sub-object to obtain the target sub-object; and determining the performance index of the system based on the target sub-object.

[0007] Optionally, determining weight data corresponding to at least one initial sub-object in the word set includes: determining a word vector corresponding to the initial sub-object; and determining the weight data based on the word vector and a performance indicator parameter of the system.

[0008] Optionally, weight data is determined based on the word vector and the performance indicator parameters of the system, including: determining the target type of the word vector based on the performance indicator parameters; and determining the target weight data of the target type as the weight data corresponding to the initial sub-object.

[0009] Optionally, determining a target type of a word vector based on a performance indicator parameter includes: in response to a similarity between a vector corresponding to the performance indicator parameter and the word vector being greater than a similarity threshold, determining the type corresponding to the performance indicator parameter as the target type.

[0010] Optionally, based on the target sub-object, determining the performance indicator of the system includes: based on the type of word vector, dividing at least one initial sub-object to obtain at least one subset, wherein the subset includes multiple sub-objects, and the multiple sub-objects are used to form the initial sub-object; based on at least one target sub-object in the subset, obtaining an average value corresponding to the subset, wherein the average value is used to characterize the sub-performance indicator corresponding to the type; based on at least one average value corresponding to at least one subset, determining the performance indicator.

[0011] Optionally, determining a performance indicator based on at least one average value corresponding to at least one subset includes: in response to the average value satisfying a target value, determining a performance indicator that satisfies the performance indicator requirement; in response to the average value not satisfying the target value, determining a performance indicator that does not satisfy the performance indicator requirement.

[0012] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a device for determining the performance index of a system is provided. The device may include: an acquisition unit for acquiring the log information of the system; an analysis unit for inputting the log information into a defect perception model for analysis to obtain a word set corresponding to the log information, wherein the defect perception model is trained based on the log information sample of the system and the word set sample corresponding to the log information sample, the word set is used to characterize the abnormal data in the log information, and the word set sample is used to characterize the abnormal data sample in the log information sample; a first determination unit is used to determine the weight data corresponding to at least one initial sub-object in the word set, wherein the weight data is used to characterize the importance of the initial sub-object to the performance index of the system; a conversion unit is used to convert the initial sub-object based on the weight data to obtain a target sub-object; a second determination unit is used to determine the performance index of the system based on the target sub-object.

[0013] Optionally, the first determination unit may further include: a conversion module, used to determine the word vector corresponding to the initial sub-object; and a first determination module, used to determine weight data based on the word vector and a performance indicator parameter of the system.

[0014] Optionally, the first determination module may also include: a first determination submodule, used to determine the target type of the word vector based on the performance indicator parameter; and a second determination submodule, used to determine the target weight data of the target type as the weight data corresponding to the initial sub-object.

[0015] Optionally, the first determination submodule may also be configured to determine the type corresponding to the performance indicator parameter as the target type in response to the similarity between the vector corresponding to the performance indicator parameter and the word vector being greater than a similarity threshold.

[0016] Optionally, the second determination unit may also include: a division module, used to divide at least one initial sub-object based on the type of word vector to obtain at least one subset, wherein the subset includes multiple sub-objects, and the multiple sub-objects are used to form the initial sub-object; a processing module, used to obtain an average value corresponding to the subset based on at least one target sub-object in the subset, wherein the average value is used to characterize the sub-performance indicator corresponding to the type; a second determination module, used to determine the performance indicator based on at least one average value corresponding to at least one subset.

[0017] Optionally, the second determination module may further include: a third determination submodule, used to determine the performance indicator that meets the performance indicator requirements in response to the average value meeting the target value; and a fourth determination submodule, used to determine the performance indicator that does not meet the performance indicator requirements in response to the average value not meeting the target value.

[0018] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a computer-readable storage medium is further provided, which may include a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located can be controlled to execute the above-mentioned method.

[0019] In order to achieve the above-mentioned purpose, according to another aspect of the present application, an electronic device is further provided, which may include: a memory storing an executable program; and a processor for running the program, wherein the above-mentioned method is executed when the program is running.

[0020] In order to achieve the above objective, according to another aspect of the present application, a computer program product is provided. The computer program product may include computer instructions. When the computer instructions are executed by a processor, the steps of the above method are implemented.

[0021] In an embodiment of the present application, the log information of the system is obtained; the log information is input into a defect perception model for analysis to obtain a word set corresponding to the log information, wherein the defect perception model is trained based on the log information sample of the system and the word set sample corresponding to the log information sample, the word set is used to characterize the abnormal data in the log information, and the word set sample is used to characterize the abnormal data sample in the log information sample; the weight data corresponding to at least one initial sub-object in the word set is determined, wherein the weight data is used to characterize the importance of the initial sub-object to the performance index of the system; based on the weight data, the initial sub-object is converted to obtain the target sub-object; based on the target sub-object, the performance index of the system is determined. That is, in this embodiment, the log information of the system is automatically pulled, the log information is identified, the word set in the log information is obtained, the weight data corresponding to the word set is determined, based on the weight data, the initial sub-object in the word set is converted to obtain the target sub-object, based on the target sub-object, the performance index of the system is determined, and the performance index can be used to determine the performance of the system, thereby achieving the technical effect of improving the accuracy of the system performance judgment and solving the technical problem of low accuracy of the system performance judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0023] Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for determining a performance indicator of a system is shown;

[0024] Figure 2 is a flow chart of a method for determining a performance indicator of a system according to an embodiment of the present application;

[0025] Figure 3 is a flow chart of a DevOps performance management method based on defect perception according to an embodiment of the present application;

[0026] Figure 4 is a schematic diagram of a device for determining a performance indicator of a system according to an embodiment of the present application;

[0027] Figure 5 It is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0031] Defect perception refers to the ability of team members to perceive and identify defects or problems in the system or application during software development or operation and maintenance, which can include the perception and identification of problems in system performance, stability, security, etc.

[0032] The software development and operation methodology (DevOps) can be used to achieve the goal of fast, efficient and continuous delivery and deployment of software by integrating the workflows and tools of the development (Dev) and operation (Ops) teams. In DevOps, defect awareness is an important concept. Team members need to be able to discover and solve problems in the system in a timely manner to ensure the quality and stability of software delivery and deployment. At the same time, DevOps also emphasizes continuous monitoring and feedback in order to perceive and respond to defects and problems in the system in a timely manner.

[0033] It should be noted that the collected information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with relevant laws, regulations and standards, necessary confidentiality measures are taken, and public order and good customs are not violated, and corresponding operation entrances are provided for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or institutions to provide users with corresponding operation entrances for users to choose to agree or refuse the results of automated decision-making; if the user chooses to refuse, the expert decision-making process will be entered.

[0034] Example 1

[0035] According to an embodiment of the present application, an embodiment of a method for determining performance indicators of a system is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0036] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 The hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for determining a performance indicator of a system is shown. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0037] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As involved in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0038] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the performance indicator determination method of the system in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the performance indicator determination method of the system described above is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0039] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0040] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0041] Under the above operating environment, this application provides Figure 2 A method for determining performance indicators of the system shown. Figure 2 is a flow chart of a method for determining a performance indicator of a system according to an embodiment of the present application. Figure 2 As shown, the method may include the following steps:

[0042] Step S201 obtains the system log information.

[0043] In this embodiment, the above log information may be a record generated during the operation of the system, and may include information such as system status, operation results, errors or warnings. In a DevOps environment, the log information may cover the entire life cycle from code submission, construction, testing, deployment to operation, such as logs built by an open source continuous integration tool (Jenkins), code review logs, test result logs, etc. The above log information may be used as basic data for defect perception model analysis and may be used to monitor and evaluate system performance. The above system may be an operating system, a tool system, etc. It should be noted that this is only an example, and there is no specific restriction on the content of the log information or the type of system.

[0044] Optionally, log information can come from various tools in the DevOps environment, such as build information tools, code quality information tools, monitoring indicators, etc., which can be used to characterize each link from code submission, construction, testing to deployment.

[0045] Step S202: input the log information into the defect perception model for analysis to obtain a word set corresponding to the log information, wherein the defect perception model is trained based on the system's log information samples and word set samples corresponding to the log information samples, the word set is used to characterize abnormal data in the log information, and the word set samples are used to characterize abnormal data samples in the log information samples.

[0046] In this embodiment, the above-mentioned defect perception model can be obtained by training based on the log information samples of the system and the word set samples corresponding to the log information samples, and can be a neural network model, which can be used to generate the word set corresponding to the log information. The above-mentioned word set can include synonyms and antonyms, and can be a group of words extracted from the log information, which can be associated with a specific system state or event. In DevOps performance management, the word set can include words that describe normal and abnormal events in the DevOps process, such as "build completed" and "test passed" as normal words (that is, just words), "build failed" and "test timeout" as abnormal words (that is, antonyms). The word set can be an intermediate step for model analysis and indicator evaluation, and can be used to characterize the content of log information. The above-mentioned abnormal data can refer to the unexpected state or event that occurs during the operation of the system, which can be used to characterize the defects or performance problems of the system. In the DevOps scenario, the abnormal data can include the number of build failures, the cases where the test failed, the error information in the deployment process, etc., which can be automatically identified and classified from the log information by the defect perception model, and can be used to evaluate and improve system performance.

[0047] Optionally, the above-mentioned defect perception model can be a model obtained through machine learning training, which can be trained based on a large number of log information samples and corresponding abnormal data word set samples, and can be used to identify and classify abnormal data in log information. The defect perception model can be trained based on a large number of log information samples and word set samples to learn to identify the characteristics of abnormal events. For example, the model can be trained to identify the occurrence of specific patterns such as "build failure" and "deployment exception", which indicate potential problem points in the DevOps process.

[0048] For example, words such as "build timeout", "test failure", and "deployment exception" can represent abnormal data and constitute word set samples. By analyzing these samples, the model learns to identify and classify abnormal data in logs.

[0049] In this embodiment, the log information of each system can be automatically extracted, and the log information can be analyzed by calling the defect perception model to obtain a word set corresponding to the log information, which can include righteous words and antonyms, and can be a set of righteous words and antonyms, which can be used to determine abnormal conditions in the system. Among them, the above-mentioned righteous words can be data used to determine the correct operation of the system, and can include words such as "test success" and "no error", and antonyms can be used to determine that the abnormal data in the log information is "test failure" or "error occurrence".

[0050] Optionally, the defect perception model may pre-store standard log information under normal circumstances or when performance indicators meet performance requirements. After obtaining the log information, the standard log information and the log information may be compared to obtain a word set corresponding to the log information. Alternatively, the defect perception model may identify the semantic information and contextual information of the log information, and based on the semantic information and contextual information, a word set corresponding to the log information may be identified.

[0051] Optionally, during the model training phase, log information samples related to the application of each tool system can be obtained, and the relevant information in the log information samples can be classified. Based on the principle of word embedding, a template representation method can be designed to accurately extract semantic information from the log template. This method not only captures the contextual information in the template, but also captures the semantic information, which is used to deal with scenarios where different system log information samples return different results. Among them, log information with synonyms in the log information sample can be regarded as similar events, and log information with antonyms can be regarded as different events. Further, the model can be trained by machine learning methods, and a set of synonyms and antonyms in the log information sample (that is, a word set sample) can be constructed, the word vector corresponding to the log information sample can be generated, and the template vector can be calculated. The error code in the log information sample is included in the defect perception model using the template. When the relevant log information is automatically pulled later, the model can obtain information such as which data has changed, how large the impact of the change is, and the reason for the change based on the defect perception model.

[0052] For example, before training, the log information samples can be sorted based on preset information to obtain the content of the rule mapping table. Based on the rule mapping table, a synonym and antonym set (that is, a word set sample) can be obtained. Further, the log information samples can be processed by the defect perception model to obtain an initial word set sample. The initial word set sample and the word set sample can be checked for consistency, and the tool system can be manually logged in to perform data matching verification. If the initial word set sample and the word set sample are inconsistent, the cause of the inconsistency between the initial word set sample and the word set sample can be determined, and the defect perception model can be continuously optimized to finally train the defect perception model.

[0053] Optionally, Table 1 is a rule mapping table. As shown in Table 1, the rule mapping table may include a rule number. By arranging the log information according to the rule mapping table, a table field mapping rule may be obtained. For example, in an external data source A, field A table. A field, and internal field B table. B field, the corresponding matching rule is rule 1. It should be noted that this is only an example, and there is no specific limitation on the content contained in the rule mapping table.

[0054] Table 1 Rule mapping table

[0055] Table field mapping rules: External Data Sources Fields Internal fields Matching rules External data source A Table A. Field A B table. B field Rule 1 Rule Mapping Table Rule Number Department Field Internal fields Rule 1 A a Rule 1 B b

[0056] Step S203: determining weight data corresponding to at least one initial sub-object in the word set, wherein the weight data is used to characterize the importance of the initial sub-object to the performance index of the system.

[0057] In this embodiment, the initial sub-object may refer to a specific word or event related to system performance extracted during the model analysis process. In DevOps performance management, the initial sub-object may be text such as "build time", "test pass rate", "test success", "calculation failure", etc. The initial sub-object may be a word in a word set. After model analysis, the occurrence and frequency of the initial sub-object may be used to evaluate the status of the system performance indicators. The weight data may be used to characterize the importance of the initial sub-object to the system's performance indicators, and may be used to weight when calculating performance indicators. For example, if "build failure" has a significant impact on the determination of system performance, a higher weight data may be assigned to the initial sub-object. The allocation of the weight data may be based on historical data analysis and the experience and knowledge of domain experts, and may be used to ensure that abnormal situations of key performance indicators can be identified and processed with priority. It should be noted that this is only an example, and there is no specific restriction on the type of the initial sub-object and the method for determining the weight data.

[0058] Optionally, after obtaining a word set including at least one initial sub-object, weight data corresponding to at least one initial sub-object in the word set may be determined, so that the initial sub-object may be adjusted based on the weight data.

[0059] For example, in the word set, "build timeout" may have a greater impact on system performance, so this initial sub-object can be assigned a higher weight data to indicate its importance to the system performance indicators. If the abnormal data "build failure" frequently appears in the system's log information, through the defect perception model analysis, this word may be identified as the initial sub-object of the antonym and assigned high weight data. When calculating the performance indicator "build success rate", due to the high weight of "build failure", the model will focus on its impact on performance indicators, and may eventually determine that there are serious problems with the "build success rate" indicator, which requires the team's immediate attention and resolution. This process automatically and objectively evaluates system performance, helps to quickly locate and fix problems, and improves DevOps efficiency.

[0060] Step S204: transform the initial sub-object based on the weight data to obtain a target sub-object.

[0061] In this embodiment, the initial sub-object may be transformed based on the weight data to obtain a target sub-object, which may better characterize the performance of the system.

[0062] Step S205: determining the performance index of the system based on the target sub-object.

[0063] In this embodiment, the above performance indicators may be quantitative standards for measuring the operating status and efficiency of the system or service. For DevOps, the performance indicators may include "build time", "test pass rate", "deployment success rate", "code submission frequency", etc. It should be noted that this is only an example, and there is no specific limitation on the type of performance indicators.

[0064] Optionally, by analyzing log information and word sets, the defect awareness model can help determine the current status of these performance indicators. For example, it can be used to determine whether the performance indicators are within the normal range or whether the performance indicators have experienced abnormal fluctuations, thereby guiding the DevOps team to perform corresponding optimization and problem solving.

[0065] For example, by analyzing the frequency and severity of the target sub-object, the abnormal situation of the system performance indicators can be quantified. For example, the "build time" indicator is displayed as a defective state due to the frequent occurrence of "build timeout". For example, assuming that the records of "build timeout" and "test failure" frequently appear in the system log, through the defect perception model analysis, we get a word set containing these two abnormal words. Further analysis shows that "build timeout" has a significant impact on R&D efficiency, so it is given high weight data. The weight of "test failure" is slightly lower, but it still needs attention. Next, after converting the initial sub-object according to the weight data, the target sub-object can be obtained, which can reveal the defects of the two performance indicators "build time" and "test pass rate". Finally, the system can display the specific problems of the "build time" and "test pass rate" indicators based on the above target sub-objects, including problem frequency, impact range, etc., so as to help the team optimize the DevOps process and improve system performance. This process makes full use of the semantic information of the data, automatically identifies and prioritizes key performance issues, and significantly improves the efficiency and accuracy of DevOps performance management.

[0066] In the above steps S201 to S204, the log information of the system is obtained; the log information is input into the defect perception model for analysis to obtain a word set corresponding to the log information, wherein the defect perception model is trained based on the log information sample of the system and the word set sample corresponding to the log information sample, the word set is used to characterize the abnormal data in the log information, and the word set sample is used to characterize the abnormal data sample in the log information sample; the weight data corresponding to at least one initial sub-object in the word set is determined, wherein the weight data is used to characterize the importance of the initial sub-object to the performance index of the system; based on the weight data, the initial sub-object is converted to obtain the target sub-object; based on the target sub-object, the performance index of the system is determined. That is, in this embodiment, the log information of the system is automatically pulled, the log information is identified, the word set in the log information is obtained, the weight data corresponding to the word set is determined, based on the weight data, the initial sub-object in the word set is converted to obtain the target sub-object, based on the target sub-object, the performance index of the system is determined, and the performance index can be used to determine the performance of the system, thereby achieving the technical effect of improving the accuracy of the system performance judgment and solving the technical problem of low accuracy of the system performance judgment.

[0067] The embodiments of the present application are described in detail below in combination with the above steps.

[0068] As an optional implementation, step S203, determining weight data corresponding to at least one initial sub-object in the word set, includes: determining a word vector corresponding to the initial sub-object; and determining weight data based on the word vector and a performance indicator parameter of the system.

[0069] In this embodiment, after obtaining the word set, the initial sub-objects in the word set can be converted respectively, for example, through natural language processing techniques such as word embedding, to determine the word vector corresponding to the initial sub-object, the word vector can be displayed in a mathematical space, can be a binary code, it should be noted that this is only an example, and there is no specific restriction on the form of expression of the word vector. The above performance indicator parameter can be a Devops indicator, which can include information such as software deployment time, build time, deployment success rate, etc., and the performance indicator parameter can be a vector expression.

[0070] Optionally, based on a set of synonyms and antonyms, multiple corresponding word vectors can be generated. Furthermore, based on these word vectors, by analyzing the relationship between the word vectors and performance indicator parameters (such as test pass rate), the weight data corresponding to each initial sub-object can be determined. For example, if the correlation between the word vector and the performance indicator parameter is large, a higher weight data can be assigned to the initial sub-object.

[0071] Optionally, the initial sub-object can be an abnormal event or vocabulary identified by the defect perception model from the log information. For example, the above initial sub-object can include information such as "deployment failure" and "build timeout". Through word embedding technology, these initial sub-objects in text form can be converted into word vectors in numerical form. The word vector can capture the position of the vocabulary in the semantic space, so that the model can understand the relationship between the sub-object and the performance indicator. After calculating the word vector, the impact of this initial sub-object on the system performance indicator can be evaluated. For example, as an initial sub-object, the word vector of "deployment failure" is compared with the vector of the performance indicator "deployment success rate". By calculating the distance or correlation between them, the negative impact of "deployment failure" on "deployment success rate" can be determined, and then the weight data corresponding to the initial sub-object can be calculated. The size of the weight data can directly reflect the importance and impact of the initial sub-object on the performance indicator.

[0072] For example, assuming that the word set contains the initial sub-object "deployment failure", the machine learning model can convert "deployment failure" into a word vector. Then, by calculating the distance between the "deployment failure" word vector and the "deployment success rate" performance indicator vector, the model finds that when "deployment failure" occurs frequently, the "deployment success rate" drops significantly. Therefore, the model will assign a higher weight data to "deployment failure", indicating that this event has a negative and important impact on the "deployment success rate" indicator.

[0073] In this step, by calculating the word vectors and their weight data, the impact of abnormal events on system performance indicators is automatically quantified, avoiding the subjectivity and uncertainty of manual evaluation. The weight data provides a basis for prioritizing abnormal events, allowing the DevOps team to first solve the problems that have the greatest impact on performance indicators and improve the efficiency of problem solving. With the continuous training and optimization of the model, the weight data will more accurately reflect the true state of the system, thereby supporting more accurate performance evaluation and problem location. At the same time, based on the analysis results of the weight data, the DevOps team can quickly identify key performance issues in the system and take targeted measures to improve the overall system performance and stability.

[0074] In summary, the step of determining the weight data corresponding to at least one initial sub-object in the word set can quantify the specific impact of abnormal logs on system performance indicators. By combining text analysis with performance indicator evaluation, a more scientific and accurate DevOps performance management method is provided.

[0075] As an optional implementation, weight data is determined based on word vectors and system performance indicator parameters, including: determining the target type of the word vector based on the performance indicator parameters; and determining the target weight data of the target type as the weight data corresponding to the initial sub-object.

[0076] In this embodiment, based on the performance indicator parameters, the target type of the word vector can be determined, and based on the target weight data of the target type, the weight data corresponding to the initial sub-object can be determined. That is, the weight data corresponding to different performance indicator parameters can be set in advance. After the word vector is obtained, the performance indicator parameters corresponding to the word vector can be determined, and the target weight data corresponding to the performance indicator parameters can be determined as the weight data corresponding to the initial sub-object.

[0077] Optionally, the word vector is compared with the DevOps indicator, and the weight data corresponding to the word vector can be determined according to the pre-assigned target weight data corresponding to each indicator data, thereby determining the weight data corresponding to the initial sub-object.

[0078] Optionally, before analyzing the word vector, you can first clarify which performance indicator parameters are the focus of current attention. For example, in a DevOps environment, you may focus on indicators such as "build time", "deployment success rate", and "test pass rate". The choice of target type will directly affect the allocation of weight data, ensuring that the model analysis focuses on the abnormal events that have the greatest impact on system performance. Each performance indicator parameter has its inherent importance in the system, which can be reflected by the target weight data. For example, "build time" may have a greater impact on R&D efficiency, so its weight data is preset to be high. When the model analyzes abnormal events related to "build time", such as "build timeout", the preset high weight data will be directly assigned to "build timeout", which means that the impact of "build timeout" on the performance indicator of "build time" is significant. Therefore, after determining the target type of the word vector, the target weight data of the target type can be determined as the weight data corresponding to the initial sub-object.

[0079] For example, suppose the performance indicator we are concerned about is "build time". According to historical data analysis, the center found that "build time" has a very large direct impact on project progress, so its weight data is preset to 0.8. The abnormal event "build timeout" can be analyzed from the log information. Since "build timeout" is directly related to "build time", according to the above rules, the preset "build time" weight data 0.8 is directly determined as the weight data of "build timeout". Similarly, if another performance indicator "code review time" has a preset weight of 0.6, then abnormal events related to "code review time" (such as "code review delay") will be assigned a weight data of 0.6.

[0080] In this step, the target weight data can be used to focus on abnormal events that have a significant impact on system performance, avoiding excessive attention to minor issues. After determining the weight data of the initial sub-objects, it is possible to quickly identify which abnormal events need to be handled first, speeding up the decision-making process for problem solving and improving the efficiency of the DevOps process. The weight data provides a basis for resource allocation, allowing more energy and resources to be invested in the issues that have the greatest impact on system performance, thereby achieving optimal use of resources.

[0081] Optionally, as the DevOps process is optimized and system performance is improved, the target weight data can be adjusted accordingly to ensure the accuracy and effectiveness of the model analysis and support continuous performance improvement.

[0082] As an optional implementation, determining the target type of the word vector based on the performance indicator parameter includes: in response to the similarity between the vector corresponding to the performance indicator parameter and the word vector being greater than a similarity threshold, determining the type corresponding to the performance indicator parameter as the target type.

[0083] In this embodiment, the performance indicator parameter can be a vector, so the similarity between the performance indicator parameter and the word vector can be determined by calculating the distance between the two vectors, such as the Euclidean distance. If the similarity between the performance indicator and the word vector is greater than a similarity threshold, the type corresponding to the performance indicator parameter can be determined to be the target type. It should be noted that the above method for calculating similarity is only for illustration and is not specifically limited here.

[0084] As an optional implementation, based on the target sub-object, determining the performance indicators of the system includes: based on the type of word vector, dividing at least one initial sub-object to obtain at least one subset, wherein the subset includes multiple sub-objects, and the multiple sub-objects are used to form the initial sub-object; based on at least one target sub-object in the subset, obtaining an average value corresponding to the subset, wherein the average value is used to characterize the sub-performance indicator corresponding to the type; based on at least one average value corresponding to at least one subset, determining the performance indicator.

[0085] In this embodiment, at least one initial sub-object can be divided based on the type of word vector to obtain at least one subset containing word vectors of the same type. Furthermore, based on at least one target sub-object in the subset, an average value corresponding to the subset can be obtained, wherein the average value is used to characterize the sub-performance indicator corresponding to the type, and the performance indicator can be determined based on at least one average value corresponding to at least one subset.

[0086] Optionally, this embodiment may perform weighted averaging on sub-objects of the same type to obtain a score value corresponding to the performance indicator parameter. In this way, score values ​​corresponding to multiple indicator parameters may be obtained.

[0087] Optionally, this embodiment can classify abnormal events or keywords related to different performance indicators identified from the log information according to the performance indicator types behind them. For example, words such as "build timeout" and "build failure" can be divided into subsets related to "build time", while "test failure" and "test timeout" are classified as subsets related to "test pass rate". Through this classification, word vectors can be associated with specific performance indicators, which is convenient for subsequent analysis. For each subset, the average value of all target sub-object word vectors can be calculated. The target sub-object refers to those abnormal event words that are directly related to the performance indicator. The average value reflects the general state of abnormal events in the entire subset, that is, the overall health of a certain performance indicator. For example, if the average value of the word vector of "build timeout" is high, it means that the overall build time is long and the build process may need to be optimized. By analyzing the average values ​​of different subsets, the quantitative evaluation of system performance indicators can be directly determined or affected. For example, if the average value of the subset related to "build time" (including "build timeout", "build failure", etc.) is high, then the score of the performance indicator "build time" may be low, otherwise it is high. This process directly links the specific conditions of abnormal events with the quantitative assessment of performance indicators, making the assessment more objective and data-driven.

[0088] For example, suppose we are interested in the "build time" performance indicator. The model identifies two initial sub-objects, "build timeout" and "build failure", from the log information, which represent the excessive time and errors in the build process, respectively. Based on the word vectors of these words, they can be classified into subsets related to "build time" and the average word vectors of "build timeout" and "build failure" are calculated. If the average value shows that abnormal events in the build process occur frequently, the model will reflect this result in the evaluation of the "build time" performance indicator, and may give a lower performance score, prompting the team to prioritize solving problems in the build process to optimize the "build time" indicator.

[0089] Optionally, the average value may be used to characterize the sub-performance indicator corresponding to the type, and thus, at least one average value corresponding to at least one subset may be used to determine the final performance indicator.

[0090] In this step, by calculating the average value of word vectors, we can directly quantify the impact of abnormal events on performance indicators, making the evaluation results more objective and accurate. By classifying abnormal events and calculating the average value of their word vectors, we can promptly discover abnormalities that have a significant impact on performance indicators.

[0091] As an optional implementation, a performance indicator is determined based on at least one average value corresponding to at least one subset, including: in response to the average value satisfying the target value, determining the performance indicator that satisfies the performance indicator requirements; in response to the average value not satisfying the target value, determining the performance indicator that does not satisfy the performance indicator requirements.

[0092] In this embodiment, if the average value meets the target value, it can be determined that the performance indicator meets the performance requirement; if the average value does not meet the target value, it can be determined that the performance indicator does not meet the performance requirement. The target value can be a preset value.

[0093] Optionally, the weight corresponding to each indicator is determined to obtain a weight rule. According to the weight rule, the relevant data can be displayed in the foreground.

[0094] For example, if the average value of the target sub-object word vector in a subset is lower than a preset threshold, the performance indicator corresponding to the subset is considered to meet the performance requirements. The target value can be set based on historical data, industry standards, or corporate goals, and is used to determine whether the performance indicator is within an acceptable range. For example, if the average value of the subset related to "build time" is lower than the preset threshold, it may mean that there are relatively few abnormal situations in the build process (such as build timeouts, build failures), and the build process is relatively stable and efficient. On the contrary, if the average value of the target sub-object word vector in a subset is higher than the preset threshold, it is determined that the performance indicator does not meet the performance requirements. This usually indicates that specific abnormal events or defects occur frequently and have a negative impact on system performance. For example, if the average value of the subset related to "code review time" is higher than the preset threshold, it may mean that there are a lot of delays in code review and the review process needs to be optimized.

[0095] For another example: suppose we are concerned about the performance indicator "deployment success rate". In the defect perception model, the subset related to "deployment success rate" may include descriptions such as "deployment failure" and "abnormal environment configuration". The average value of these target sub-object word vectors can be calculated. If the average value is lower than the preset threshold (for example, lower than 0.3), then the "deployment success rate" is considered to be in good condition and meets the performance requirements. On the contrary, if the average value is higher than the preset threshold, it is considered that there is a problem with the "deployment success rate" and the performance requirements are not met, and further analysis and optimization of the deployment process are required.

[0096] In this step, by setting target values ​​and performing dynamic evaluations, fluctuations in system performance can be discovered in a timely manner, and measures can be taken quickly when performance indicators deviate from the normal range. The comparison between the average value and the target value provides a quantitative basis for decision-making, helping the team make performance optimization decisions based on data rather than relying on subjective judgment. When the average value does not reach the target value, it is possible to directly locate which abnormal events or defects have a significant impact on the performance indicators, accelerating the resolution of the problem. Through the above method, real-time and accurate evaluation of the performance of each key link in the DevOps process can be achieved, providing the DevOps team with an effective tool to continuously optimize the R&D and operation and maintenance processes.

[0097] In an embodiment of the present application, the log information of the system is automatically pulled, the log information is identified, a word set in the log information is obtained, weight data corresponding to the word set is determined, and based on the weight data, the initial sub-objects in the word set are converted to obtain the target sub-objects. Based on the target sub-objects, the performance indicators of the system are determined. The performance indicators can be used to determine the performance of the system, thereby achieving the technical effect of improving the accuracy of system performance judgment and solving the technical problem of low accuracy of system performance judgment.

[0098] Another optional specific implementation is described in detail below.

[0099] At present, the maturity of Devops applications is usually evaluated through seven major areas, including online collaboration processes, technology and business value collaboration, highly automated assembly lines, and comprehensive and efficient testing systems. Devops uses four key indicators as the overall driving goal, the improvement of four core capabilities (efficient collaboration, continuous delivery, quality assurance, and environmental support) as the main line, and one organizational-level promotion as the starting point to conduct systematic design and centralized breakthroughs for key issues in various fields.

[0100] In related technologies, DevOps improves the overall DevOps maturity of applications by benchmarking indicators in seven major areas. However, the indicator data in each area are scattered in different application systems in the center. Applications need to switch to different systems to obtain corresponding data, and they need to obtain it manually. They cannot automatically pull relevant information, which is very cumbersome. In addition, applications need to actively obtain data, and cannot push relevant quantitative data and DevOps defect indicators. As a result, a lot of time and cost is spent on obtaining relevant data, and there is a technical problem of low accuracy in judging system performance.

[0101] In view of the above problems, this embodiment provides a DevOps performance management tool based on defect perception, which provides a visual and automated management tool for DevOps benchmarking work.

[0102] Optionally, the above tool may include two parts. One part is based on the defect perception method to automatically pull the indicator-related data of the application in each tool system. The second part is based on the weight of each indicator to display the indicator-related data in the foreground according to the weight ranking.

[0103] Next, the defect-aware DevOps performance management method in this application is further introduced.

[0104] Figure 3 is a flow chart of a DevOps performance management method based on defect perception according to an embodiment of the present application, such as Figure 3 As shown, the method may include the following steps:

[0105] Step S301, obtaining a log information sample.

[0106] In this embodiment, log information samples related to the application of each tool system can be obtained.

[0107] Step S302: construct a set of synonyms and antonyms.

[0108] In this embodiment, relevant information in the log information sample can be classified, and a simple and effective template representation method is designed based on the principle of word embedding. This method can accurately extract semantic information from the log template. This method not only captures the contextual information in the template, but also captures the semantic information, which is used to deal with scenarios where different system logs return different results. Generally, logs with synonyms are regarded as similar events, and logs with antonyms usually indicate different events.

[0109] Step S303, generating word vectors.

[0110] In this embodiment, the model is trained by a machine learning method, and a set of synonyms and antonyms in the log information sample is constructed, a word vector corresponding to the log information sample is generated, and a template vector is calculated.

[0111] Step S304, training the model and outputting the results.

[0112] In this case, use the template to incorporate the error codes in the log information sample into the defect perception model. When the relevant log information is automatically pulled later, the model can obtain information such as which data has changed, the scope of the change, and the reason for the change based on the defect perception model.

[0113] Step S305, checking whether the perception result is correct.

[0114] In this embodiment, the correctness of the perception result can be checked, and data matching verification can be performed by manually logging into the tool system.

[0115] Optionally, if there is inconsistency, it is necessary to determine how the inconsistency is caused and execute step S304 to continue optimizing the model to form final result data. If there is consistency, step S306 can be executed.

[0116] Step S306, assign weights and display them on the front desk.

[0117] In this embodiment, based on the synonym and antonym set samples formed in the first part, multiple corresponding word vector samples are generated on this basis, compared with the Devops index, and weights of each indicator data are assigned. Finally, the weighted average of the word vectors in the template is calculated to generate a template vector (weighted average) to form relevant weight rules.

[0118] Optionally, according to the weight rules, the relevant data can be displayed in the foreground.

[0119] Step S307, checking whether the result is accurate.

[0120] In this embodiment, it is possible to check whether the weight rule is accurate. If it is consistent, step S308 is executed. If it is inconsistent, step S306 is executed again.

[0121] Optionally, if there is an inconsistency, it can also be warned that the weighting rule may be inaccurate.

[0122] Step S308, the defect perception process ends.

[0123] Optionally, constructing a synonym and antonym set may refer to searching for synonyms and antonyms of natural language words in the template content in a database, and some synonyms and antonyms of domain-specific business knowledge need to be identified by operation and maintenance personnel and converted into normal natural language vocabulary.

[0124] Optionally, word embeddings can be generated by applying a distributed vocabulary contrastive embedding model to generate word embeddings for the words in the template.

[0125] Optionally, the template vector may be a weighted average of word vectors of words in the template.

[0126] In this embodiment, a Devops performance management tool based on defect perception is provided, which can automatically perceive defective Devops-related indicators by obtaining system log information of different tools, thereby reducing the difficulty of data collection for managers due to data being scattered in different systems. It can also reduce the work of managers in analyzing indicator defects, save labor costs, reduce errors caused by manual statistics, and improve the processing efficiency and accuracy of managers.

[0127] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0128] Example 2

[0129] The embodiment of the present application also provides a system performance indicator determination device. It should be noted that the system performance indicator determination device of the embodiment of the present application can be used to execute the system performance indicator determination method provided by the embodiment of the present application. The following introduces the system performance indicator determination device provided by the embodiment of the present application.

[0130] According to an embodiment of the present application, a device for implementing the performance indicator determination method of the above system is also provided. Figure 4 is a schematic diagram of a device for determining a performance indicator of a system according to an embodiment of the present application, such as Figure 4 As shown, the performance indicator determination device 400 of the system includes: an acquisition unit 401 , an analysis unit 402 , a first determination unit 403 , a conversion unit 404 and a second determination unit 405 .

[0131] The acquisition unit 401 is used to acquire the log information of the system.

[0132] The analysis unit 402 is used to input the log information into the defect perception model for analysis to obtain a word set corresponding to the log information, wherein the defect perception model is trained based on the system's log information samples and the word set samples corresponding to the log information samples, the word set is used to characterize the abnormal data in the log information, and the word set samples are used to characterize the abnormal data samples in the log information samples.

[0133] The first determining unit 403 is used to determine weight data corresponding to at least one initial sub-object in the word set, wherein the weight data is used to represent the importance of the initial sub-object to the performance index of the system.

[0134] The conversion unit 404 is used to convert the initial sub-object based on the weight data to obtain a target sub-object.

[0135] The second determining unit 405 is configured to determine a performance indicator of the system based on the target sub-object.

[0136] The performance indicator determination device of the system provided in the embodiment of the present application obtains the log information of the system through an acquisition unit; inputs the log information into a defect perception model for analysis through an analysis unit to obtain a word set corresponding to the log information, wherein the defect perception model is trained based on the log information samples of the system and the word set samples corresponding to the log information samples, the word set is used to characterize the abnormal data in the log information, and the word set samples are used to characterize the abnormal data samples in the log information samples; determines the weight data corresponding to at least one initial sub-object in the word set through a first determination unit, wherein the weight data is used to characterize the importance of the initial sub-object to the performance indicator of the system; converts the initial sub-object based on the weight data through a conversion unit to obtain a target sub-object; determines the performance indicator of the system based on the target sub-object through a second determination unit, thereby achieving the technical effect of improving the accuracy of system performance judgment and solving the technical problem of low accuracy of system performance judgment.

[0137] It should be noted that the acquisition unit 401, the analysis unit 402, the first determination unit 403, the conversion unit 404, and the second determination unit 405 correspond to steps S201 to S204 in Example 1, and the four modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above-mentioned embodiment 1. It should be noted that the above-mentioned modules or units can be hardware components or software components stored in a memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n), and the above-mentioned modules can also be run in the computer terminal 10 provided in the embodiment 1 as part of the device.

[0138] Example 3

[0139] An embodiment of the present application may provide an electronic device, Figure 5 is a structural block diagram of an electronic device according to an embodiment of the present application. Figure 5 As shown, the electronic device may include: one or more ( Figure 5 (only one is shown) processor 502, memory 504, storage controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0140] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0141] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain the system log information; input the log information into the defect perception model for analysis to obtain a word set corresponding to the log information, wherein the defect perception model is trained based on the system log information samples and the word set samples corresponding to the log information samples, the word set is used to characterize the abnormal data in the log information, and the word set samples are used to characterize the abnormal data samples in the log information samples; determine the weight data corresponding to at least one initial sub-object in the word set, wherein the weight data is used to characterize the importance of the initial sub-object to the performance indicators of the system; based on the weight data, transform the initial sub-object to obtain the target sub-object; based on the target sub-object, determine the performance indicators of the system.

[0142] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: determine the word vector corresponding to the initial sub-object; determine the weight data based on the word vector and the performance indicator parameters of the system.

[0143] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: determine the target type of the word vector based on the performance indicator parameters; determine the target weight data of the target type as the weight data corresponding to the initial sub-object.

[0144] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: in response to the similarity between the vector corresponding to the performance indicator parameter and the word vector being greater than a similarity threshold, determining the type corresponding to the performance indicator parameter as the target type.

[0145] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: based on the type of word vector, divide at least one initial sub-object to obtain at least one subset, wherein the subset includes multiple sub-objects, and the multiple sub-objects are used to form the initial sub-object; based on at least one target sub-object in the subset, obtain the average value corresponding to the subset, wherein the average value is used to characterize the sub-performance indicator corresponding to the type; based on at least one average value corresponding to at least one subset, determine the performance indicator.

[0146] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: in response to the average value meeting the target value, determine the performance indicator that meets the performance indicator requirements; in response to the average value not meeting the target value, determine the performance indicator that does not meet the performance indicator requirements.

[0147] By adopting the embodiment of the present application, a method for determining the performance indicators of a system is provided, which automatically pulls the log information of the system, identifies the log information, obtains a word set in the log information, determines the weight data corresponding to the word set, and transforms the initial sub-objects in the word set based on the weight data to obtain the target sub-object. Based on the target sub-object, the performance indicator of the system is determined. The performance indicator can be used to determine the performance of the system, thereby achieving the technical effect of improving the accuracy of system performance judgment and solving the technical problem of low accuracy of system performance judgment.

[0148] It can be understood by those skilled in the art that Figure 5 The structure shown is for illustration only, and the electronic device may also be a terminal device such as a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, abbreviated as MID), a personal access device (Personal Access Device, abbreviated as PAD), etc. Figure 5 The structure of the electronic device is not limited. Figure 5 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 5 Different configurations are shown.

[0149] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0150] Example 4

[0151] The embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the method for determining the performance indicator of the system provided in the first embodiment.

[0152] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0153] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing the steps of the method for determining the performance indicator of the system.

[0154] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0155] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0156] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0157] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0158] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0159] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), mobile hard disk, disk or optical disk and other media that can store program codes.

[0160] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for determining a performance indicator of a system, characterized in that: include: Get the system log information; Input the log information into a defect perception model for analysis to obtain a word set corresponding to the log information, wherein the defect perception model is trained based on log information samples of the system and word set samples corresponding to the log information samples, the word set is used to characterize abnormal data in the log information, and the word set samples are used to characterize abnormal data samples in the log information samples; Determining weight data corresponding to at least one initial sub-object in the word set, wherein the weight data is used to characterize the importance of the initial sub-object to the performance index of the system; Based on the weight data, transform the initial sub-object to obtain a target sub-object; Based on the target sub-object, a performance indicator of the system is determined.

2. The method according to claim 1, characterized in that The determining of weight data corresponding to at least one initial sub-object in the word set includes: Determine the word vector corresponding to the initial sub-object; The weight data is determined based on the word vector and the performance indicator parameters of the system.

3. The method according to claim 2, characterized in that The determining the weight data based on the word vector and the performance indicator parameter of the system includes: Based on the performance indicator parameter, determining a target type of the word vector; The target weight data of the target type is determined as the weight data corresponding to the initial sub-object.

4. The method according to claim 3, characterized in that The determining the target type of the word vector based on the performance indicator parameter includes: In response to the similarity between the vector corresponding to the performance indicator parameter and the word vector being greater than a similarity threshold, the type corresponding to the performance indicator parameter is determined as the target type.

5. The method according to claim 3, characterized in that: The determining the performance indicator of the system based on the target sub-object includes: Based on the type of the word vector, at least one of the initial sub-objects is divided to obtain at least one subset, wherein the subset includes a plurality of sub-objects, and the plurality of sub-objects are used to form the initial sub-object; Based on at least one target sub-object in the subset, obtaining an average value corresponding to the subset, wherein the average value is used to characterize a sub-performance indicator corresponding to the type; The performance indicator is determined based on at least one of the average values ​​corresponding to at least one of the subsets.

6. The method according to claim 5, characterized in that The determining the performance indicator based on at least one of the average values ​​corresponding to at least one of the subsets includes: In response to the average value satisfying the target value, determining the performance indicator that satisfies the performance indicator requirement; In response to the average value not satisfying the target value, determining that the performance indicator does not satisfy the performance indicator requirement.

7. A device for determining a performance indicator of a system, characterized in that: include: An acquisition unit, used to acquire system log information; An analysis unit, configured to input the log information into a defect perception model for analysis to obtain a word set corresponding to the log information, wherein the defect perception model is trained based on log information samples of the system and word set samples corresponding to the log information samples, the word set is used to characterize abnormal data in the log information, and the word set samples are used to characterize abnormal data samples in the log information samples; A first determining unit, configured to determine weight data corresponding to at least one initial sub-object in the word set, wherein the weight data is used to characterize the importance of the initial sub-object to a performance indicator of the system; A conversion unit, configured to convert the initial sub-object based on the weight data to obtain a target sub-object; The second determining unit is configured to determine a performance indicator of the system based on the target sub-object.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 6 when running.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images