Share insights between pre- and post-deployment to enhance cloud workload security

Abstract

Embodiments can provide techniques that can provide more accurate and actionable alerts by a cloud workload security system in order to improve overall cloud workload security. For example, in an embodiment, a method can be implemented in a computer system that includes a processor, a memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor, and the method can include generating performance and security information related to a software system during development of the software system, generating performance and security information related to the software system during a deployment operation of the software system, matching the performance and security information generated during the development of the software system with the performance and security information generated during the deployment operation of the software system to determine performance and security alerts to be elevated, and reporting the elevated performance and security alerts.

Description

Background Technology

[0001] This invention relates to a technology that can provide more accurate and actionable alerts through a cloud workload security system to improve overall cloud workload security.

[0002] Cloud workload security is one of the fastest-growing security issues today. Runtime workload security systems often generate alerts with high false positive rates. A high false positive rate can cause systems to ignore many positive alerts and thus miss some true positives. As a result, many attacks can evade security defenses. Furthermore, pre-deployment security testing can suffer from the same problem, and due to the large number of false positive alerts in pre-deployment security testing, true positives (valid alerts) may sometimes be ignored and / or not verified.

[0003] Therefore, there is a need for technologies that can provide more accurate and actionable alerts through cloud workload security systems to improve overall cloud workload security. Summary of the Invention

[0004] The embodiments can provide techniques that can improve overall cloud workload security by providing more accurate and actionable alerts through cloud workload security systems.

[0005] Implementation examples can combine development testing (pre-deployment) and runtime (post-deployment) monitoring of cloud software systems to share insights between pre-deployment and post-deployment software environments, providing more accurate and actionable alerts. For example, processing such as application log analysis can be used to share insights. Similarly, call graphs can be used for local-log-sequence sharing. Furthermore, error handling can serve as an indication of security vulnerabilities.

[0006] For example, in one embodiment, a method can be implemented in a computer system including a processor, memory accessible to the processor, and computer program instructions stored in the memory and executable by the processor. The method may include generating performance and security information related to the software system during software system development, generating performance and security information related to the software system during software system deployment operations, matching the performance and security information generated during software system development with the performance and security information generated during software system deployment operations to determine performance and security alerts to be improved, and reporting the improved performance and security alerts.

[0007] In embodiments, generating performance and security information related to the software system during its development may include at least one of the following: performing static and dynamic tests of the software system's code during development to collect a history of alerts generated during testing; performing dynamic tests of subsystems of the software system's code during development to generate application logs for the generated alerts; and performing dynamic tests of code units of the software system during development to generate error handling logs for the generated alerts. The method may also include using a generated call graph representing call relationships between routines in the software system to associate each generated alert with an immediate log sequence. Generating performance and security information related to the software system during deployment operations may include monitoring the deployment operations of the software and collecting alerts generated by the monitoring. Generating performance and security information related to the software system during deployment operations may include collecting information about configuration changes and / or alerts about configuration changes, and this matching includes matching the collected information about configuration changes and / or alerts about configuration changes with the performance and security information generated during the deployment operations of the software system to determine performance and security alerts to escalate. Matching performance and security information generated during software system development with that generated during software system deployment can include using the history of alerts generated during testing to determine the root cause of alerts generated by monitoring, and boosting either the monitored alert or both the monitored alert and the alerts in the history of alerts generated during testing when the root cause of the monitored alert matches a recent alert in the history of alerts generated during testing. Matching the root cause of monitored alerts with recent alerts in the alert history can be based on similar log sequences or log groups and can be performed using machine learning techniques. Matching performance and security information generated during the development of a software system with performance and security information generated during the deployment of the software system can also include matching the root cause of alerts generated by monitoring with the most recent alerts in the alert history based on similar log sequences or log groups and using machine learning techniques; matching unit test error handling logs of alerts generated during testing with alerts generated by monitoring by checking application logs using the most recent time window; searching for matches with known log signatures of the problem; increasing the severity of alerts generated by monitoring when at least one match is found; and associating information related to alerts generated by monitoring with the source code that caused the alerts generated during testing.

[0008] In an embodiment, the system may include a processor, a memory accessible to the processor, and computer program instructions stored in the memory and executable by the processor to perform actions such as generating performance and security information related to the software system during software system development, generating performance and security information related to the software system during software system deployment operations, matching the performance and security information generated during software system development with the performance and security information generated during software system deployment operations to determine performance and security alerts to be improved, and reporting the improved performance and security alerts.

[0009] In an embodiment, the computer program product may include a non-transitory computer-readable storage device having program instructions embodied therein, which are executable by a computer to cause the computer to perform a method comprising generating performance and security information related to the software system during the development of the software system, generating performance and security information related to the software system during deployment operations, matching the performance and security information generated during the development of the software system with the performance and security information generated during deployment operations of the software system to determine performance and security alerts to be improved, and reporting the improved performance and security alerts. Attached Figure Description

[0010] The details of the invention regarding its structure and operation can be best understood by referring to the accompanying drawings, in which the same reference numerals and designations refer to the same elements.

[0011] Figure 1 An exemplary environment according to an embodiment of the present technology is shown, in which cloud workload applications can be developed, tested, and used in a production deployment.

[0012] Figure 2 This is an exemplary flowchart of the operation process according to an embodiment of the present technology.

[0013] Figure 3 This is an exemplary block diagram of a computer system in which the processes described in the embodiments herein can be implemented. Detailed Implementation

[0014] The embodiments can provide techniques that can improve overall cloud workload security by providing more accurate and actionable alerts through cloud workload security systems.

[0015] Implementation examples can combine development testing (pre-deployment) and runtime (post-deployment) monitoring of cloud software systems to share insights between pre-deployment and post-deployment software environments, providing more accurate and actionable alerts. For example, processing such as application log analysis can be used to share insights. Similarly, call graphs can be used for local-log-sequence sharing. Furthermore, error handling can serve as an indication of security vulnerabilities.

[0016] Continuous Integration (CI) is a coding technique where small changes to code are made and frequently checked into a version control repository within a version control system. This allows CI to provide a consistent and automated way to build, package, and test applications. This allows for more frequent code changes to be committed, leading to better collaboration and software quality. Continuous Delivery (CD) automates the delivery of applications to chosen infrastructure environments. Most teams work in multiple environments besides production, such as development and testing environments, and CD ensures that there are automated ways to push code changes to them. Continuous Integration and Continuous Delivery (CI / CD) combine these two technologies into an integrated coding and testing workflow.

[0017] Figure 1 An exemplary environment 100 is illustrated, in which cloud workload applications can be developed, tested, and used in a production deployment. In this example, environment 100 may include a CI / CD development environment 102 and a cloud production deployment 104. The CI / CD development environment 102 may include processing blocks such as a code merging block 106, a static testing block 108, a build block 110, a dynamic testing block 112, and a deployment block 114. The code merging block 106 can perform code merging (also known as integration), which is the operation of coordinating multiple changes made to software source code files under version control. For example, if files are modified on two separate branches, the different branches can be merged to form a single set of files containing all the sets of changes. In this embodiment, automatic merging may be performed by the version control system in the CI / CD development environment 102 to coordinate changes that occur simultaneously (in a logical sense). If automatic merging is ineffective or does not work, manual merging may be performed if necessary.

[0018] Static test block 108 can perform testing and analysis of computer software without actually executing a program, such as a merged version of the source code and / or object code output from code merging block 106. In embodiments, such analysis can be performed by automated tools. Build block 110 can perform the construction of a complete application or software system, including compiling source code into object code, linking object code and library object code, and other software resources to form an executable code set. Dynamic test block 112 can perform analysis of computer software by executing a program on a real or virtual processor to monitor the actual behavior of the software during operation. Deployment block 114 can perform the deployment of computer software, including, for example, generating an installation package to allow easy installation of the software, and installing the software and associated software required for operation.

[0019] Cloud production deployment 104 can provide an environment for real-time or production operations of computer software applications and systems, and can provide an environment that users (humans, client computers, other servers, etc.) can directly interact with. Cloud production deployment 104 may include cloud workload monitoring block 116 and CI / CD insights 118. Cloud workload monitoring block 116 can monitor the operation and workload parameters of the deployed software in cloud production deployment 104, and can generate alerts based on the monitored operations. CI / CD insights 118 may include information about the operation of the deployed software obtained from CI / CD development environment 102, such as alert history, and can be used by cloud workload monitoring block 116 to provide enhanced information about the operation of the deployed software in cloud production deployment 104.

[0020] exist Figure 2 An exemplary process 200 according to this embodiment is illustrated. It is best to combine... Figure 1 Come and check Figure 2Processing from 200 to 202, in which CI / CD tests can be executed, and the alert history from such tests can be saved, for example, in CI / CD Insights 118. Such CI / CD tests may include, for example, static tests (such as configuration tests and code tests performed by static test block 108) and dynamic tests performed by dynamic text block 112. During static tests, a call graph can be created to represent the call relationships between routines in the computer software. Each generated alert can be associated with its possible immediate log sequence (a finite number of steps) as recorded in the call graph. At 204, dynamic tests of the CI / CD subsystem can be executed, and application logs of the generated alerts can be collected. The collected application logs can be saved, for example, in CI / CD Insights 118. At 206, CI / CD unit tests can be executed, and error handling logs can be collected, which may include sequences indicating unexpected control flow. Furthermore, information about configuration changes and / or alerts about configuration changes can be collected. Additionally, the log signatures of the error handling unit tests can be characterized. The collected error handling logs and features can be stored in, for example, CI / CD Insights 118.

[0021] Once the software has been deployed to cloud production deployment 104, i.e., after deployment, at 208, the cloud workload monitoring block 116 can monitor the operation of cloud production deployment 104 and collect all alerts generated through such monitoring. When such a workload alert occurs, the application logs can be checked using the most recent time window to detect recent log anomalies. At 210, the CI / CD alert history 118 can be used for root cause analysis of each workload alert. At 212, if the root cause of a workload alert matches a recent alert in the CI / CD alert history 118, the alert score of that workload alert can be increased, or the alert scores of both the workload alert and the alert in the CI / CD alert history 118 can be increased, and the root cause analysis can be reported, for example, by publishing it to CI on GitHub. For example, process 200 can be linked to Git commits based on code commits. That is, information related to the alert can be associated with the source code identified as causing the alert in the version control system of the CI / CD development environment 102.

[0022] The workload alerts can be matched with alerts found during static analysis (CI / CD alert history 118) based on similar log sequences (or log groups). Matching techniques can be used to determine the matches, including machine learning techniques and techniques such as one-to-one matching, sequence alignment, DNA comparison, etc. For example, matching can be performed via a DNN (Deep Neural Network) classifier trained to associate static test results, configuration changes, Git logs, and configuration alert logs from the development phase, as well as sequential application log data from the testing and initial deployment phases. These can then be temporally matched with alerts during runtime. At 214, unit test error handling logs can be matched with workload alerts. An example can perform such matching by examining application logs collected at 204 using a recent time window. Then, at 206, processing 200 can search for matches with known log signatures of the problematic scenario. When one or more such matches are found, the severity of the workload alert can be increased or amplified, and can be reported, for example, by linking to the code. That is, information related to the alert can be associated with the source code identified as causing the alert in the version control system of CI / CD development environment 102. Furthermore, alerts found in the post-deployment phase can be matched with information and / or alerts about configuration changes collected during CI / CD testing.

[0023] exist Figure 3 An exemplary block diagram of a computer system 300 is shown, in which the processes involved in the embodiments described herein can be implemented. The computer system 300 can be implemented using one or more programmed general-purpose computer systems, such as embedded processors, system-on-a-chip, personal computers, workstations, server systems, and minicomputers or mainframes, or can be implemented in a distributed, networked computing environment. The computer system 300 may include one or more processors CPUs 302A-302N, input / output circuitry 304, a network adapter 306, and memory 308. The CPUs 302A-302N execute program instructions to perform the functions of this communication system and method. Typically, the CPUs 302A-302N are one or more microprocessors. Figure 3An embodiment is shown in which the computer system 300 is implemented as a single multiprocessor computer system, wherein multiple processors CPUs 302A-302N share system resources, such as memory 308, input / output circuitry 304, and network adapter 306. However, the communication system and method also include embodiments in which the computer system 300 is implemented as multiple networked computer systems, which may be single-processor computer systems, multiprocessor computer systems, or a combination thereof.

[0024] Input / output circuitry 304 provides the ability to input data to or output data to computer system 300. For example, input / output circuitry may include input devices such as a keyboard, mouse, touchpad, trackball, scanner, analog-to-digital converter, etc.; output devices such as video adapters, monitors, printers, etc.; and input / output devices such as modems. Network adapter 306 interfaces computer system 300 with network 310. Network 310 may be any public or private LAN or WAN, including but not limited to the Internet.

[0025] Memory 308 stores program instructions executed by CPUs 302A-302N and data used and processed by CPUs 302A-302N to perform the functions of computer system 300. Memory 308 may include, for example, electronic memory devices such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), flash memory, etc., and electromechanical memory such as disk drives, tape drives, optical disk drives, etc., which may use an Integrated Drive Electronics (IDE) interface, or its variants or enhancements, such as Enhanced IDE (EIDE) or Ultra Direct Memory Access (UDMA), or a Small Computer System Interface (SCSI) based interface, or its variants or enhancements, such as Fast SCSI, Wide SCSI, Fast and Wide SCSI, etc., Serial or Advanced Technology Accessory (SATA), or its variants or enhancements, or a Fibre Channel Arbitrated Loop (FC-AL) interface.

[0026] The contents of memory 308 can change according to the functions that computer system 300 is programmed to perform. Figure 3 The example shown illustrates exemplary memory contents of routines and data representing embodiments of the processes described above. However, those skilled in the art will recognize that these routines and the associated memory contents may not be included in a single system or device, but may be distributed across multiple systems or devices based on known engineering considerations. This system and method may include any and all such arrangements.

[0027] exist Figure 3 In the example shown, memory 308 may include CI / CD routines 312 and cloud production deployment 324. CI / CD routines 312 may include code merging routines 314, static testing routines 316, build routines 318, dynamic testing routines 320, and deployment routines 322. Cloud production deployment 324 may include cloud workload monitoring routines 326, CI / CD insight data 328, and operating system 330. CI / CD routines 312 may include software routines to implement... Figure 1 The CI / CD development environment 102 shown is as described above. Code merging routine 314 may include software routines to perform code merging (also known as integration), which is the operation of coordinating multiple changes made to software source code files under version control, as described above. Static testing routine 316 may include software routines to perform testing and analysis of computer software without actually executing the program, as described above. Build routine 318 may include software routines to perform the construction of a complete application or software system, including compiling source code into object code, linking object code and library object code, and other software resources, to form an executable set of code, as described above. Dynamic testing routine 320 may include software routines to perform analysis of computer software, which is performed by monitoring the actual behavior of the software during operation by executing programs on a real or virtual processor, as described above. Deployment routine 322 may include software routines to perform the deployment of computer software, including, for example, generating an installation package to allow easy installation of the software, and installing the software, associated software required for operation, as described above. Cloud production deployment 324 may include software routines to provide an environment for real-time or production operations of computer software applications and systems, and may provide an environment that users (humans, client computers, other servers, etc.) can directly interact with, as described above. Cloud workload monitoring routine 326 may include software routines to perform monitoring of the operations and workload parameters of the deployed software in the cloud production deployment, and may generate alerts based on the monitored operations, as described above. CI / CD insight data 328 may include information about the operations of the deployed software obtained from CI / CD development environment 102, such as alert history, and may be used to provide enhanced information about the operations of the deployed software in the cloud production deployment, as described above. Operating system 330 can provide overall system functionality.

[0028] like Figure 3As shown, this communication system and method can be implemented on one or more systems providing multiprocessor, multitasking, multiprocessing, and / or multithreaded computing, as well as on systems providing only single-processor, single-threaded computing. Multiprocessor computing involves using more than one processor to perform computation. Multitasking computing involves using more than one operating system task to perform computation. A task is an operating system concept, referring to a combination of a program being executed and bookkeeping information used by the operating system. Whenever a program is executed, the operating system creates a new task for it. This task is like an envelope for the program, as it uses a task number to identify the program and attaches other bookkeeping information to it. Many operating systems are capable of running many tasks simultaneously and are called multitasking operating systems. Multitasking is the ability of an operating system to execute more than one executable program simultaneously. Each executable program runs in its own address space, meaning that executable programs cannot share any of their memory. This is advantageous because it is impossible for any program to corrupt the execution of any other program running on the system. However, programs cannot exchange any information except through the operating system (or by reading files stored on the file system). Multi-process computing is similar to multi-tasking computing because the terms task and process are often used interchangeably, although some operating systems distinguish between the two.

[0029] This invention can be a system, method, and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to perform aspects of the invention. The computer-readable storage medium may be a tangible device capable of retaining and storing instructions for use by an instruction execution device.

[0030] Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory sticks, floppy disks, mechanical encoding devices (such as punched cards or raised structures in recesses) on which instructions are recorded, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses transmitted through fiber optic cables), or electrical signals transmitted through wires.

[0031] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device, or downloaded via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network) to an external computer or external storage device. The network may include copper cables, optical fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the suitable computing / processing device.

[0032] Computer-readable program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages ​​such as Smalltalk, C++, etc.) and procedural programming languages ​​(such as the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including local area network (LAN) or wide area network (WAN)) or may be connected to an external computer (e.g., via the Internet provided by an Internet service provider). In some embodiments, to perform aspects of this invention, electronic circuits, including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute the computer-readable program instructions by personalizing the electronic circuits with state information from the computer-readable program instructions.

[0033] This document describes aspects of the invention with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0034] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more boxes of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can instruct a computer, programmable data processing apparatus, and / or other device to operate in a particular manner, such that the computer-readable storage medium in which the instructions are stored includes an article of writing comprising instructions for implementing aspects of the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0035] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer-implemented process, such that the instructions, which execute on the computer, other programmable apparatus or other device, perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0036] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than indicated in the figures. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order, depending on the functions involved. It will also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0037] Although specific embodiments of the invention have been described, those skilled in the art will understand that other equivalent embodiments exist. Therefore, it should be understood that the invention is not limited to the embodiments specifically shown, but only to the scope of the appended claims.

Claims

1. A method implemented in a computer system, the computer system comprising a processor, a memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor, the method comprising: During the development of the software system, first performance and security information related to the software system is generated; Secondary performance and security information related to the software system is generated during the deployment operation of the software system; The first performance and security information generated during the development of the software system is matched with the second performance and security information generated during the deployment operation of the software system to determine one or more performance and security alerts to be improved. as well as The report includes improved performance and security alerts. The generation of the second performance and security information related to the software system during the deployment operation of the software system includes: collecting at least one of information about one or more configuration changes and at least one of one or more alarms generated about the one or more configuration changes; and the matching includes: matching the collected information about at least one of the one or more configuration changes and the one or more alarms generated about the one or more configuration changes with the second performance and security information generated during the deployment operation of the software system to determine the performance to be improved and the one or more performance and security alarms.

2. The method according to claim 1, wherein, The generation of the first performance and security information related to the software system during the development of the software system includes at least one of the following: Static and dynamic tests of the software system's code are performed during the development of the software system to collect a history of one or more generated alerts; During the development of the software system, first dynamic tests are performed on one or more subsystems of the software system's code to generate one or more application logs of the one or more generated alerts; as well as During the development of the software system, a second dynamic test is performed on one or more units of the software system's code to generate one or more error handling logs for the one or more generated alerts.

3. The method according to claim 2, further comprising: Using one or more generated call graphs that represent one or more call relationships between routines in the software system, each generated alarm in the one or more generated alarms is associated with an instant log sequence.

4. The method according to claim 3, wherein, Generating the second performance and security information related to the software system during the deployment operation of the software system includes: monitoring the deployment operation of the software and collecting one or more monitoring alerts generated by monitoring the deployment operation of the software.

5. The method according to claim 4, wherein, Matching the first performance and security information generated during the development of the software system with the second performance and security information generated during the deployment of the software system includes: The history of alerts generated during the static and dynamic tests is used to determine the root cause of alerts generated by monitoring the deployment operations of the software; and In response to an alarm generated by monitoring the deployment of the software, if the root cause is matched with the most recent alarm in the history of alarms generated during the static and dynamic testing, either the alarm generated by monitoring the deployment of the software or both the alarm generated by monitoring the deployment of the software and the alarm in the history of alarms generated during the static and dynamic testing are amplified.

6. The method according to claim 5, wherein, Matching the first performance and security information generated during the development of the software system with the second performance and security information generated during the deployment operation of the software system to determine the one or more performance and security alerts to be improved further includes: Based on one or more similar log sequences and at least one of one or more log groups, the root cause of an alert generated by monitoring the deployment operation of the software is matched with the most recent alert in the history of the alerts, and one or more machine learning techniques are used to perform the matching; By examining the one or more application logs using a recent time window, one or more unit test error handling logs containing alerts generated during static and dynamic testing are matched with one or more monitoring alerts generated by monitoring the deployment of the software. One or more matches with known log signatures of the problem are searched, and when at least one match is found, the severity of the alert generated by monitoring the deployment of the software is increased; and Information relating to the one or more alerts generated by monitoring the deployment of the software will be associated with the source code that caused the one or more alerts to be generated during the static and dynamic tests.

7. A computer system comprising a processor, a memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor to perform the following operations: During the development of the software system, first performance and security information related to the software system is generated; Secondary performance and security information related to the software system is generated during the deployment operation of the software system; The first performance and security information generated during the development of the software system is matched with the second performance and security information generated during the deployment operation of the software system to determine one or more performance and security alerts to be improved. as well as The report includes improved performance and security alerts. The generation of the second performance and security information related to the software system during the deployment operation of the software system includes: collecting at least one of information about one or more configuration changes and at least one of one or more alarms generated about the one or more configuration changes; and the matching includes: matching the collected information about at least one of the one or more configuration changes and the one or more alarms generated about the one or more configuration changes with the second performance and security information generated during the deployment operation of the software system to determine the performance to be improved and the one or more performance and security alarms.

8. The computer system according to claim 7, wherein, The generation of the first performance and security information related to the software system during the development of the software system includes at least one of the following: Static and dynamic tests of the software system's code are performed during the development of the software system to collect a history of one or more generated alerts; During the development of the software system, first dynamic tests are performed on one or more subsystems of the software system's code to generate one or more application logs of the one or more generated alerts; as well as During the development of the software system, a second dynamic test is performed on one or more units of the software system's code to generate one or more error handling logs for the generated alerts.

9. The computer system according to claim 8, further comprising: Using one or more generated call graphs that represent one or more call relationships between routines in the software system, each generated alarm in the one or more generated alarms is associated with an instant log sequence.

10. The computer system according to claim 9, wherein, Generating the second performance and security information related to the software system during the deployment operation of the software system includes: monitoring the deployment operation of the software and collecting one or more monitoring alerts generated by monitoring the deployment operation of the software.

11. The computer system according to claim 10, wherein, Matching the first performance and security information generated during the development of the software system with the second performance and security information generated during the deployment of the software system includes: The history of alerts generated during the static and dynamic tests is used to determine the root cause of alerts generated by monitoring the deployment operations of the software; and In response to an alarm generated by monitoring the deployment of the software, if the root cause is matched with the most recent alarm in the history of alarms generated during the static and dynamic testing, either the alarm generated by monitoring the deployment of the software or both the alarm generated by monitoring the deployment of the software and the alarm in the history of alarms generated during the static and dynamic testing are amplified.

12. The computer system according to claim 11, wherein, Matching the first performance and security information generated during the development of the software system with the second performance and security information generated during the deployment operation of the software system to determine the one or more performance and security alerts to be improved further includes: Based on one or more similar log sequences and at least one of one or more log groups, the root cause of an alert generated by monitoring the deployment operation of the software is matched with the most recent alert in the history of the alerts, and one or more machine learning techniques are used to perform the matching; By examining the one or more application logs using a recent time window, one or more unit test error handling logs of alerts generated during the static and dynamic tests are matched with one or more monitoring alerts generated by monitoring the deployment operations of the software. One or more matches with known log signatures of the problem are searched, and when at least one match is found, the severity of the alert generated by monitoring the deployment operations of the software is increased; and Information relating to the one or more alerts generated by monitoring the deployment of the software will be associated with the source code that caused the one or more alerts to be generated during the static and dynamic tests.

13. A computer program product comprising program instructions executable by a computer to cause the computer to perform a method, the method comprising: During the development of the software system, first performance and security information related to the software system is generated; Secondary performance and security information related to the software system is generated during the deployment operation of the software system; The first performance and security information generated during the development of the software system is matched with the second performance and security information generated during the deployment operation of the software system to determine one or more performance and security alerts to be improved. as well as The report includes improved performance and security alerts. The generation of the second performance and security information related to the software system during the deployment operation of the software system includes: collecting at least one of information about one or more configuration changes and at least one of one or more alarms generated about the one or more configuration changes; and the matching includes: matching the collected information about at least one of the one or more configuration changes and the one or more alarms generated about the one or more configuration changes with the second performance and security information generated during the deployment operation of the software system to determine the performance to be improved and the one or more performance and security alarms.

14. The computer program product according to claim 13, wherein, The generation of the first performance and security information related to the software system during the development of the software system includes at least one of the following: Static and dynamic tests of the software system's code are performed during the development of the software system to collect a history of one or more generated alerts; During the development of the software system, first dynamic tests are performed on one or more subsystems of the software system's code to generate one or more application logs of the one or more generated alerts; as well as During the development of the software system, a second dynamic test is performed on one or more units of the software system's code to generate one or more error handling logs for the generated alerts.

15. The computer program product according to claim 14, further comprising: Using one or more generated call graphs that represent one or more call relationships between routines in the software system, each generated alarm in the one or more generated alarms is associated with an instant log sequence.

16. The computer program product according to claim 15, wherein, Matching the first performance and security information generated during the development of the software system with the second performance and security information generated during the deployment of the software system includes: The history of alerts generated during the static and dynamic tests is used to determine the root cause of alerts generated by monitoring the deployment operations of the software; and In response to the root cause of an alarm generated by monitoring the deployment of the software being matched with the most recent alarm in the history of alarms generated during the static and dynamic testing, either the alarm generated by monitoring the deployment of the software or both the alarm generated by monitoring the deployment of the software and the alarm in the history of alarms generated during the static and dynamic testing are amplified. The root cause of an alert generated by monitoring the deployment of the software is matched with the most recent alerts in the alert history based on similar log sequences or log groups, and machine learning techniques are used to perform the matching.

17. The computer program product according to claim 16, wherein, Matching the first performance and security information generated during the development of the software system with the second performance and security information generated during the deployment operation of the software system to determine the one or more performance and security alerts to be improved further includes: Based on one or more similar log sequences and at least one of one or more log groups, the root cause of an alert generated by monitoring the deployment operation of the software is matched with the most recent alert in the history of the alerts, and one or more machine learning techniques are used to perform the matching; By examining the one or more application logs using a recent time window, one or more unit test error handling logs of alerts generated during the static and dynamic tests are matched with one or more monitoring alerts generated by monitoring the deployment operations of the software. One or more matches with known log signatures of the problem are searched, and when at least one match is found, the severity of the alert generated by monitoring the deployment operations of the software is increased; and Information relating to the one or more alerts generated by monitoring the deployment of the software will be associated with the source code that caused the one or more alerts to be generated during the static and dynamic tests.

Images