AI-powered vulnerability testing and countermeasure generation system

Abstract

In vulnerability verification and countermeasure document generation using AI models, there were issues such as insufficient verification of ownership of the target of verification, non-destructive execution, and reflection of verification results in subsequent evaluations. Furthermore, because the verification template and countermeasure document varied for the same input, the reproducibility of verification results, auditability, and consistency of countermeasure documents were compromised. [Solution] A means for ownership verification, an AI template generation means equipped with an idempotent key, a non-destructive execution means, a means for reflecting verification result correction values ​​in the effective technical risk indicator, and an AI countermeasure text generation means equipped with a multi-axis combination key are integrated and the reproducibility of the verification template and countermeasure text is guaranteed by a two-stage independent idempotent key configuration.

Description

Technical Field

[0001] The present invention relates to active verification of vulnerabilities in the field of cybersecurity and automatic generation of countermeasure texts for vulnerabilities, and in particular, relates to an apparatus, method, and program that use artificial intelligence (AI) models such as Large Language Models (LLMs) to dynamically generate verification templates for a system to be verified, actively verify vulnerabilities by executing the templates in a non-destructive manner, and generate vulnerability countermeasure texts by AI based on the verification results.

Background Art

[0002] In cybersecurity vulnerability management, there is a demand for (a) a technology that does not rely solely on passive detection by scanners, but actively verifies whether each detected vulnerability finding is actually exploitable (Dynamic Application Security Testing (DAST), or vulnerability verification), and (b) a technology that automatically generates countermeasure texts for verified vulnerabilities. In recent years, with the widespread adoption of large language models, attempts to automate both of these (a) and (b) using AI models have been progressing. As the first related prior art group (system of automated penetration testing frameworks), European Patent No. 2,462,716 (Patent Document 1), U.S. Patent Application Publication No. 2021 / 0357507 (Patent Document 2), U.S. Patents No. 7,958,560 / No. 8,095,983 / No. 8,316,447 (Patent Documents 3-5), U.S. Patent No. 6,574,737 (Patent Document 6), U.S. Patent No. 7,073,198 (Patent Document 7), and Chinese Patent Application Publication No. 104,363,236A (Patent Document 8) disclose automated penetration testing, attack tree frameworks, protocol exploitation analysis, network penetration testing, vulnerability detection by reflected packets, or automated vulnerability verification methods, respectively. These systems do not require prior verification of ownership of the system under verification (typically using Domain Name System (DNS) text records, HTTP file trails, email infrastructure domain verification, authentication linkages, etc.), and their inventive contribution lies in executing penetration testing campaigns against any target. In this respect, their architecture is fundamentally different from the ownership verification means of this device. As a second related prior art group (classic lineage of web application dynamic application security verification), U.S. Patent No. 9,015,844 (Patent Document 9), U.S. Patent No. 9,736,177 / No. 7,765,597 (Patent Documents 10-11), U.S. Patent Application Publication No. 2002 / 0010855 / No. 2003 / 0233581 and U.S. Patent No. 6,584,569 (Patent Documents 12-14), and U.S. Patent No. 7,178,166 (Patent Document 15) disclose web scanning, request interception type verification, web integrated crawling and auditing, web vulnerability detection, or workstation vulnerability assessment, respectively. While these lineages disclose vulnerability detection through web crawling and dynamic interaction, they do not disclose (i) dynamic template generation by AI models, (ii) idempotency key mechanisms in said generation, or (iii) reflection of verification result correction values ​​in verification results. As a third related prior art group (AI-driven security verification systems), U.S. Patent No. 12,603,912 / No. 11,962,610 (Patent Documents 16-17), U.S. Patent Application Publication No. 2022 / 0030008 (Patent Document 18), and Japanese Patent Publication No. 2019-511030 (Patent Document 19) disclose broad claims for automated security verification using neural networks, exploit prevention setting adjustment using machine learning, or computer security using artificial intelligence, respectively. While these systems are located in the area closest to the present configuration, (i) Patent Documents 16-17 employ an audited agent-based command and control structure and a distributed breach / attack simulation architecture for multiple target networks and lack means of ownership verification; (ii) Patent Document 18 does not deal with the idempotency of AI output; and (iii) Patent Document 19 lacks specific disclosure regarding the idempotency key mechanism of AI output. As a fourth related prior art group (remote / researcher-assisted systems), U.S. Patent No. 7,346,929 (Patent Document 20), U.S. Patent Application Publication No. 2001 / 0034847 (Patent Document 21), and U.S. Patent No. 10,628,764 (Patent Document 22) disclose remote computer system security audits, client-side vulnerability verifiers via encrypted connections, or automated task generation based on the logical location between the researcher and the system under test, respectively. These systems either require a human researcher as an essential element, or make verification via generic remote audit instructions, or client-side vulnerability verifiers based on the audited agent as their inventive contributions. All of them require a human researcher or the audited agent in the verification flow, fail to satisfy the non-destructive execution constraints of this configuration, and lack disclosures regarding dynamic template generation by AI models and idempotency keys. As a fifth related prior art group (intelligence graph system), U.S. Patent No. 9,886,581 (Patent Document 23) discloses automated construction and defense automation using an intelligence graph. This technology lacks disclosure regarding the idempotency of the AI ​​output and ownership verification / non-destructive execution. As a sixth related prior art (network self-security evaluation system), U.S. Patent No. 6,185,689 (Patent Document 24) discloses vulnerability assessment by connection testing from an arbitrary host to a target. This document does not disclose (a) dynamic template generation by an AI model, (b) idempotency key mechanism, or (c) reflection of verification result correction values ​​in the verification results, in that (i) connection testing is performed from an arbitrary host without prior verification of ownership, and (ii) the verification means is a predefined static test. As a seventh related prior art group (AI-based countermeasure text generation systems), U.S. Patent No. 11,503,061 (Patent Document 25), U.S. Patent No. 7,680,880 (Patent Document 26), U.S. Patent No. 11,308,218 (Patent Document 27), U.S. Patent No. 12,355,798 (Patent Document 28), and U.S. Patent Application Publication No. 2005 / 0015622 and U.S. Patent No. 8,561,175 / 9,094,434 (Patent Documents 29-31) disclose countermeasure plan ranking using machine learning exploitability models, remediation package distribution, open-source software code library countermeasure tools, mitigation prioritization using process-aware analytical attack graphs, or automated policy audits and countermeasure texts, respectively. These approaches disclose one of the following: (i) a ranking of countermeasure plans based on machine learning-trained models, (ii) simple remediation package distribution, (iii) application of specific countermeasures to open-source software code libraries, or (iv) countermeasures initiated by policy audits, but do not disclose (a) the idempotency of the AI ​​model's generation of countermeasure text itself, (b) an independent uniqueness key design including at least four axes: risk band identifier, section classification identifier, output presentation derived attribute, and cryptographic hash of input data, or (c) the link between verification results and countermeasure text generation. Through the design and operational studies of the software as a Continuous Threat Exposure Management (CTEM) service conducted by the applicant, it was confirmed that the following two failure events related to AI model output can systematically occur. (α) Non-idempotency in AI verification template generation: If the AI ​​model generates different verification templates each time for the same system under verification, the reproducibility of the verification results will be impaired, making it difficult to preserve evidence during audits or litigation. (β) Non-idempotency in AI-generated countermeasures: If the AI ​​model generates different countermeasures each time for the same vulnerability, it can lead to loss of customer trust, difficulty in tracking the progress of the response, and race conditions in automated subsequent processing. The above (α) and (β) are failure events observed when an AI model is operated as the core of software for a continuous threat exposure management service, and existing prior art documents, including Patent Documents 1 to 31, have not comprehensively resolved these two phenomena. The non-idempotency of (α) AI verification template generation and (β) AI countermeasure text generation only manifest as individual output variances when evaluating the AI ​​model in a test environment or short-term prototype, and the resulting commercial operation-specific consequences such as (i) difficulty in preserving evidence during audits, (ii) breakdown of output consistency between customers, and (iii) race conditions in subsequent automated processing are not observed. In other words, the AI-driven security systems including Patent Documents 1 to 31 implicitly assume a short-term evaluation environment or a one-off execution scenario, and therefore have no motivation to resolve (α) and (β) simultaneously on the same platform. Through the applicant's design and operational studies of the software as a Continuous Threat Exposure Management (CTEM) service, it is found that not only are (α) and (β) observed independently, but the non-idempotent template generated in (α) fluctuates the cryptographic hash of the input data in (β), resulting in a two-stage chain event where the countermeasure text for the same vulnerability finding loses consistency. This invention structurally solves this two-stage chain using two types of independent idempotent key mechanisms arranged on a unified base. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] European Patent No. 2,462,716 [Patent Document 2] U.S. Patent Application Publication No. 2021 / 0357507 Specification [Patent Document 3] U.S. Patent No. 7,958,560 [Patent Document 4] U.S. Patent No. 8,095,983 [Patent Document 5] U.S. Patent No. 8,316,447 [Patent Document 6] U.S. Patent No. 6,574,737 [Patent Document 7] U.S. Patent No. 7,073,198 [Patent Document 8] Chinese Patent Application Publication No. 104,363,236A [Patent Document 9] U.S. Patent No. 9,015,844 [Patent Document 10] U.S. Patent No. 9,736,177 [Patent Document 11] U.S. Patent No. 7,765,597 [Patent Document 12] U.S. Patent Application Publication No. 2002 / 0010855 [Patent Document 13] U.S. Patent Application Publication No. 2003 / 0233581 [Patent Document 14] U.S. Patent No. 6,584,569 [Patent Document 15] U.S. Patent No. 7,178,166 [Patent Document 16] U.S. Patent No. 12,603,912 [Patent Document 17] U.S. Patent No. 11,962,610 [Patent Document 18] U.S. Patent Application Publication No. 2022 / 0030008 [Patent Document 19] Special Publication No. 2019-511030 [Patent Document 20] U.S. Patent No. 7,346,929 Specification

Patent Document 21

Patent Document 22

Patent Document 23

Patent Document 24

Patent Document 25

Patent Document 26

Patent Document 27

Patent Document 28

Patent Document 29

Patent Document 30

Patent Document 31

Non-Patent Document

[0004]

Non-Patent Document 1

Non-Patent Document 2

Non-Patent Document 3

[0005] Conventional vulnerability verification and countermeasure generation using AI models had the following challenges: (a) If the verification was performed without prior verification of the ownership of the system under verification, there was a problem in ensuring ethical or legal legitimacy. (b) The generation of verification templates using AI models resulted in variations in output even for the same input, impairing the reproducibility of verification results and making it difficult to preserve evidence during audits or litigation. (c) There was a problem where executing a verification template resulted in permanent changes to the system under verification, which could damage the system under verification. (d) There was a lack of a mechanism for automatically linking the verification results to the subsequent layer (effective technology risk indicator scoring device), and the linking of verification and evaluation relied on manual intervention. (e) AI models generated response messages, but the output varied even for the same input, leading to a loss of customer trust, difficulty in tracking the progress of responses, and conflicts in subsequent automated processing. (f) In the conventional configuration, where (a) through (e) above are implemented as separate, independent patches, there was a problem in that it was not possible to structurally prevent complex inconsistencies caused by the interaction of these five types of means (e.g., discrepancies between verification results and countermeasure text, the same verification results for the same input but different countermeasure text). The present invention aims to provide an AI-based vulnerability verification and countermeasure generation apparatus, method, and program that solve these problems. [Means for solving the problem]

[0006] The present invention is based on a common principle of designing a unique key for AI output, and provides two types of AI generation means (AI template generation means and AI countermeasure text generation means) that follow this principle, each equipped with an independent idempotent key mechanism, and integrates these with ownership verification means, non-destructive execution means, and verification result reflection means to solve the above problems (a) to (f). The ownership verification means confirms the declaration of ownership by the administrator of the system under verification via a predetermined ownership relationship verification mechanism (at least one of the following: Domain Name System (DNS) text records, HTTP file trails, email infrastructure domain verification, authentication linkage, or a combination thereof). The AI ​​template generation means returns the same template for the same input using a cache reference mechanism, based on the input key generated from a combination of attribute identification information and vulnerability category identification information of the system under verification. The non-destructive execution method executes the verification template under the constraint that it does not cause any permanent changes to the system under verification (such as writing new files, placing executable files, or changing user accounts). The verification result reflection mechanism automatically reflects the verification results as verification result correction values ​​to the subsequent layer (effective technology risk index scoring device) via the Application Programming Interface (API). The AI ​​countermeasure text generation means includes a risk band identifier, section classification identifier, output language identifier, and cryptographic hash of input data as essential components, and further includes an independent unique key mechanism that includes at least one of an output target hierarchy identifier or an output format identifier as an additional component, and returns the same countermeasure text for the same combination key. [Effects of the Invention]

[0007] According to the present invention, the two phenomena of idempotency in the generation of verification templates and idempotency in the generation of countermeasure text, which occur systematically in vulnerability verification and countermeasure generation using AI models, can be resolved in a mutually consistent and structural manner not by accumulating individual, independent countermeasures, but by two types of idempotency key mechanisms integrated on a unified platform based on the common principle of unique key design for AI output. This configuration has technical significance in that, unlike existing simple AI query-type verification and countermeasure generation, it simultaneously guarantees the re-executability of generation, auditability, and output branching for each customer. As a result, it becomes possible to guarantee the reproducibility and auditability of AI verification results, the consistency and customer trust of AI countermeasures documents, and the automated coordination of verification, evaluation, and countermeasures, making it possible to put the software into practical use as an AI-based vulnerability verification service. Firstly, the four-axis independent idempotent key mechanism in the AI ​​countermeasure text generation means of the present invention adopts a design that includes all four axes—(i) risk band identifier, (ii) section classification identifier, (iii) output language identifier, and (iv) input data cryptographic hash—as independent dimensions in the cache key component. This four-axis independent dimension design cannot be achieved with general configurations (typical machine learning output caching methods) that cache the AI ​​model output with only a single input hash key. For example, in a single-dimensional cache that hashes only the input data, the same cache entry is returned for multilingual parallel requests with different (iii) output language identifiers, resulting in a structural failure where the output of Japanese and English countermeasure texts for the same vulnerability is returned in the same language. Similarly, in configurations that do not include the risk band identifier (i) as an independent dimension, the urgency expression of the countermeasure text is not integrated among customers to whom different risk bands apply for the same vulnerability category. The present invention solves the design difficulty of integrating these four axes into a single cache mechanism while explicitly maintaining their functional independence. Secondly, each of the means of the present invention is essential for realizing the effects of the present invention in the sense that the absence of such means would result in the following structural problems: (i) If the ownership verification means is absent, non-destructive verification of any object to be verified becomes indistinguishable in legal terms from an intrusion testing campaign that does not involve prior verification of ownership. (ii) If the idempotency key mechanism in the AI ​​template generation means is absent, the verification template for the same object to be verified will differ each time it is executed, making it difficult to preserve evidence of the verification results during audits or litigation (event (α) above). (iii) If the non-destructive execution means is absent, the possibility that the verification template will cause permanent changes on the system to be verified cannot be ruled out, making it structurally impossible to prevent customer damage liability risks in commercial use. (iv) If the verification result reflection means is absent, the verification results will not be automatically linked to the subsequent layer, the effective technology risk index scoring device, and the consistency between verification and evaluation will depend on manual intervention at the operational site and will easily be lost. (v) If the multi-axis idempotency key mechanism in the AI ​​countermeasure text generation means is missing, the countermeasure text for the same vulnerability will vary among customers and requests, leading to a loss of customer trust and a race condition in subsequent automated processing (event (β) above). (vi) If the two-stage independent configuration in which the idempotency key of the AI ​​template generation means and the combination key of the AI ​​countermeasure text generation means are held separately as independent dimensional configurations is lacking, and a configuration using only one of the idempotency keys is adopted, the non-idempotency of one means will cause the input hash of the other means to fluctuate, resulting in a structural failure to prevent a two-stage chain event in which the countermeasure text for the same vulnerability loses consistency (see Example 4). Thirdly, the present invention provides software as an AI-driven vulnerability verification and countermeasure document generation service that, compared to a general configuration without an idempotency key mechanism for AI model output, (a) structurally guarantees the reproducibility of verification templates for the same verification target, thereby facilitating evidence preservation during audits and litigation; (b) structurally guarantees the consistency of countermeasure documents for the same combination key, thereby reducing the workload for responding to customer inquiries caused by variations in countermeasure documents among customers; and (c) automatically links the verification results to the effective technical risk index scoring device as verification result correction values, thereby maintaining consistency between verification and evaluation without manual intervention at the operational site. Furthermore, the means of the present invention are described in separate claims as the respective problems to be solved (unclear ownership, idempotency of verification, permanent damage, lack of evaluation linkage, idempotency of countermeasures) and the functions for those problems (means to prevent them), and do not limit the specific mechanism names. As a result, the effects of the present invention are maintained even with implementation changes using equivalent ownership verification mechanisms or equivalent idempotency key mechanisms. [Brief explanation of the drawing]

[0008] [Figure 1] This is an overall configuration diagram of the apparatus in one embodiment of the present invention. [Figure 2] This is a sequence diagram of DNS text record ownership verification in one embodiment of the present invention. [Figure 3] This is a diagram of the idempotent key structure for AI template generation in one embodiment of the present invention. [Figure 4] This is a flowchart illustrating the verification of non-destructive execution constraints in one embodiment of the present invention. [Figure 5] This is an API linkage diagram for the verification result correction value in one embodiment of the present invention. [Figure 6] This is a diagram illustrating the four-axis idempotent key structure of AI-controlled text generation in one embodiment of the present invention. [Figure 7] This is a feedback loop diagram of the generative review in one embodiment of the present invention. [Figure 8] This is a flowchart illustrating the skill group selection and application process in one embodiment of the present invention. [Figure 9] This is a flowchart illustrating execution suppression by matching a set of known negative examples in one embodiment of the present invention. [Modes for carrying out the invention]

[0009] In this specification, "effective technical risk index" refers to the effective technical threat level calculated by comprehensively considering the public vulnerability score (Common Vulnerability Scoring System, CVSS), the exploit prediction score (EPSS), the degree of agreement with the known exploit catalog, and the quality of the detection evidence. In this specification, "verification result correction value" refers to the correction value that the verification result reflection means of this device adds to the effective technical risk index of the relevant vulnerability finding, and is added to the correction term of the effective technical risk index in subsequent layers. In this specification, "identical input" means that the cryptographic hash values ​​after performing a predetermined normalization process on the input data (such as removing whitespace, normalizing character codes (Unicode normalization), and sorting the argument order) match. Subsequent changes to the internal state of the AI ​​model (such as resetting temperature parameters or updating system prompts) do not affect this determination. In this embodiment, the idempotent key mechanism is implemented as a cache reference mechanism that prioritizes reusing existing outputs if they match the given key and exist in the cache; the same existing output is returned for the same key. When operational changes occur that require generation updates of the AI ​​model, revisions to the normalization process for input data, or additions or modifications to the idempotent key components, a version identifier that identifies the change is newly added as a component of the cache key, thereby adopting a configuration that allows the old and new versions to coexist as separate cache entries. In this case, the condition for determining the allowable difference is that the reuse of the same existing output is permitted only if the key and the version identifier match. This ensures that the traceability of the old version output and the regeneration of the new version output are both possible when the AI ​​model is updated. In this embodiment, "identical key" primarily refers to cases where the cryptographic hash values ​​of the input data after the normalization process match. However, for minor acquired differences (e.g., the inclusion of invisible characters, differences in consecutive spaces, inputs where only the order of argument appearance differs), the system can determine them as "substantially identical" by revising the normalization process to absorb such minor differences or by implementing an equivalence determination mechanism. This substantial identity determination is implemented as an extension of the normalization process that generates identical cache key elements, and this extension is introduced in a way that allows it to coexist with older versions by assigning the version identifier described above. This ensures that regeneration suppression and cache reference priority are maintained even for substantially identical inputs resulting from operational differences in the AI ​​model and inference environment. 1. Embodiment of the ownership verification means (corresponding to claim 2) As one embodiment, the ownership verification means is implemented by verifying, via a standard DNS resolution library, whether the administrator of the system under verification has set an authentication token of a predetermined format specified by the device operator (typically a Universally Unique Identifier (UUID) version 4 format or a 256-bit secure hash algorithm (SHA-256) hash value) in the Domain Name System (DNS) text record pertaining to the system's domain, along with a predetermined prefix. 2. Implementation of idempotency key for AI template generation (corresponding to claim 3) As one embodiment, the idempotent key mechanism in the AI ​​template generation means uses a cache key obtained by hashing (i) the technical stack identification information of the system to be verified (a signature generated from a combination of the operating system, web application framework, programming language runtime version, etc.) and (ii) the vulnerability category identifier to be verified (Common Vulnerability Type Identifier (CWE) number, OWASP Top 10 category, etc.) using a 256-bit secure hash algorithm (SHA-256). If an existing template exists for that cache key, the cache reference mechanism returns the existing template without regenerating it using the AI ​​model. Publicly known AI models (e.g., publicly known large-scale language models or publicly known AI agent frameworks) may be used as template generators. This configuration employs an idempotency guarantee mechanism using cache keys and references, rather than relying on the operation of these AI models themselves. 3. Embodiment of non-destructive execution means (corresponding to claim 4) As one embodiment, the non-destructive execution means employs known open-source vulnerability testing software (e.g., a template-based vulnerability scanner or web crawling tool) as the template execution engine, and implements a constraint in the execution context of the open-source software that only allows testing templates that do not generate any of the following: (a) write HTTP methods (such as POST, PUT, DELETE, etc., which may cause side effects on the system under test), (b) execution of operational system commands on the system under test, or (c) file uploads to the system under test. This constraint is enforced by static analysis of the template (checking HTTP methods within the template) and dynamic monitoring (checking outgoing requests during execution). 4. Embodiment of the means for reflecting the correction value of the verification result (corresponding to claim 5) Here, the verification result correction value in this embodiment is not limited to a value that directly raises or lowers the severity of the vulnerability itself, but is a correction value that reflects the uncertainty of the detected findings, the possibility of false positives, or whether or not the item is verified in the subsequent effective technical risk index scoring device. The sign, absolute value, and interpretation of the correction value are determined based on the division of labor design between the role of this device (independent execution of verification and provision of verification results) and the role of the subsequent scoring device (the risk assessment itself), and the specific numerical values ​​in this embodiment are merely examples of such division of labor design. As one embodiment, the verification result reflection means adds a -1 level correction (in the direction of reducing uncertainty of detected findings) to the effective technical risk index of the relevant vulnerability finding when the verification result "verifies that a vulnerability exists", a -2 level correction (in the direction of reducing false positives) when "verifies that a vulnerability does not exist", and a 0 level correction when "verification was not performed (unconfirmed)". These correction values ​​are automatically notified to the effective technical risk index scoring device (a separate application by the present applicant), which is a higher layer, via an Application Programming Interface (API), and are reflected in the calculation of the effective technical risk index in the said device. 5. Implementation of a 4-axis idempotent key for AI-controlled text generation (corresponding to claim 6) As one embodiment, the multi-axis idempotent key mechanism in the AI-countermeasure text generation means uses the following four axes as independent cache key components. (i) Risk band identifier: Risk band identifier for the vulnerability (e.g., S, A, B, C, or D) (ii) Section identifier: Section identifier of the countermeasure document (e.g., summary, detailed procedure, verification method, side effects) (iii) Output language identifier: Output language identifier (e.g., Japanese, English) (iv) Cryptographic hash of input data: SHA-256 hash value of the input data (vulnerability category, detection result, verification result, associated asset attribute) concatenated in a normalized order. If existing countermeasure texts for these four-axis cache keys exist in the cache, the AI ​​model (such as a well-known large-scale language model) will not regenerate them, and the existing texts will be returned. This ensures the idempotency of countermeasure texts for the same input. The four axes described in (i) to (iv) above are just one example of the core configuration in this embodiment. The multi-axis idempotent key mechanism in the AI ​​countermeasure text generation means can adopt at least four arbitrary identifiers as combination key components from a total of six identifiers, including the four axes described above (risk band identifier, section classification identifier, output language identifier, input data cryptographic hash), as well as the output target hierarchy identifier and output format identifier, depending on the operational requirements of the system. For example, if emphasis is placed on output presentation derivation (multi-language parallel, multi-target hierarchy parallel, multi-output format parallel), a five-axis configuration can be adopted by adding the output target hierarchy identifier or output format identifier to the four axes described above (corresponding to claim 6). Other four-axis combinations depending on operational requirements (e.g., section classification identifier, output language identifier, output target hierarchy identifier, output format identifier) ​​can also be adopted in the same way, and identifiers not adopted in each combination will not be components of the combination key. 6. Implementation of a separate session for AI template generation review (corresponding to claim 7) In one embodiment, the verification template generated by the AI ​​template generation means is input to a separate review means, which evaluates the paths, matchers, and response characteristics to the verification target within the template and returns an acceptance / rejection decision and correction instructions. The review results are fed back to the generation means as input for the next generation, and the generation means regenerates based on this feedback. At this time, the review means is executed in a separate dialogue session from the generation means, and the state of the generation session and the state of the review session are maintained independently of each other. 7. Preliminary structural verification of AI template generation (corresponding to claim 8) As one embodiment, a structural verification is performed in advance on the verification template generated by the AI ​​template generation means. If the structural verification fails, the evaluation by the review means is omitted, and the reason for the failure is fed back into the next generation input. This avoids unnecessary reviews of templates with formal defects while structurally retaining correction instructions for regeneration. 8. Review pass threshold for AI template generation (corresponding to claim 9) In one embodiment, the review means adopts a verification template only if it meets predetermined acceptance criteria based on approval results, reliability, and at least one of a specific route or specific matcher. If the acceptance criteria are not met, the reasons for the judgment and any points raised are fed back into the next generation input. This ensures that only templates with sufficient specificity for verification are adopted, rather than merely receiving formal approval. 9. Report output gate when verification results match (corresponding to claim 10) As one embodiment, the report generation means outputs a vulnerability report only if it is approved by the review means and the execution result by the non-destructive execution means shows a match to the target of verification in the verification template. If either of these conditions is not met, the vulnerability report is not output, and only a customer summary is returned. 10. Integration with publicly known open-source software and publicly known AI models The apparatus of the present invention may employ a large-scale language model known at the time of filing as the AI ​​model used in Sections 2 and 5 above, and may employ open-source software for vulnerability verification known at the time of filing as the template execution engine used in Section 3. This configuration does not make the operation of these known AI models and known open-source software itself a contribution to the invention, but rather configures the means described in Sections 1 to 5 as an architecture that integrates the AI ​​output uniqueness key design as a common principle on top of the known technology. 11. Execution suppression by matching with a set of known negative examples (corresponding to claim 12) As one embodiment, the verification template generated by the AI ​​template generation means is compared with a known set of negative examples. If the verification template matches the known set of negative examples, the non-destructive execution means does not execute the template, and the adoption of the verification template and the output of the vulnerability report are suppressed. This eliminates false positives caused by known generalized patterns in advance. 12. Distinction between the two-stage independent idempotent key mechanism of the present invention and the prior art system The two-stage independent idempotent key mechanism of the present invention (the idempotent key mechanism pair introduced in claims 1(2) and (5) above) is not limited to the specific implementation forms in Sections 2 to 5 above, but is interpreted in terms of structural differences from the following known systems. (a) Distinction from automated intrusion testing campaign systems (Patent Documents 1-8): This system does not require prior verification of ownership, and its contribution is the execution of an intrusion testing campaign against any target for verification. The present invention structurally ensures legal legitimacy through means of ownership verification. (b) Distinction from classical dynamic application security verification systems (Patent Documents 9-15): Since these systems do not employ dynamic template generation by AI models, (α) AI generation idempotency itself does not occur. The present invention solves (α) on the premise of employing an AI model. (c) Distinction from AI-driven security systems (Patent Documents 16-19): Patent Documents 16 and 17 employ an audited agent-based command and control structure and lack ownership verification means, Patent Document 18 does not deal with the idempotency of AI output, and Patent Document 19 lacks an idempotency key mechanism for AI output. The present invention comprehensively solves ownership verification and a two-stage independent idempotency key. (d) Distinction from AI countermeasure text systems (Patent Documents 25-31): These systems disclose countermeasure plan ranking using machine learning exploitability models, remediation package distribution, open-source software code library countermeasure tools, or mitigation prioritization using analytical attack graphs, but (β) lack an idempotent key mechanism for the generation of countermeasure text itself (the four-axis independent combination key described in claim 1(5)). The present invention provides an independent uniqueness key mechanism that solves (β). [Examples]

[0010] [Example 1: DNS text record ownership verification, AI dynamic verification, non-destructive execution] Organization X requested a vulnerability check for its web application, subject.example.com. In this embodiment, the device first issued an authentication token in UUID format to the administrator of Organization X and requested that this token be set in the DNS text record for subject.example.com along with a predetermined prefix. Once the administrator completed the configuration, the device performed DNS resolution and verified that the text record contained the issued token. This verified the ownership relationship of organization X to subject.example.com. Next, the device identified the technology stack of the system under verification (web application language, web server, database management system) and checked for existing templates for a cache key hashed with SHA-256 using the technology stack identification information and the vulnerability category "SQL injection". Since no existing template existed for this cache key, a SQL injection verification template specific to this technology stack was dynamically generated using a publicly known AI model. This template was then saved to the cache. This device executed the generated template non-destructively against subject.example.com using publicly known open-source software for template-based vulnerability testing as the execution engine. Static analysis and dynamic monitoring confirmed that during this execution, the device did not generate any POST, PUT, or DELETE methods, execute operational system commands, or upload files. As the verification results confirmed the existence of an SQL injection vulnerability, this device automatically notified the effective technical risk index scoring device via API that it would add a verification result correction value of -1 level to the effective technical risk index for the related vulnerability finding. [Example 2: AI-controlled text generation - Avoiding regeneration of countermeasure text using a 4-axis idempotency key] Continuing from Example 1, this device performed a process to generate countermeasures against the verified SQL injection vulnerability for organization X. The input was as follows: • Risk band identifier: A (High risk) • Section Classification Identifier: Detailed Instructions • Output language identifier: Japanese • Cryptographic hash of input data: SHA-256 (SQL injection, subject.example.com, web application language version, verified, template hash) Upon checking existing countermeasures documents for the given 4-axis cache key, this device confirmed that a detailed Japanese procedure countermeasure document, previously generated for the same cache key, already existed in the cache. Therefore, this device did not perform regeneration using the AI ​​model and returned the existing countermeasure document stored in the cache to organization X. For another organization Y, if validation is performed on the same technology stack and the same vulnerability category, and the validation results are known, the same cache key will be generated and the same mitigation message will be returned. This structurally guarantees consistency in mitigation messages between organization X and organization Y. Conversely, if organization Y requests a countermeasure document for the same vulnerability with a different output presentation (e.g., output language identifier is English, target audience is management, or output format is JSON), the (iii) axis of that request will differ from that of organization X's request (output language identifier is Japanese, target audience is engineers, output format is Markdown), resulting in the generation of a different cache key. If no existing countermeasure document exists for that cache key, the AI ​​model will generate a new one. This newly generated countermeasure document is stored as a cache entry unique to the value of the (iii) axis, and subsequent requests for the same (iii) axis will return the existing countermeasure document. This structurally guarantees idempotency, ensuring identical output for identical inputs, regardless of whether the output presentation is derived from parallel requests for the same vulnerability (multi-language parallel, multi-target audience parallel, multi-output format parallel). [Example 3: A configuration that employs a publicly known AI agent framework as the AI ​​model] Organization Z chose to adopt a publicly known AI agent framework (open-source software, capable of using large-scale language models as a backend) as its AI agent on its CTEM platform. In this embodiment, the publicly known AI agent framework was adopted as the AI ​​model in Sections 2 and 5, and publicly known open-source software for template-based vulnerability verification was adopted as the template execution engine in Section 3. The operation of the publicly known AI agent framework and template execution engine itself is not covered; the means described in Sections 1 to 5 (DNS text record ownership verification, idempotency key for AI template generation, non-destructive execution constraint, reflection of verification result correction values, and four-axis idempotency key for AI countermeasure text generation) were integrated on top of the publicly known open-source software to implement Organization Z's vulnerability verification and countermeasure generation service. [Example 4: Structural Solution of AI Non-Idempotent Chain Scenario] As a comparative example, in a configuration where the AI ​​template generation means does not have an idempotent key mechanism (a general configuration that directly employs a publicly known AI agent framework), if the AI ​​model generates a verification template T_1 in the first verification and a verification template T_2 (a derivative with a different detection path from T_1) in the second verification for the same target of verification (technology stack A, vulnerability category "SQL injection"), then (i) when the first verification result r_1=T_1 is passed to the AI ​​countermeasure text generation means, the input data hash becomes h_1, (ii) when the second verification result r_2=T_2 is passed, the input data hash becomes h_2 (≠h_1), and (iii) as a result, the countermeasure text for the same vulnerability loses consistency between the first and second verifications. In this embodiment, the device saves T_1 as an entry for the cache key K_template (SHA-256 hash of technology stack A and vulnerability category "SQL injection") in the first iteration, and reuses T_1 in the second iteration because K_template matches. As a result, the verification result r_1=r_2, the input data hash h_1=h_2, and the matching of the countermeasure text are structurally guaranteed. In this embodiment, the input data is configured to include, in addition to the verification result, the template identifier or template hash of the verification template corresponding to the verification result. Therefore, when the same verification template is reused for the same verification target, the template identifier or template hash also matches, and the input data hash h_1=h_2 is structurally guaranteed as the configuration of claim 1. This chain can only be solved in the integrated configuration of the present invention, which bundles the AI ​​template generation means and the AI ​​countermeasure text generation means in two stages independently with an idempotent key, and is a structural solution that cannot be reached with only one of the idempotent keys. [Industrial applicability]

[0011] The present invention has industrial applicability as an AI-driven vulnerability verification, AI-driven countermeasure document generation, and AI-based vulnerability verification service in the field of cybersecurity. The present invention is applicable to any software as a CTEM service or software as an AI-based vulnerability verification service that employs a known large-scale language model and known open-source software for vulnerability verification, and can structurally improve the reproducibility of verification results, the consistency of countermeasure documents, and the automatic linkage from verification to countermeasures in said service. [Explanation of Symbols]

[0012] 1 Possession confirmation method 2. AI template generation means 3. Non-destructive execution methods 4. Means for reflecting verification results 5. AI-friendly text generation method 6. Review Methods 7. Structural Verification Department 8. Skill Group Selection Section 9 Negative example set matching section 10 Vulnerability Report

Claims

1. An apparatus for performing vulnerability verification and generating countermeasure documents using an artificial intelligence (AI) model, comprising: (1) ownership verification means for confirming ownership of the system to be verified; (2) AI template generation means for dynamically generating a verification template for the system to be verified whose ownership has been confirmed, and returning the existing template without regenerating the verification template for the same input; (3) non-destructive execution means for non-destructively executing the verification template on the system to be verified; (4) verification result reflection means for reflecting a verification result correction value in the effective technical risk index of the relevant vulnerability findings based on the execution result; and (5) AI countermeasure document generation means for generating vulnerability countermeasure documents based on the verification result, and controlling the reuse of the existing countermeasure document using a combination key separate from the AI ​​template generation means, wherein at least four of the following are used as a combination key: a vulnerability risk band identifier, a section classification identifier of the countermeasure document, an output language identifier, an output target hierarchy identifier, an output format identifier, and a cryptographic hash of the input data, and the existing countermeasure document is returned for the same combination key. Hereinafter, the AI ​​template generation means outputs a template identifier or template hash corresponding to the generated or reused verification template in association with the execution result, and the AI ​​countermeasure text generation means generates the cryptographic hash of the input data based on the execution result, the template identifier or template hash, and the input data normalized from the associated vulnerability findings, and the idempotence key in the AI ​​template generation means and the combination key in the AI ​​countermeasure text generation means adopt a two-stage independent configuration in which they are held separately as independent dimensional configurations, thereby providing a structural guarantee that the non-idempotency of the verification template does not induce the non-idempotency of the countermeasure text, and an AI vulnerability verification and countermeasure generation device comprising the means (1) to (5) above.

2. The device according to claim 1, wherein the ownership verification means is implemented in such a manner that it verifies, via DNS resolution, whether the administrator of the system to be verified has set an authentication token (Universally Unique Identifier (UUID) format or hash value) of a predetermined format specified by the operator of the device in a Domain Name System (DNS) text record relating to the domain of the system.

3. The apparatus according to claim 1 or claim 2, wherein the idempotent key mechanism in the AI ​​template generation means uses a key generated from a combination of technical stack identification information of the system to be verified (a combination of an operating system, a web application framework, a programming language runtime, etc.) and a vulnerability category identifier to be verified as a cache key, and is implemented as a cache reference mechanism that returns the existing template without performing regeneration by the AI ​​model if an existing template exists for the cache key.

4. The apparatus according to Claim 1, characterized in that the non-destructive execution means executes the verification template under the constraint that it does not perform any of the following: (a) writing a new file to the system under verification, (b) modifying the contents of an existing file on the system under verification, (c) placing an executable file (malware, remote access tool, persistent script, etc.) on the system under verification, (d) creating a new user account or permission setting on the system under verification, and (e) modifying the authentication information of an existing user account on the system under verification.

5. The apparatus according to Claim 1, wherein the verification result reflecting means adds a predetermined verification result correction value to the effective technical risk index of the relevant vulnerability finding, depending on whether the execution result indicates the presence, absence, or unconfirmation of the vulnerability.

6. The apparatus according to Claim 1, wherein the multi-axis idempotent key mechanism in the AI ​​countermeasure text generation means uses a combination key that includes a risk band identifier, a section classification identifier, an output language identifier, and a cryptographic hash of the input data as essential components, and further includes at least one of an output target hierarchy identifier or an output format identifier as an additional component.

7. The apparatus according to Claim 1, further comprising a feedback loop in which a review means separate from the AI ​​template generation means evaluates the verification template generated by the AI ​​template generation means, and the AI ​​template generation means regenerates the template by feeding back the determination result of the review means to the next generation input, wherein the review means is executed in a separate dialogue session from the generation means.

8. The apparatus according to claim 7, wherein, before evaluation by the review means, a structural verification is performed on the verification template generated by the AI ​​template generation means, and if the structural verification fails, the review is omitted and feedback based on the reason for the failure is given to the next generation input.

9. The apparatus according to claim 7, characterized in that the regeneration of the AI ​​template generation means is terminated and the verification template is adopted only when the determination result of the review means satisfies predetermined acceptance conditions based on approval, reliability and specific route or specific matcher.

10. The apparatus according to claim 7, further comprising a report generation means for generating a vulnerability report, wherein the report generation means outputs the vulnerability report only if it is approved by the review means and the execution result by the non-destructive execution means shows a match to the verification target of the verification template.

11. The apparatus according to claim 10, wherein the AI ​​template generation means, the review means, or the report generation means select a group of skills defined by role based on the vulnerability category identifier to be verified, and apply the group of skills to the dialogue input of each of the means.

12. The apparatus according to claim 10, wherein the verification template generated by the AI ​​template generation means is compared with a known set of negative examples, and if the verification template matches the known set of negative examples, the non-destructive execution means does not perform execution, and the adoption of the verification template and the output of the vulnerability report are suppressed.

13. A method for verifying vulnerabilities and generating countermeasures using an artificial intelligence (AI) model, comprising: (1) a step of confirming ownership; (2) a step of dynamically generating a verification template for the system under verification whose ownership has been confirmed, using a cache key generated from a combination of attribute identification information or technology stack identification information of the system under verification and vulnerability category identification information, and if an existing template exists for the cache key, not generating a new one but returning the existing template; (3) a step of non-destructively executing the verification template; and (4) related vulnerabilities based on the execution results. A vulnerability verification and countermeasure generation method using AI, comprising the steps of: (1) reflecting a verification result correction value in the effective technical risk index of the findings; (2) generating a vulnerability countermeasure document based on the verification result, wherein at least four of the following are used as a combination key separate from the cache key of step (2), and an existing countermeasure document is returned for the same combination key, the cache key of step (2) and the combination key of step (5) are kept separately as independent dimensional configurations.

14. A computer program for causing a computer to function as the ownership verification means, the AI ​​template generation means, the non-destructive execution means, the verification result reflection means, and the AI ​​countermeasure text generation means in the apparatus according to claim 1.

15. A non-temporary, computer-readable recording medium that stores the computer program described in Claim 14.

Images