Verification program, verification device, and verification method

The verification program addresses the challenge of ensuring document consistency by generating a graph structure from a metamodel to automate the verification process, reducing manual effort and costs through AI-assisted evaluation and correction of inconsistencies.

JP2026182545APending Publication Date: 2026-11-19DENSO CORP
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2025078861
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2025-05-09
Publication Date
2026-11-19

AI Technical Summary

Technical Problem

Existing document verification technologies struggle to ensure content quality and consistency within documents, leading to increased development costs and delays due to manual reviews and the inability to verify semantic consistency between descriptions.

Method used

A verification program that uses a metamodel to generate a graph structure from documents, allowing for automated verification of document consistency by identifying nodes and edges representing relationships between description blocks, and employing AI for evaluation and improvement suggestions.

Benefits of technology

Facilitates efficient and automated verification of document consistency, reducing manual effort and costs by enabling automated detection and correction of inconsistencies, thereby improving the quality of document content.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2026182545000001_ABST
    Figure 2026182545000001_ABST
Patent Text Reader

Abstract

The consistency of the document's description is appropriately verified. [Solution] The verification program comprises an extraction unit, an identification unit, and a generation unit. The extraction unit extracts multiple descriptive blocks from the document. The identification unit identifies the nodes of the metamodel that store each of the descriptive blocks extracted by the extraction unit. The generation unit generates a graph structure of the document based on the metamodel and the identification results from the identification unit. The metamodel is a model for generating a graph structure from a document, and has multiple nodes and edges that show the relationships between the nodes. The graph structure is a collection of descriptive blocks extracted by the extraction unit stored in the nodes of the metamodel.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to a verification program and the like for evaluating documents.

Background Art

[0002] In product development (particularly software development), low quality of specifications may lead to numerous inquiries and rework in subsequent processes, which may cause an increase in development costs and delays in the development schedule. For this reason, it is required to discover and correct problems in specifications at an early stage.

[0003] Here, the quality of documents such as specifications includes expression quality and content quality. The expression quality is a quality that allows even a person without specialized knowledge to determine the presence or absence of problems, and relates to, for example, typographical errors, omitted characters, notation inconsistencies, entry omissions, and the like. With the recent development of natural language processing technology, software for detecting and correcting this type of problem with high accuracy has been developed. On the other hand, the content quality is a quality related to semantics, and sufficient verification cannot be performed by superficial or formal analysis, so it has conventionally been difficult to verify content quality by a computer. For this reason, it was necessary to ensure the content quality of specifications through manual reviews, which increased the man-hours required for reviews as the development scale increased, and made it difficult to homogenize the content quality of specifications.

[0004] In contrast, Patent Document 1 discloses a technique for detecting sentences that are grammatically correct but have erroneous content. In this technique, independent words included in an input sentence are extracted, and a word group similar to the extracted independent words is further extracted from a proofreading database. Then, the content of the input sentence is verified by collating the independent words with the word group similar to the independent words.

[0005] Furthermore, Patent Document 2 discloses a technique for facilitating the verification of a document after it has been modified, when a document containing multiple blocks has been modified. This technique detects blocks whose implication relationships with other blocks have changed due to the modification and presents them to the user. This allows the user to easily identify areas in the modified document that may require further modification. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2022-169011 [Patent Document 2] Japanese Patent Publication No. 2024-46629 [Overview of the Initiative] [Problems that the invention aims to solve]

[0007] However, while the technology described in Patent Document 1 can verify the validity of each individual sentence, it cannot verify the consistency between descriptions within a document. Furthermore, the technology described in Patent Document 2 is intended for verifying revisions to a sentence, and it is difficult to verify the consistency between descriptions within a document.

[0008] In one aspect of this disclosure, it is desirable to suitably verify the consistency of the document's description. [Means for solving the problem]

[0009] One aspect of this disclosure is a verification program that operates a computer as a verification device (1) for verifying the contents of documents (4A, 4B), and operates the computer as an extraction unit (S110), an identification unit (S110), and a generation unit (S120). The extraction unit extracts a plurality of description blocks (41) from the document. The identification unit identifies a node (20) in a metamodel (2) that stores each of the description blocks extracted by the extraction unit. The generation unit generates a graph structure (5A, 5B) of the document based on the metamodel and the identification results in the identification unit. The metamodel is a model for generating a graph structure from a document, and has a plurality of nodes and edges (21A~21D) that show the relationships between the nodes. The graph structure is a structure in which each of the description blocks extracted by the extraction unit is stored in one of the nodes in the metamodel.

[0010] According to the above configuration, the relationships between nodes can be understood based on the graph structure, making it easier to verify the consistency of description blocks stored in multiple related nodes. Therefore, the consistency of the document description can be effectively verified. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram of the verification device. [Figure 2] This is an example of a metamodel. [Figure 3] This is a flowchart of the verification process. [Figure 4] Figures 4(a) and 4(b) are explanatory diagrams showing the structure of the document format, and an example of a markup language used to describe the document format. [Figure 5] Figures 5(a) and 5(b) illustrate the process of extracting items, descriptive blocks belonging to those items (in other words, Keys and Values), and parent-child relationships between descriptive blocks from a document under consideration. [Figure 6] Figures 6(a) and 6(b) illustrate the process of standardizing the names of keys extracted from the documents under consideration. [Figure 7]Figures 7(a) and 7(b) are explanatory diagrams of a process of storing extracted description blocks in nodes of a metamodel to generate a graph structure. [Figure 8] This is an example of evaluation items displayed on the display device of the verification apparatus. [Figure 9] This is an explanatory diagram showing the graph structure of a document before change and the graph structure of the document after change. [Figure 10] This is an example of evaluation based on related information. [Figure 11] This is an example of evaluation based on other document evaluation results. [Mode for Carrying Out the Invention]

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. [1. Overview] The verification apparatus 1 of the present embodiment is configured to perform document verification (see FIG. 1). The document may be, for example, a requirement specification when developing a product, and more specifically, may be a software requirement specification. Of course, the present invention is not limited to this, and the document may be, for example, a hardware requirement specification, or the verification apparatus 1 may verify documents other than requirement specifications.

[0013] The verification apparatus 1 generates a graph structure from a document using a metamodel 2 (see FIG. 2), and verifies the quality of the document using the graph structure. The verification apparatus 1 is configured as a PC as an example, and includes a CPU 10, a memory 11, a storage unit 12, a user I / F 13, a display device 14, and a communication unit 15.

[0014] The memory 11 is a semiconductor memory configured as a ROM, a RAM, or the like, and the CPU 10 executes programs stored in the ROM and programs loaded into the RAM. The storage unit 12 is configured by, for example, an HDD or the like, and stores various data such as programs executed by the CPU 10 and data indicating a metamodel 2 to be described later.

[0015] The user I / F 13 includes a keyboard, a mouse, and the like, and outputs a signal corresponding to an operation received from a user to the CPU 10. The display device 14 displays an image generated by the CPU 10.

[0016] The communication unit 15 accesses a network such as the Internet in response to an instruction from the CPU 10, and communicates with other devices. The communication unit 15 is also capable of communicating with a past case DB 16 and a related information DB 17 via the network.

[0017] The past case DB 16 is a database that stores evaluation results obtained according to predetermined evaluation items (details will be described later) for description blocks (hereinafter referred to as other document description blocks) in other documents that have been performed in the past, which will be described later. The evaluation results may be evaluation results obtained by verification processing to be described later.

[0018] The related information DB 17 is a database that stores information related to a document to be subjected to verification processing to be described later (hereinafter referred to as related information). More specifically, the related information may be, for example, highly specialized information that is difficult to acquire from the Internet (for example, information included in a technical term dictionary) or information referred to as so-called tacit knowledge.

[0019] The past case DB 16 and the related information DB 17 may be provided in the storage unit 12. An OS and an application that causes the PC to operate as the verification device 1 (hereinafter referred to as a verification program) are installed in the PC serving as the verification device 1. That is, the verification program is stored in the storage unit 12, and the CPU 10 causes the verification program loaded from the storage unit 12 to the memory 11 to run on the OS, whereby the PC operates as the verification device 1. Further, as the verification program operates, a verification method corresponding to the verification program is executed.

[0020] Furthermore, the verification device 1 may be configured not as a PC, but as a dedicated device for verifying documents. In this case, all or part of the functions of the verification device 1 may be realized by hardware composed of dedicated electronic circuits instead of a CPU operated by a verification program. In this case, the electronic circuits may be digital circuits including a large number of logic circuits, analog circuits, or a combination thereof.

[0021] [2. Metamodel] Verification device 1 uses metamodel 2 to create a graph structure of the document (see Figure 2). Metamodel 2 is a model of the graph structure generated by verification device 1 and may have multiple nodes 20 and multiple edges 21A to 21D. Nodes 20 correspond to items in the document and store the content of the description belonging to the corresponding item (hereinafter referred to as a description block). In addition, if there is no description block belonging to the item corresponding to node 20, data indicating that there is no description block may be stored. Edges 21A to 21D connect multiple nodes 20, meaning that the connected multiple nodes 20 have a predetermined relationship with each other.

[0022] Figure 2 shows an example of Metamodel 2 for graph-structuring software specifications. As an example, Metamodel 2 has 20 nodes of the following types: "Purpose", "Higher-Level Requirements", "Requirements", "Reason", "Interface Requirements", "Technical Requirements", "Functional Requirements", "External Environment", "Interface", "Quality Requirements", "Constraints", "Triggers", "Data", "Input / Output Data", "Input / Output Triggers", "Behavior", "Threshold", and "Initial Value". The name of each node 20 corresponds to the items in the document formats 3A and 3B (see Figure 4) described later. Note that in Metamodel 2, there are no multiple nodes 20 with the same name (type). However, the graph structure generated from Metamodel 2 may have multiple nodes 20 with the same name (type).

[0023] Furthermore, the edge 21A, shown by a line without an arrow connecting "Purpose" and "Higher-Level Requirements," means that "Higher-Level Requirements" are defined as a lower level of "Purpose." Similarly, the edge 21A connecting "Higher-Level Requirements" and "Requirements" also means that "Requirements" are defined as a lower level of "Higher-Level Requirements."

[0024] Similarly, Edge 21A, which connects "Requirements" with "Interface Requirements," "Technical Requirements," and "Functional Requirements," defines "Interface Requirements," "Technical Requirements," and "Functional Requirements" as sub-hierarchies of "Requirements." Edge 21A defines that "Interface" is a sub-hierarchy of "Interface Requirements," "Triggers" and "Data" are sub-hierarchies of "Interface," and "Thresholds" and "Initial Values" are sub-hierarchies of "Data." Furthermore, Edge 21A indicates that "Quality Requirements" and "Constraints" are sub-hierarchies of "Technical Requirements," and "Input / Output Data," "Input / Output Triggers," and "Behavior" are sub-hierarchies of "Functional Requirements."

[0025] Furthermore, edge 21B, shown by a line without an arrow connecting "request" and "reason," indicates that "reason" complements "request." Similarly, edge 21B indicates that "external environment" complements "interface requirement."

[0026] Furthermore, edge 21C, indicated by lines connecting both ends to "functional requirements," shows that if multiple "functional requirements" exist, multiple "functional requirements" can be connected at edge 21C. "Functional requirements" connected at edge 21C have a parent-child relationship, or in other words, a relationship between a higher and lower hierarchy. The same applies to edge 21C connected to "technical requirements."

[0027] Furthermore, edge 21D, which is an arrow extending from "input / output data" to "data," means that "input / output data" refers to the description block in "data." In other words, it indicates which description block in "data" the description block stored in "input / output data" refers to. The same applies to edge 21D, which is an arrow extending from "input / output trigger" to "trigger."

[0028] [3. Overview of the Verification Process] Verification device 1 verifies documents using the verification process shown in Figure 3. The verification process is executed by the CPU 10 of verification device 1, which is operated by a verification program. The document to be verified (hereinafter also referred to as the document to be verified) is a file that can be processed by a computer, and may be, for example, a Word file or a PDF file. In the verification process, first, a graph structure of the document to be verified is generated based on the metamodel 2, and then evaluation is performed focusing on the nodes 20 in the graph structure.

[0029] Furthermore, in the validation process, various processes are performed using AI (Artificial Intelligence), as will be described later. AI may be, for example, a generative AI based on a probabilistic model, and generative AI may be, for example, a language model trained to understand and generate natural language. In addition, the language model may be, for example, an LLM (Large Language Model) or a VLM (Visual Language Model). An LLM may be a sequence decoder or encoder based on a deep neural network, such as ChatGPT or BERT (Bidirectional Encoder Representation from Transformers). Besides these, AI may be a generative model that is not a sequence decoder or encoder type, such as a diffusion model, or it may be configured to realize each process of the validation process by having a dedicated learning model.

[0030] [4. Details of the verification process] Next, the details of the verification process will be explained using the flowchart in Figure 3. The verification device 1 starts the document verification process according to the operation received via the user interface 13. The verification device 1 may also select the document to be verified according to the operation received via the user interface 13. The document to be verified may be stored in the storage unit 12 or obtained from the network via the communication unit 15. In addition, one document may be the document to be verified, or multiple documents may be the document to be verified.

[0031] In S100, the verification device 1 converts the data to be verified into a data format that can be processed by language processing (e.g., Markdown, HTML, JSON, XML, or text data), and then proceeds to S105. The verification device 1 may, for example, use a Python library to convert the data format.

[0032] In S105, the verification device 1 selects the document format 3A, 3B (see Figure 4) that matches or is most similar to the document to be verified, and proceeds to S110. The document formats 3A, 3B may be stored in the storage unit 12 or obtained from the network via the communication unit 15. Specifically, for example, the verification device 1 may input the document to be verified and the multiple document formats 3A, 3B to the AI ​​and have the AI ​​select the document format 3A, 3B that matches or is most similar to the document to be verified. Alternatively, the verification device 1 may perform a similar selection by pattern matching, or select the document formats 3A, 3B according to an operation received via the user interface 13.

[0033] Furthermore, the verification device 1 may present the selected document formats 3A and 3B to the user via the display device 14, and may also receive an indication from the user via the user interface 13 whether or not they agree to the verification based on the document formats 3A and 3B. If the verification device 1 has confirmed the user's agreement, it may proceed with the verification based on the document formats 3A and 3B.

[0034] Here, document formats 3A and 3B are data that indicates the structure of a document and include a markup language (e.g., Markdown or XML) that describes the document structure. Each document format 3A and 3B has at least one item 30 and at least one description block 31 belonging to each item 30. Furthermore, each document format 3A and 3B describes the structure of each description block 31, including the names of elements that may be included in each description block 31 (hereinafter referred to as Key 32) and the area (hereinafter referred to as Value 33) that describes the content corresponding to Key 32. In addition, each document format 3A and 3B defines the relationships between multiple description blocks 31 belonging to item 30, for example, parent-child relationships 34. Furthermore, item 30 corresponds to one of the nodes 20 in the metamodel 2. In addition, each document format 3A and 3B defines the chapter, section, or subsection number (hereinafter referred to as location information 35) to which the description block 31 belongs.

[0035] For example, the document format 3A shown in Figures 4(a) and 4(b) has one item 30 called "4-1. Requirements Specification". Furthermore, three description blocks 31 are provided within this item 30, each containing a Key 32 called "Requirement ID", "Description", and "Rationale", and a Value 33 corresponding to that Key 32. Additionally, "4-1-1" is defined as the location information 35 for these description blocks 31. This item 30 corresponds to a node 20 called "Functional Requirements".

[0036] As an example, the document format 3B in Figures 4(c) and 4(d) has one item 30 called "D-1. Requirements Specification". Furthermore, three description blocks 31 are provided within this item 30, and each description block 31 has a Key 32 called "Requirement ID", "Basic Requirements", and "Reason", and a Value 33 corresponding to the Key 32. Additionally, "D-1-1" is defined as the location information 35 for these description blocks 31. This item 30 also corresponds to a node 20 called "Functional Requirements".

[0037] Furthermore, in document formats 3A and 3B, Value 33 contains no description. Also, in document formats 3A and 3B, the description block 31 located at the top of the diagram corresponds to the parent, and the remaining description blocks 31 correspond to the children.

[0038] Furthermore, multiple metamodels 2 may be provided, and the verification device 1 may perform the processing from S110 onward using a metamodel 2 corresponding to the selected document formats 3A and 3B. Of course, this is not limited to this, and the verification device 1 may, for example, perform the processing from S110 onward using a metamodel 2 selected by the user from among multiple metamodels 2.

[0039] In S110, the verification device 1 extracts items 40 and description blocks 41 belonging to each item 40 from the documents to be verified 4A and 4B based on the document formats 3A and 3B selected in S105 (see Figure 5). At this time, the Key 42 and Value 43 of the description block 41 and the parent-child relationships 44 between the description blocks 41 in each item 40 are extracted based on the document formats 3A and 3B. Furthermore, location information 45 of each description block 41 may also be extracted based on the document formats 3A and 3B. Then, the verification device 1 identifies the node 20 of the metamodel 2 corresponding to the extracted item 40 based on the document formats 3A and 3B, and identifies the node 20 as the storage location for the description block 41 belonging to the item 40.

[0040] As an example, in the verification target 4A in Figure 5(a), "4-1. Requirements Specification" is extracted as item 40, and the "Functional Requirements" of Metamodel 2 are identified as node 20 corresponding to "4-1. Requirements Specification". In addition, "Requirement ID", "Description", and "Rationale" are extracted as Key 42, and the descriptions corresponding to each of these are extracted as Value 43. Then, the combination of each of "Requirement ID", "Description", and "Rationale" and the corresponding Value 43 is identified as a description block 41, and the "Functional Requirements" are identified as the storage location for each description block 41.

[0041] Furthermore, in the verification target 4A, the description block 41 whose "Requirement ID" Value 43 is "SRS_WXICON_3110100" is the parent, and the description blocks 41 whose "Requirement ID" Value 43 is "SRS_WXICON_3110101" and "SRS_WXICON_3110102" are the children. The parent-child relationships of these description blocks 41 are also extracted. In addition, "4-1-1" is extracted as the location information 45 of these description blocks 41, and the same node 20 as each of these description blocks 41 may be the storage location of said location information 45.

[0042] As an example, in the verification target 4B in Figure 5(b), "D-1. Requirements Specification" is extracted as item 40, and the "Functional Requirements" of Metamodel 2 are identified as node 20 corresponding to "D-1. Requirements Specification". In addition, "Requirement ID", "Basic Requirements", and "Reason" are extracted as Key 42, and the descriptions corresponding to each of these are extracted as Value 43. Then, the combination of each of "Requirement ID", "Basic Requirements", and "Reason" and the corresponding Value 43 is identified as a description block 41, and the "Functional Requirements" are identified as the storage location for each description block 41.

[0043] Furthermore, in the verification target 4B, the description block 41 with a "request ID" Value 43 of "1-1" is the parent, and the description blocks 41 with "request ID" Value 43 of "1-1-1" and "1-1-2" are the children. The parent-child relationships of these description blocks 41 are also extracted. In addition, "D-1-1" is extracted as the location information 45 of these description blocks 41, and the same node 20 as each of these description blocks 41 may be the storage location of the location information 45.

[0044] The verification device 1 may input document formats 3A and 3B and the verification targets 4A and 4B into the AI, and have the AI ​​extract this information. Alternatively, the verification device 1 may extract this information by, for example, pattern matching or sequence labeling.

[0045] In the following step S115, the verification device 1 modifies the names of Key42 extracted in S110 in order to standardize the names of Key42, and then proceeds to S120 (see Figure 6). The verification device 1 may perform the modification based on dictionary data or ontology data stored in the memory unit 12, or it may perform the modification by accessing similar data stored on an external server via the network.

[0046] In Figure 6(a), as an example, the names of Key42, "Requirement ID," "Description," and "Rationale," have been changed to "ID," "Description," and "Reason," respectively. Similarly, in Figure 6(b), as an example, the names of Key42, "Request ID," "Basic Request," and "Reason," have been changed to "ID," "Description," and "Reason," respectively.

[0047] In S120, the verification device 1 stores the description block 41 extracted in S110 into node 20 of the metamodel 2, which was identified as the storage destination in S110. In other words, it generates data that associates the description block 41 with node 20, which is the storage destination, thereby generating graph structures 5A and 5B of the verification targets 4A and 4B (see Figure 7). At this time, the verification device 1 may display the graph structures 5A and 5B on the display device 14.

[0048] At this time, the parent-child relationships between the nodes 20 that store the description blocks 41 are determined according to the parent-child relationships of the description blocks 41, and the edges 21C of these nodes 20 are set based on these parent-child relationships.

[0049] As an example, in Figures 7(a) and 7(b), description blocks 41 with different Value 43s for the Key 42 "ID" in the verification targets 4A and 4B are each stored in multiple nodes 20 called "Function Requests". Edges 21C indicating the parent-child relationships 44 of these description blocks 41 are set between these nodes 20. Of course, this is not the only way; these description blocks 41 in each of the verification targets 4A and 4B may be stored in a single "Function Request", in which case the parent-child relationships of these description blocks 41 may be stored in the node 20.

[0050] In the subsequent S125, the verification device 1 displays multiple evaluation items 6A to 6C on the display device 14 and accepts the user's selection of evaluation items 6A to 6C to be executed via the user I / F 13 (see Figure 8). These evaluation items 6A to 6C are pre-prepared and may be stored in the memory unit 12, for example, or obtained from an external source via a network. Each evaluation item 6A to 6C corresponds to one of the nodes 20 in the graph structures 5A and 5B, and indicates what kind of evaluation to perform on the description block 41 stored in the corresponding node 20 (hereafter referred to as the evaluation content).

[0051] Note that a node 20 does not necessarily have to be associated with each evaluation item. In this case, the verification device 1 may, for example, use AI to extract the node 20 corresponding to the evaluation item based on the graph structures 5A and 5B and the evaluation content of the evaluation item to be performed.

[0052] Furthermore, the verification device 1 may execute all or some of the executable evaluation items in a predetermined order. Each evaluation item has a predetermined priority, and if an evaluation item with a relatively low priority is selected, the verification device 1 may present an evaluation item with a higher priority on the display device 14. Then, the verification device 1 may execute the presented evaluation item in response to an operation received via the user I / F 13, and then execute the selected evaluation item.

[0053] In the following step S130, the verification device 1 extracts the description block 41 stored in node 20 corresponding to the evaluation item to be performed from graph structures 5A and 5B, and then proceeds to S135.

[0054] In S135, the verification device 1 accesses the related information DB 17 via the communication unit 15 and obtains related information to be verified from the related information DB 17. Specifically, the verification device 1 may use AI to extract information similar to or related to the description block 41 to be evaluated from the related information DB 17.

[0055] Furthermore, in S140, which is executed after S125, the verification device 1 accesses the past case DB 16 via the communication unit 15. The verification device 1 then obtains evaluation results from the past case DB 16 for other documents that were previously evaluated according to the same or similar evaluation items as those being executed this time (hereinafter referred to as other document evaluation results). The other document evaluation results include the evaluation items, the description blocks that were the subject of evaluation according to the evaluation items in the other documents, and the evaluation results for said description blocks.

[0056] Furthermore, if multiple evaluation results for other documents are obtained, the verification device 1 may use AI to identify the evaluation item that is most similar to the description block being evaluated among the description blocks included in these other document evaluation results. The other document evaluation results containing the identified description block may then be used in subsequent processing. Also, if multiple evaluation results for other documents are obtained, the verification device 1 may use AI to summarize the description blocks and evaluation results included in these other document evaluation results, or expand or modify these summaries, as the other document evaluation results to be used in subsequent processing.

[0057] Then, in S145, which follows the execution of S135 and S140, the verification device 1 has the AI ​​execute each evaluation item and evaluate the description block 41 that is the target of evaluation for that item. If the verification device 1 obtains an evaluation result indicating that there is a problem with the target of evaluation, it may have the AI ​​generate a solution to that problem. The verification device 1 then presents the evaluation result and the solution via, for example, the display device 14.

[0058] Specifically, the verification device 1 may input the evaluation items selected in S125, the description block 41 extracted in S130, the related information acquired in S135, and the evaluation results of other documents acquired in S140 to the AI. The verification device 1 may then have the AI ​​evaluate the description block 41 according to the evaluation items based on the related information and the evaluation results of other documents, and may also have the AI ​​generate improvement suggestions. As an example, if an LLM (e.g., ChatGPT) is used as the AI, the evaluation items, description block, and evaluation result from the evaluation results of other documents may be input as a Few Shot prompt to train the AI ​​on the evaluation method. After that, the evaluation may be performed by inputting the evaluation items selected in S125, the description block 41 extracted in S130, and the related information acquired in S135.

[0059] In addition, the verification device 1 may, for example, input evaluation items and description blocks 41 to the AI, as well as input either related information or evaluation results of other documents to the AI. The verification device 1 may then have the AI ​​evaluate the description blocks 41 according to the evaluation items based on either the related information or evaluation results of other documents, and further generate improvement suggestions. Alternatively, the verification device 1 may, for example, input evaluation items and description blocks 41 to the AI, have the AI ​​evaluate the description blocks 41 according to the evaluation items, and further generate improvement suggestions.

[0060] Furthermore, as mentioned above, if the description block 41 is evaluated according to the evaluation criteria by an AI with a dedicated learning model instead of an AI, the learning model may be trained using the evaluation results of other documents used in the evaluation.

[0061] In the following S150, the verification device 1 receives input of feedback (in other words, judgment results) regarding the validity of the evaluation results for each evaluation item via the user I / F 13. If the verification device 1 receives positive feedback regarding the validity, it accesses the past case DB 16 via the communication unit 15. The verification device 1 also saves the evaluation item for which the feedback was input, the description block 41 that is the target of evaluation for that evaluation item, and the evaluation result in the past case DB 16 (S155). After that, the verification device 1 terminates the verification process.

[0062] [5. Specific examples of evaluation] Next, we will explain specific examples of evaluations performed in steps S125 to S145 of the verification process. Figure 8 shows evaluation items 6A to 6C displayed on the display device 14.

[0063] The evaluation content for evaluation item 6A is "confirmation of consistency of requirement specifications between functions," and evaluation item 6A corresponds to node 20, which is "functional requirements." Therefore, if evaluation item 6A is selected in S125, in S130, the description block 41 stored in "functional requirements" is extracted from the graph structure as the target for evaluation.

[0064] Furthermore, the evaluation content for evaluation item 6B is "confirmation of consistency between input and output data between functions," and evaluation item 6B corresponds to node 20 labeled "Functional Requirements" and node 20 labeled "Input / Output Data." Therefore, if evaluation item 6B is selected in S125, in S130, the description blocks 41 stored in "Functional Requirements" and "Input / Output Data" are extracted from the graph structure as targets for evaluation.

[0065] Furthermore, the evaluation content of evaluation item 6C (in other words, the evaluation item at the time of change) is "confirmation of the impact of document changes" to evaluate the impact of changes to the document, and evaluation item 6C corresponds to node 20 which stores the description block 41 in which the change occurred. In other words, evaluation item 6C is used in a verification process that verifies the document before the change and the document that has been changed (hereafter referred to as the modified document).

[0066] Therefore, if evaluation item 6C is selected in S125, S125 detects the differences between the pre-modification document and the modified document that are being verified. The differences may also be detected based on the graph structure generated from the pre-modification document and the graph structure generated from the modified document. Evaluation item 6C corresponds to the node 20 that stores the description block 41 where the differences occurred, and in S130, the description block 41 where the differences occurred is extracted as the evaluation target from the graph structure of the pre-modification document and the graph structure of the modified document.

[0067] Furthermore, in S125, after the differences are detected, other nodes 20 that have a predetermined relationship (for example, connected by a predetermined edge) with the node 20 storing the description block 41 with the differences may be detected based on the graph structure. Also, in S130, in addition to the description block 41 where the differences occurred, description blocks 41 stored in nodes 20 that have a predetermined relationship with the graph structure of the document before modification and the graph structure of the document after modification may be extracted as targets for evaluation.

[0068] Subsequently, in S135-S145, the evaluation of the description block 41 being evaluated is performed in the same manner. In addition, in S145, the evaluation of the description block 41 of the modified document being evaluated may also be performed.

[0069] In Figure 9, as an example, there is a difference in the description block 41 stored in "Functional Requirements," and it is shown that the description block 41 is extracted from the graph structure 5C of the document before the change and the graph structure 5D of the document after the change.

[0070] Furthermore, in a verification process that examines both the original and modified documents, the graph structure of the modified document may be generated based on the differences between the original and modified documents and the graph structure of the original document.

[0071] In other words, in the verification process S100 to S120, a graph structure of the document before modification is generated, and if evaluation item 6C is selected in S125, then in S125, the differences between the document before modification and the document after modification are detected, along with a description block 41 containing the differences and location information 45 of the description block 41. The description block 41 and location information 45 can be detected from the document before modification, the graph structure based on the document, or the document after modification.

[0072] Then, in S125, the graph structure of the modified document is generated by updating the graph structure of the original document based on the detected differences, the description block 41, and the location information 45. Specifically, a node 20 containing the detected location information 45 is identified from the graph structure of the original document. Subsequently, for example, the description block 41 stored in the node 20 may be replaced with the description block 41 in the modified document that contains the differences. Alternatively, for example, only the description relating to the differences in the description block 41 stored in the node 20 may be replaced with the description relating to the differences in the modified document.

[0073] Furthermore, the related information 7 in Figure 10 is, as an example, extracted by the verification device 1 in S135 using AI from the related information DB 17 based on the content of the description block 41A. Then, in S145, the verification device 1 uses AI to evaluate the description block 41A, which is the target of evaluation, based on the related information 7, and as an example, evaluation result 8A is obtained.

[0074] In other words, description block 41A states that the determination of whether the rain is heavy or light is based on a threshold of 1 mm of rainfall. In contrast, related information 7 states that the threshold is 10 mm, and based on this, evaluation result 8A indicates that the threshold for determining whether the rain is heavy or light is 10 mm. By performing the evaluation based on related information 7, the contents of description block 41A can be evaluated in more detail.

[0075] Furthermore, node 20 of "Functional Requirements" in Figure 11 corresponds to node 20 of evaluation item 3A (Figure 8) selected in S125, and the other document evaluation result 9 was obtained in S140 from past case DB16 based on evaluation item 3A. Then, in S145, the verification device 1 uses AI to evaluate the description block 41 included in node 20 of "Functional Requirements," and as an example, evaluation result 8B is obtained.

[0076] In evaluation result 8B, the Value 43 of Key 42, "ID," in the node 20 is shown as the subject of evaluation. Furthermore, in evaluation result 8B, the Value 43 of Key 42, "Description," in the description block 41 of these nodes 20 is shown as the part of the description block 41 related to the evaluation result, along with the evaluation result itself.

[0077] [6. Effects] (1) According to the above embodiment, graph structures 5A and 5B are generated during the verification process, and graph structures 5A and 5B are displayed on the display device 14. As a result, the relationships between nodes 20 can be easily understood, and the consistency of the description blocks 41 stored in multiple related nodes 20 can be easily verified. Therefore, the consistency of the descriptions of the verification targets 4A and 4B can be suitably verified.

[0078] Furthermore, the graph structures 5A and 5B make it easy to identify the nodes 20 corresponding to the purpose of verifying the verification targets 4A and 4B. Therefore, even when the volume of the verification targets 4A and 4B is large, the descriptive blocks related to the purpose of verification can be easily identified, reducing the cost of the verification work.

[0079] (2) In addition, in step S145 of the verification process, the description block 41 stored in either node 20 of the graph structure 5A or 5B is evaluated. Therefore, the parts to be evaluated in the verification targets 4A and 4B can be easily identified, and as a result, the cost of the verification work can be reduced.

[0080] (3) Furthermore, if problems are detected through evaluation based on evaluation items 6A to 6C, improvement plans for those problems will be proposed. This reduces the cost required to correct the problems in the verification targets 4A and 4B.

[0081] (4) Furthermore, the verification process is performed according to evaluation items 6A to 6C selected by the user. This allows for the selection of descriptive blocks to be evaluated. (5) Furthermore, if an evaluation item with a lower priority is selected, an evaluation item with a higher priority may be presented, and the evaluation based on that item may be carried out first. Therefore, when it is necessary to carry out each evaluation item in a predetermined order, by setting a higher priority for these evaluation items in order from the evaluation item that should be carried out first, these evaluation items can be carried out in the appropriate order. As a result, the quality of the evaluation is improved.

[0082] (6) In addition, during the verification process, the description blocks 41 stored in the node 20 corresponding to the evaluation items 6A to 6C selected by the user are to be evaluated. Therefore, the evaluation targets can be determined automatically, and the workload of the verification process can be reduced.

[0083] (7) In addition, in the evaluation according to evaluation item 6C, the description block 41, which includes the differences between the document before the change and the document after the change, is examined. Therefore, it is easy to verify whether or not the change had an impact.

[0084] (8) Furthermore, in the verification process, the description blocks 41 stored in the node 20 that has a predetermined relationship with the node 20 that stores the description blocks 41 containing the differences are extracted as targets for evaluation. Therefore, description blocks 41 that may be affected by the changes can also be appropriately evaluated.

[0085] (9) Furthermore, the nodes 20 of the graph structures 5A and 5B store location information 45 of the description blocks 41 stored in those nodes 20. This makes it easy to determine which part of the subject 4A and 4B each node 20 in the graph structures 5A and 5B corresponds to based on the location information 45 of the graph structures 5A and 5B. In addition, based on the location information 45, it becomes possible to set up links between the description blocks 41 in the graph structures 5A and 5B and the description blocks 41 in the subject 4A and 4B.

[0086] (10) In a verification process that verifies both the original document and the modified document, the graph structure of the modified document may be generated from the graph structure of the original document using the location information 45 of the description block 41 that contains the differences between the original document and the modified document. This reduces the processing load when generating the graph structure of the modified document.

[0087] (11) Furthermore, in the verification process S105, when graph structuring is performed, the description block 41 is extracted from the verification targets 4A and 4B based on the document format 3A and 3B that is most similar to the verification targets 4A and 4B. Therefore, the description block 41 can be suitably extracted from various verification targets 4A and 4B.

[0088] (12) Furthermore, document formats 3A and 3B, selected as the most similar to the documents 4A and 4B to be verified, are presented to the user. If the user agrees to use the selected document formats 3A and 3B, verification is performed based on those document formats 3A and 3B. This makes it possible to perform verification using the appropriate document formats 3A and 3B.

[0089] (13) In addition, in the verification process S120, when graph structuring is performed, a node 20 is determined to be the storage location for the description blocks 41 extracted from the verification targets 4A and 4B, based on the document formats 3A and 3B. Therefore, when graph structuring is performed, the description blocks 41 can be stored in the appropriate node 20.

[0090] (14) In addition, in the verification process S115, the names of Key 42 included in the description block 41 extracted from the verification targets 4A and 4B are standardized. This allows for proper evaluation of the description block.

[0091] (15) In addition, during the verification process, an evaluation is performed that takes into account the evaluation results of other documents 9 obtained from past case DB16 in S140. This improves the quality of the evaluation. (16) Furthermore, in S150 of the verification process, feedback on the evaluation results can be entered, and if positive feedback on the evaluation results is entered, the evaluation results are saved in the past case DB16. This makes it possible to improve the quality of evaluations based on other document evaluation results9.

[0092] (17) In addition, the verification process takes into account the related information 7 obtained from the related information DB17 in S135. Therefore, the quality of the evaluation is improved. [7. Other Embodiments] (1) In the verification process in the above embodiment, the evaluation of the verification targets 4A and 4B is performed automatically based on the graph structures 5A and 5B generated from the verification targets 4A and 4B. However, the user may also perform the evaluation of the verification targets 4A and 4B based on the graph structures 5A and 5B generated in the verification process.

[0093] (2) A plurality of functions possessed by one component in the above-described embodiment may be implemented by a plurality of components, or one function possessed by one component may be implemented by a plurality of components. Furthermore, a plurality of functions possessed by a plurality of components may be implemented by one component, or one function implemented by a plurality of components may be implemented by one component. In addition, a part of the configuration of the above-described embodiment may be omitted. Moreover, at least a part of the configuration of the above-described embodiment may be added to or replaced with the configuration of another of the above-described embodiments.

[0094] (3) In addition to the verification apparatus 1 described above, the present disclosure can also be implemented in various forms such as a verification program for causing a computer to function as the CPU 10 of the verification apparatus 1, a non-transitory tangible recording medium such as a semiconductor memory storing the verification program, and a method implemented by the verification program.

[0095] [8. Correspondence of Wording] S105 of the verification processing corresponds to the selection unit, S110 corresponds to the extraction unit and the identification unit, S115 corresponds to the correction unit, S120 corresponds to the generation unit and the display unit, S125 corresponds to the reception unit, the presentation unit, the difference detection unit, the node detection unit, the change point verification unit, and the regeneration unit, S130 and S145 correspond to the evaluation unit, S135 corresponds to the second acquisition unit, S140 corresponds to the first acquisition unit, S150 corresponds to the determination reception unit, and S155 corresponds to the first storage unit. [Description of Reference Signs]

[0096] 1... Verification apparatus, 10... CPU, 11... Memory, 12... Storage unit, 13... User I / F, 14... Display device, 15... Communication unit, 16... Past case DB, 17... Related information DB, 2... Metamodel, 20... Node, 21A to 21D... Edge, 3A, 3B... Document format, 30... Item, 31... Description block, 4A, 4B... Review target, 40... Item, 41... Description block, 5A, 5B... Graph structure, 6A to 6C... Evaluation item, 7... Related information, 9... Other document evaluation result.

Claims

1. A verification program that operates a computer as a verification device (1) for verifying the contents of documents (4A, 4B), An extraction unit (S110) extracts multiple descriptive blocks (41) from the aforementioned document, The extraction unit identifies a node (20) of the metamodel (2) that stores each of the description blocks extracted by the extraction unit, Based on the metamodel and the results of the identification in the identification unit, the computer is operated as a generation unit (S120) to generate the graph structure (5A, 5B) of the document. The metamodel is a model for generating the graph structure from the document, and has a plurality of nodes and edges (21A to 21D) that indicate the relationships between the nodes. The aforementioned graph structure is one in which each of the description blocks extracted by the extraction unit is stored in one of the nodes in the metamodel. Verification program.

2. A verification program as described in claim 1, As a display unit (S120) that displays the graph structure on the display device (14), Furthermore, it's a verification program that runs the computer.

3. A verification program according to claim 1 or claim 2, As an evaluation unit (S130, S145) that evaluates the description block stored in any of the nodes in the graph structure, Furthermore, it's a verification program that runs the computer.

4. A verification program according to claim 3, The evaluation unit further generates improvement proposals for the problems identified in the evaluation of the description block. Verification program.

5. A verification program according to claim 3, The computer is further operated as a reception unit (S125) that receives instructions from the user to select one of several predetermined evaluation items (6A to 6C). The evaluation unit evaluates the description block according to the evaluation item selected by the instruction received by the reception unit. Verification program.

6. A verification program according to claim 5, Each of the aforementioned evaluation items has a priority assigned to it. If there is an evaluation item with a higher priority than the evaluation item selected by the instruction received by the reception unit, the presentation unit (S125) presents the evaluation item to the user, Furthermore, it's a verification program that runs the computer.

7. A verification program according to claim 5, The evaluation unit identifies the node in the graph structure based on the evaluation item selected by the instruction received by the reception unit, and evaluates the description block stored in the identified node. Verification program.

8. A verification program according to claim 5, As one of the multiple evaluation items, a change evaluation item (6C) is provided for evaluating the impact of changes to the document. A difference detection unit (S125) detects the differences between the document before the aforementioned modification and the document to which the aforementioned modification has been made, which is the subject of evaluation by the evaluation unit, by further operating a computer. If the evaluation unit selects the change evaluation item based on the instruction received by the reception unit, the evaluation unit evaluates the description block containing the difference detected by the difference detection unit. Verification program.

9. A verification program according to claim 8, Based on the graph structure, a node detection unit (S125) detects the node that stores the evaluation item including the difference, and also detects other nodes related to that node, and further operates a computer. When the evaluation unit selects the change evaluation item based on the instruction received by the reception unit, it evaluates the description block containing the difference detected by the difference detection unit, as well as the description blocks stored in other nodes detected by the node detection unit. Verification program.

10. A verification program as described in claim 1, The extraction unit further extracts positional information (45) from the document, indicating the position of the description block in the document. The identifying unit further identifies the node storing the description block as the storage location for the position information of the description block. Verification program.

11. A verification program according to claim 10, A change verification unit (S125) detects differences between the document in which the graph structure was generated by the generation unit and the document in which changes were made to the document, and also detects the description block containing the differences and the position information of the description block. Based on the differences detected by the change verification unit, the description blocks, and the position information, the regeneration unit (S125) updates the graph structure of the document before the change to generate the graph structure of the document after the change. Furthermore, it's a verification program that runs the computer.

12. A verification program as described in claim 1, A selection unit (S105) further operates the computer to select the document that is most similar to the document among several document formats (3A, 3B). The extraction unit extracts a plurality of descriptive blocks from the document based on the document format selected by the selection unit. Verification program.

13. A verification program according to claim 12, The selection unit presents the selected document format to the user. Verification program.

14. A verification program according to claim 12 or claim 13, The identification unit identifies the node that stores each of the description blocks extracted by the extraction unit, based on the document format selected by the selection unit. Verification program.

15. A verification program according to claim 1 or claim 2, The editing unit (S115) modifies the text contained in the description block extracted by the extraction unit to predetermined text. Furthermore, it's a verification program that runs the computer.

16. A verification program according to claim 5, As a first acquisition unit (S140) that acquires evaluation results for other documents, which are evaluation results that follow the same or similar evaluation items as those for other documents, the computer is further operated. The evaluation unit evaluates the description block based on the evaluation items and the other document evaluation results obtained by the first acquisition unit. Verification program.

17. A verification program according to claim 16, The first acquisition unit acquires the other document evaluation results from the first storage unit (16), A judgment receiving unit (S150) receives the user's judgment result regarding the validity of the evaluation result of the description block by the evaluation unit, When the determination receiving unit receives a positive determination result regarding the evaluation result of the description block, the first storage unit (S155) stores the evaluation result in the first storage unit. Furthermore, it's a verification program that runs the computer.

18. A verification program according to claim 3, The computer is further operated as a second acquisition unit (S135) that acquires related information related to the description block of the document from a second storage unit (17) where information related to the content of the document is stored, The evaluation unit evaluates the description block based on the related information about the description block acquired by the second acquisition unit. Verification program.

19. A verification program according to claim 1 or claim 2, The aforementioned document is a requirements specification for developing software. Verification program.

20. Verification device (1) for verifying the contents of documents (4A, 4B), An extraction unit (S110) extracts multiple descriptive blocks (41) from the aforementioned document, The extraction unit identifies a node (20) of the metamodel (2) that stores each of the description blocks extracted by the extraction unit, The system comprises a generation unit (S120) that generates a graph structure (5A, 5B) of the document based on the metamodel and the identification result in the identification unit, The metamodel is a model for generating the graph structure from the document, and has a plurality of nodes and edges (21A to 21D) that indicate the relationships between the nodes. The aforementioned graph structure is one in which each of the description blocks extracted by the extraction unit is stored in one of the nodes in the metamodel. Verification device.

21. A verification method for a computer to verify the contents of a document (4A, 4B), Extracting multiple descriptive blocks (41) from the aforementioned document (S110), Identifying the node (20) of the metamodel (2) that stores each of the extracted description blocks (S110), The process includes generating a graph structure (5A, 5B) of the document based on the metamodel and the specific result (S120), The metamodel is a model for generating the graph structure from the document, and has a plurality of nodes and edges (21A to 21D) that indicate the relationships between the nodes. The aforementioned graph structure is one in which each of the extracted description blocks is stored in one of the nodes in the metamodel. Verification method.

Citation Information

Patent Citations

  • Input sentence verification device and program

    JP2022169011A

  • Method for supporting document preparation and system for supporting document preparation

    JP2024046629A