Design document review support system, method, and program

Abstract

This technology provides support for identifying deficiencies and trends in design documents. [Solution] A design document review support system that assists in the review of design documents includes a deficiency extraction unit that checks whether the descriptions in the first design document created in the first process and the second design document created in the second process, which is a downstream process of the first process, are appropriate and whether the first design document and the second design document are consistent with each other, and extracts deficiencies; and a weakness prediction unit that analyzes the extracted deficiencies and identifies weakness trends that indicate what kinds of deficiencies are likely to occur.

Description

Technical Field

[0006] , , , ,

[0001] This disclosure relates to a technology for assisting in the review of design documents.

Background Art

[0002] In the development of software and the like, design documents for each process are created in each process from the upstream to the downstream of the business flow. For example, in the requirements definition process, a requirements definition document is created, in the basic design process, a basic design document is created, and in the functional design process, a functional design document is created. These design documents are expected to contain sufficient content and for the design documents of the upstream process and the downstream process to be consistent with each other. However, there are cases where the design documents of the upstream process and the downstream process are not consistent, or where information that should be described in the design document itself is missing.

[0003] Therefore, it is important to review the design documents in software development and extract deficiencies.

[0004] Patent Document 1 discloses a technique for checking the consistency between design drawings such as drawings, design documents, and specifications. The system of Patent Document 1 extracts drawing information including characters or numerical values included in drawing data to generate drawing check data, extracts it as document information from document data through predetermined character recognition processing to generate document check data, and compares the drawing information included in the drawing check data with the document information included in the document check data to determine the consistency or inconsistency of the information described among the documents.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] As mentioned above, it is important to review design documents and identify deficiencies in software development, but it is also important to analyze the identified deficiencies and feed the results back into the creation of design documents for each stage of development. The technology described in Patent Document 1 can assist in the review of drawings and documents and contribute to the identification of deficiencies, but it does not assist in identifying trends in the deficiencies found.

[0007] One of the purposes included in this disclosure is to provide a technology that can assist in identifying deficiencies and trends in design documents. [Means for solving the problem]

[0008] A design document review support system according to one embodiment of the present disclosure is a design document review support system that supports the review of a design document, and includes a deficiency extraction unit that checks whether the descriptions in a first design document created in a first step and a second design document created in a second step which is a downstream step of the first step are appropriate and whether the first design document and the second design document are consistent with each other, and extracts deficiencies; and a weakness prediction unit that analyzes the extracted deficiencies and identifies weakness trends that indicate what kinds of deficiencies are likely to occur. [Effects of the Invention]

[0009] According to one aspect of this disclosure, it becomes possible to assist in identifying deficiencies and trends in design documents. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing an example of the overall system configuration. [Figure 2] This is a block diagram showing an example configuration for a design document review support system. [Figure 3] A flowchart illustrating an example of the overall process. [Figure 4] This is a flowchart illustrating an example of the deficiency detection process. [Figure 5]It is a block diagram showing an example of the hardware configuration of a design review support system. [Figure 6] It is a diagram showing an example of a prompt management database. [Figure 7] It is a diagram showing an example of a processing target management database. [Figure 8] It is a diagram showing an example of a processing order database. [Figure 9] It is a diagram showing an example of a setting management database. [Figure 10] It is a diagram showing an example of a result management database. [Figure 11] It is a diagram showing an example of a result display database. [Figure 12] It is a sequence diagram showing an example of the operation of a design review support system. [Figure 13] It is a sequence diagram showing an example of the operation of a design review support system. [Figure 14] It is a diagram showing an example of the specifications of a device. [Figure 15] It is a conceptual diagram showing an image of the analysis of a design document by a design review support system. [Figure 16] It is a diagram showing an example of the output screen of a design review support system.

Mode for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0012] <Overall System Outline>

[0013] FIG. 1 is a block diagram showing an example of the configuration of the overall system according to the present embodiment. The design review support system 10 has a processing unit 11 and a storage unit 12.

[0014] The design document review support system 10 analyzes the design document in the upstream process (hereinafter also referred to as the "upper-level specification document") in the business flow of software development and the design document in the downstream process, which is the next process after the upstream process (hereinafter also referred to as the "lower-level specification document"), extracts deficiencies, and analyzes the extracted deficiencies to identify the weakness tendencies indicating what types of deficiencies are likely to occur. As an example, the upstream process is the basic design process, and the upstream specification document is the basic design document. And the downstream process is the functional design process, and the downstream specification document is the functional design document. Deficiencies include parts where sufficient content is not correctly described, parts with omissions or gaps in the description. For example, parts where the specifications described in the design document are inaccurate are extracted as deficiencies. Also, points where sufficient specification content is not described in the design document are extracted as deficiencies. Also, parts where the description is missing in the design document are extracted as deficiencies.

[0015] The design document review support system 10 extracts deficiencies from the upper-level specification document and the lower-level specification document through analysis using an external service of generative artificial intelligence (hereinafter also referred to as "generative AI"). The generative AI may be, for example, ChatGPT (registered trademark), Claude (registered trademark), Llama, etc. Also, the design document review support system 10 performs regression analysis by itself on the extracted deficiencies to identify weakness tendencies. For regression analysis, for example, Catboost (registered trademark) of gradient boosting may be used. Or, LightGBM or XGBoost may be used.

[0016] For the design document review support system 10, prompts that will serve as input information to the generating AI are set in advance by the expert 91 in order to control the generating AI. The prompts include rules specific to the software development project being reviewed (hereinafter also referred to as "specific rules"). Specific rules include rules described in the design document (hereinafter also referred to as "internal specific rules") and rules not described in the design document (hereinafter also referred to as "external specific rules"), but the specific rules provided by the expert 91 here are external specific rules. By providing the generating AI with external specific rules, it becomes possible to utilize the generating AI with not only its own pure knowledge (AI knowledge) but also the prior knowledge (expert knowledge) possessed by the expert 91, thereby improving the quality of the output information from the generating AI.

[0017] When user 92 inputs the higher-level specification and lower-level specification into the design document review support system 10 and instructs it to perform analysis, the processing unit 11 in the design document review support system 10 uses the generated AI service 93 according to a pre-set prompt to perform checks on the descriptions of the higher-level and lower-level specifications using AI knowledge (hereinafter also referred to as "knowledge check"), checks on the descriptions of the higher-level and lower-level specifications using external specific rules which are expert knowledge, and checks for consistency between the higher-level and lower-level specifications, and extracts any deficiencies. Furthermore, the processing unit 11 classifies the extracted deficiencies by location classification, which classifies the location of the deficiency in the design document, and by content classification, which classifies the content of the deficiency. For example, if the deficiency is in the functional design process, its location will be indicated by the function.

[0018] Furthermore, the processing unit 11 performs regression analysis on deficiencies based on location classification and content classification to identify trends (hereinafter also referred to as "weakness trends") that indicate what kinds of deficiencies are likely to occur.Then, the processing unit 11 records the internal specific rules obtained from the design document, the pre-given external specific rules, the results of the deficiency and weakness trends, and the pre-given learning parameters in the storage unit 12.Furthermore, the processing unit 11 retrieves the internal specific rules, external specific rules, and results of the deficiencies from the storage unit 12 and presents to the user 92 the results of the knowledge check of the higher-level and lower-level specifications, the results of the check of specific rules, the results of the consistency check between the higher-level and lower-level specifications, and the weakness analysis results showing the identified weakness trends.As described above, the results of the knowledge check of the higher-level and lower-level specifications, the results of the check of specific rules, and the results of the consistency check between the higher-level and lower-level specifications are the results of analysis by the generating AI.The weakness analysis results are the results of regression analysis.

[0019] <Overview of the Design Document Review Support System Configuration>

[0020] Figure 2 is a block diagram showing an example configuration of a design document review support system. Referring to Figure 2, the design document review support system 10 includes a processing unit 11 and a storage unit 12, as well as a user interface unit 13.

[0021] The processing unit 11 executes various processes using various data recorded in the storage unit 12. The storage unit 12 stores various data necessary for processing by the processing unit 11. The user interface unit 13 receives information input by the expert 91 or user 92 and provides it for processing by the processing unit 11. It also presents the information generated by the processing of the processing unit 11 to the expert 91 or user 92.

[0022] The processing unit 11 includes a main routine unit 21, a parameter management unit 22, a document management unit 23, a token quantity check unit 24, an analysis target control unit 25, a generation AI control unit 30, a regression analysis AI unit 33, and an output organization unit 38.

[0023] The main routine unit 21 controls the entire processing unit 11, coordinating each unit to perform each process.

[0024] The parameter management unit 22 manages various parameters used in each process. These parameters include prompts, which are input information for the generating AI provided by the experts 91, and learning parameters for regression analysis. Setting the prompts and learning parameters is done as preparation. The prompts are text that instructs the generating AI to extract deficiencies in the higher-level and lower-level specifications. The prompts may include external specific rules. External parameters are, for example, parameters for properly executing regression analysis. External parameters may include the learning rate and the number of epochs.

[0025] The Document Management Unit 23 manages the design documents that are subject to review, i.e., those that are processed or analyzed. The design documents subject to processing include the higher-level and lower-level specifications mentioned above.

[0026] The Token Amount Check Unit 24 checks the token amount of the design document to be processed. A token is a unit of text processed by the generating AI. The token amount of the design document indicates how many tokens make up the text of that design document.

[0027] The analysis target control unit 25 divides the text of the prompts and design documents into text of a size that can be input to the generating AI, and sets the order in which to input them to the generating AI, based on the number of tokens in the prompts and design documents and the upper limit of the number of tokens that can be input to the generating AI at one time (maximum number of tokens).

[0028] The generation AI control unit 30 is a defect extraction unit that extracts defects from higher-level and lower-level specifications through analysis using generation AI. The regression analysis AI unit 33 is a weakness prediction unit that identifies weakness trends from the extracted defects through regression analysis. The output organization unit 38 presents a screen showing defects and / or weakness trends based on the data showing the analysis results. These generation AI control unit 30, regression analysis AI unit 33, and output organization unit 38 realize the main processing of the design document review support system 10.

[0029] <Overview of the design document review support system>

[0030] The following is an overview of the processes performed by the design document review support system 10.

[0031] Figure 3 is a flowchart showing an example of the overall process.

[0032] Referring to Figure 3, first, in step S101, the design document review support system 10 performs a defect extraction process using the generation AI control unit 30.

[0033] The deficiency extraction process is a process that uses a generation AI to extract deficiencies in higher-level and lower-level specifications. The generation AI control unit 30 checks whether the descriptions in the higher-level and lower-level specifications are appropriate, checks whether the higher-level and lower-level specifications are consistent with each other, and extracts deficiencies. In doing so, the generation AI control unit 30 uses the generation AI to check the design documents. Specifically, the prompt set in the pre-preparation, along with the higher-level and lower-level specifications, are input to the generation AI as input information, and output information indicating deficiencies is obtained from the generation AI.

[0034] Figure 4 is a flowchart showing an example of the deficiency detection process.

[0035] Referring to Figure 4, in step S201, the generation AI control unit 30 requests the generation AI to perform a validity and consistency check from the perspective of viewing the downstream design document from the upstream design document (hereinafter also referred to as the "first check perspective"), and obtains a response. As a validity check, it is examined whether the upstream and downstream design documents contain specifications that are sufficiently valid, by looking at them from the upstream design document to the downstream design document. As a consistency check, it is examined whether the upstream and downstream design documents contain specifications that are consistent with the specifications described in the other design document, by looking at them from the upstream design document to the downstream design document.

[0036] Next, in step S202, the generation AI control unit 30 requests the generation AI to perform a check of validity and consistency from the perspective of viewing the upstream design document from the downstream design document (hereinafter also referred to as the "second check perspective"), and obtains a response. As a check of validity, it is examined whether the specifications in the downstream and upstream design documents are sufficiently valid, by looking at the upstream design document from the downstream design document. As a check of consistency, it is examined whether the specifications in the downstream and upstream design documents are consistent with the specifications described in the other design document, by looking at the upstream design document from the downstream design document.

[0037] Next, in step S203, the generation AI control unit 30 requests the generation AI to check the upstream and downstream design documents as a whole for any omissions and obtains a response. Here, for example, it is expected that the AI ​​will extract sections where information is missing in both the upstream and downstream specifications, or sections where both the upstream and downstream specifications contain the same errors.

[0038] Furthermore, the generation AI control unit 30 may also check for compliance with external specific rules received via the parameter management unit 22. These external specific rules may include, for example, information regarding design specifications not described in the design document, or information regarding the specifications of devices used in the target system. Furthermore, the generation AI control unit 30 may also further check for compliance with internal specific rules for each design document (for example, information regarding design specifications described in each design document). These internal specific rules may include, for example, information regarding design specifications described in each design document.

[0039] Returning to Figure 3, in step S102, the generation AI control unit 30 performs a defect classification process. The defect classification process is a process that classifies the defects extracted in step S102 in a way that is suitable for regression analysis. Specifically, the generation AI control unit 30 classifies the extracted defects by location classification, which classifies the location where the defects are located, and content classification, which classifies the content of the defects.

[0040] Next, in step S103, the regression analysis AI unit 33 analyzes the defects extracted by the generation AI control unit 30 and identifies weakness trends that indicate what kinds of defects are likely to occur. Specifically, the regression analysis AI unit 33 identifies weakness trends by performing regression analysis on the defect data.

[0041] In this regression analysis, as an example, the explanatory variables may be which design document the deficiencies were extracted from, and from which of the first and second check criteria the deficiencies were extracted, while the location classification and content classification described above may be used as the dependent variable. The regression analysis AI unit 33 can, for example, identify weakness trends based on the strength of the relationship between the explanatory variables and the dependent variable. It can be said that deficiencies in classifications where the relationship between the explanatory variables and the dependent variable is strong tend to occur. For example, it may be determined that deficiencies in classifications where the regression coefficients of each dependent variable are above a predetermined threshold tend to occur.

[0042] Next, in step S104, the output processing unit 38 presents the extracted deficiencies and / or identified weakness trends to the user 92. At this time, the output processing unit 38 may present the deficiencies and their corresponding sentences or summaries to the user, based on the sentences that point out each deficiency generated by the generation AI control unit 30.

[0043] In this way, the design document review support system 10 effectively assists in identifying deficiencies and trends in design documents, thereby contributing to the improvement of design document quality.

[0044] <Overview of the Hardware Configuration of the Design Document Review Support System>

[0045] Figure 5 is a block diagram showing an example of the hardware configuration of a design document review support system.

[0046] The design document review support system 10 is, for example, composed of a computer. The computer 50 includes a processor 51, memory 52, auxiliary storage device 53, communication interface 54, media interface 55, and input / output device 56.

[0047] The processor 51 is composed of components such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), and FPGA (Field-Programmable Gate Array). The processor 51 performs various functions by reading and executing programs stored in memory 52 or auxiliary storage device 53.

[0048] The auxiliary storage device 53 may be, for example, a hard disk or an SSD (Solid State Drive). The auxiliary storage device 53 stores programs executed by the processor 51 and various types of data. An external storage device (not shown) may be located outside the computer 50. The external storage device may be connected to the computer 50 in a way that allows communication. The external storage device may be located, for example, on the cloud. The external storage device may be located on the same premises as the computer 50. The external storage device stores programs executed by the processor 51 and various types of data.

[0049] The communication interface 54 facilitates the communication of information between the computer 50 and various devices via the network.

[0050] The media interface 55 communicates information with various media such as CDs, USB memory sticks, SD cards, DVDs, and Blu-rays.

[0051] The input / output device 56 performs information input to the computer 50 and information output from the computer 50 to an external device. For example, the input / output device 56 may output information to an external console (not shown). The external console is an input and / or output device connected directly or remotely to the computer 50. An example of an external console is a keyboard, mouse, and display.

[0052] <Details of the design document review support system operation>

[0053] This section describes the operation of the design document review support system 10 in detail.

[0054] Referring again to Figure 2, the generation AI control unit 30 has a generation AI call unit 31 and a result control unit 32. The regression analysis AI unit 33 has a preprocessing unit 34, a learning control unit 35, a regression analysis AI processing unit 36, and a result control unit 37. The operation of the generation AI call unit 31, the result control unit 32, the preprocessing unit 34, the learning control unit 35, the regression analysis AI processing unit 36, and the result control unit 37 will be explained in the description of the operation of the design document review support system 10.

[0055] The storage unit 12 stores a prompt management database (DB) 41, a processing target management database 42, a processing sequence database 43, a settings management database 44, a result management database 45, and a result display database 46.

[0056] Figure 6 shows an example of a prompt management database.

[0057] Referring to Figure 6, the prompt management database 41 stores information for each prompt, including its type, prompt text, and token quantity. The type indicates whether the prompt relates to an internal or external rule. The prompt text is the text of the prompt itself. The token quantity indicates the amount of tokens used for the prompt.

[0058] Figure 7 shows an example of a database for managing the items to be processed.

[0059] Referring to Figure 7, the processing target management database 42 stores information on the document type, document name, and token amount for each design document to be processed. The document type indicates whether the design document is a higher-level specification or a lower-level specification. The document name indicates the name of the design document. The token amount indicates the token amount of the design document.

[0060] Figure 8 shows an example of a processing sequence database.

[0061] Referring to Figure 8, the processing sequence database 43 stores information for each combination of higher-level and lower-level specifications to be processed, including the processing sequence, the name of the higher-level specification document, the name of the lower-level specification document, and the amount of tokens. The processing sequence indicates the order in which the combination is processed. The name of the higher-level specification document indicates the name of the higher-level specification for that combination. The name of the lower-level specification document indicates the name of the lower-level specification for that combination. The amount of tokens indicates the total amount of tokens for the higher-level and lower-level specifications for that combination.

[0062] Figure 9 shows an example of a configuration management database.

[0063] Referring to Figure 9, the configuration management database 44 stores pre-configured parameters. These parameters include Max Token, Learning Rate, Epoch Count, and Active Generation AI Service. Max Token indicates the upper limit of the amount of tokens that can be input to the generation AI at one time (maximum token amount). Learning Rate indicates the learning rate for regression analysis. Epoch Count indicates the number of epochs set for the regression analysis model. Active Generation AI Service indicates the generation AI service being used.

[0064] Figure 10 shows an example of a results management database.

[0065] Referring to Figure 10, the results management database 45 stores the output, type, document name, location classification, content classification, and sentence (comment) for each analysis result. The output indicates whether it is a deficiency extracted by the generation AI or a weakness trend identified by the regression analysis AI. The type indicates whether the analysis result is from the perspective of viewing lower-level specifications from higher-level specifications or from the perspective of viewing higher-level specifications from lower-level specifications. The document name indicates which design document the deficiency or weakness trend relates to. The location classification indicates the location classification of the deficiency or weakness trend. The content classification indicates the content classification of the deficiency or weakness trend. The sentence (comment) is the comment text output by the generation AI.

[0066] Figure 11 shows an example of a results display database.

[0067] Referring to Figure 11, the results display database 46 stores the output, type, document name, location classification, content classification, and sentence (comment: summary) as display data for each analysis result. The output indicates whether it is a deficiency extracted by the generation AI or a weakness trend identified by the regression analysis AI. The type indicates whether it is an analysis result from the perspective of viewing lower-level specifications from higher-level specifications or from the perspective of viewing higher-level specifications from lower-level specifications. The document name indicates which design document the deficiency or weakness trend relates to. The location classification indicates the location classification of the deficiency or weakness trend. The content classification indicates the content classification of the deficiency or weakness trend. The sentence (comment: summary) shows the comment text output from the generation AI or its summary.

[0068] Figures 12 and 13 are sequence diagrams showing an example of the operation of the design document review support system.

[0069] Referring to Figure 12, the expert 91 pre-sets prompts for the design document review support system 10, including external specific rules. These prompts include instructions for extracting deficiencies through AI knowledge checks for generating higher-level specifications, AI knowledge checks for generating lower-level specifications, specific rule checks, and higher-to-lower-level consistency checks, as well as for classifying the extracted deficiencies by their location and content. For example, an external or internal specific rule may be used to check whether the system design specifications described in the design document are compatible with the specifications specific to the devices used in that system.

[0070] Figure 14 is a diagram showing an example of device specifications. Figure 14 shows the specifications of a front camera in an automobile. It is possible to check whether the design specifications of a system using the front camera conform to the specifications of the front camera. The prompt setting is received by the user interface unit 13 as an instruction to the external prompt storage, and the prompt, along with its type and token amount, is recorded in the prompt management database 41 of the storage unit 12 via the main routine unit 21.

[0071] Subsequently, user 92 inputs the higher-level specification and lower-level specification into the design document review support system 10 and instructs the system to inspect these design documents. The instruction to inspect the design documents, the higher-level specification, and the lower-level specification are received as document management instructions by the user interface unit 13. The higher-level specification and lower-level specification are then recorded in the processing target management database 42 of the storage unit 12, along with information on their respective document types and document names, via the main routine unit 21. Furthermore, the main routine unit 21 confirms the token quantity of each of the higher-level specification and lower-level specification and records it in the processing target management database 42 of the storage unit 12.

[0072] Next, the main routine unit 21 instructs the analysis target control unit 25 to control the analysis target.

[0073] Upon receiving instructions for control of the analysis target, the analysis target control unit 25 first accesses the prompt management database 41 of the storage unit 12 to request the amount of internal and external prompt tokens and obtains information on the amount of tokens for the prompts (prompt token amount 26). Furthermore, the analysis target control unit 25 accesses the processing target management database 42 of the storage unit 12 to request the amount of tokens for each document and obtains information on the amount of tokens for the higher-level specification and lower-level specification (each document token amount 27). In addition, the analysis target control unit 25 accesses the setting management database 44 of the storage unit 12 to request the maximum amount of tokens that can be processed and obtains information on the maximum amount of tokens for the generated AI (maximum token amount 28).

[0074] Next, the analysis target control unit 25 calculates the number of times input information to be entered into the generating AI (number of repeating cycles) based on the token amount of the prompt, the token amounts of the higher-level and lower-level specifications, and the maximum token amount of the generating AI. Based on the prompt, the higher-level specifications, and the lower-level specifications, it generates input information for each cycle. The information, including the document names and token amounts of the higher-level and lower-level specifications for each cycle corresponding to the number of repeating cycles, is recorded in the processing order database 43 of the storage unit 12 as the processing order setting 29, and the completion of analysis target control is notified to the main routine unit 21.

[0075] Upon receiving notification that the control of the analysis target has been completed, the main routine unit 21 instructs the analysis target control unit 25 to perform generation AI control. Upon receiving the instruction to perform generation AI control, the analysis target control unit 25 requests a generation AI call from the generation AI control unit 30.

[0076] Upon receiving a request for a generation AI call, the generation AI control unit 30 accesses the processing sequence database 43 in the storage unit 12 as a check of the number of repeat cycles, and obtains the number of repeat cycles and information to be used as input to the generation AI for each cycle. Subsequently, the generation AI control unit 30 repeats the cycle for the number of repeat cycles in which the generation AI call unit 31 inputs input information to the generation AI service 93 via API call and obtains output information from the generation AI, and the result control unit 32 records the analysis results based on the output information in the result management database 45 in the storage unit 12. This process is repeated for the number of repeat cycles. As a result, generation AI knowledge checks are performed on the higher-level specification, generation AI knowledge checks are performed on the lower-level specification, unique rule checks are performed, and higher-level-lower-level consistency checks are performed, and the extracted deficiencies are classified into location classification and content classification. The result management database 45, as illustrated in Figure 11, records the type, document name, location classification, content classification, and pointed-out sentences based on the output information from the generation AI for each deficiency extracted by the check, with the output type being generation AI.

[0077] When the execution of the specified number of cycles is complete, the generation AI control unit 30 notifies the analysis target control unit 25 that the generation AI call has been completed. Upon receiving notification of the completion of the generation AI call, the analysis target control unit 25 notifies the main routine unit 21 that the generation AI control has been completed.

[0078] Upon receiving notification that the generation AI control is complete, the main routine unit 21 requests the regression analysis AI unit 33 to perform an analysis using the regression analysis AI. In the regression analysis AI unit 33, upon receiving the analysis request, the preprocessing unit 34 accesses the result management database 45 in the storage unit 12 and obtains the analysis result data. Then, in the regression analysis AI unit 33, the learning control unit 35 accesses the setting management database 44 in the storage unit 12 as learning control, obtains hyperparameters to be used for machine learning of the regression analysis AI, and performs machine learning of the model using the hyperparameters. Furthermore, in the regression analysis AI unit 33, the regression analysis AI processing unit 36 ​​performs multiple regression analysis on the analysis results of the generation AI using the machine-learned model to identify weakness trends. At this time, for example, the type of entry and process (or document name) of the output type in the result management database 45 exemplified in Figure 11 can be used as explanatory variables, and location classification and content classification can be used as target variables for machine learning. Machine learning may also be performed using a high-speed learning (gradient boosting) function. Alternatively, a model may be generated using the functions of the multiple regression analysis AI, such as missing value imputation, categorical features, overfitting control, and feature acquisition. In the regression analysis AI unit 33, the results control unit 37 records the analysis results of the multiple regression analysis in the results management database 45 of the storage unit 12, with the output type being regression analysis AI.

[0079] Once the regression analysis is complete, the regression analysis AI unit 33 notifies the generation AI control unit 30 that the regression analysis is complete. The generation AI control unit 30, having been notified of the completion of the regression analysis, notifies the analysis target control unit 25 that the regression analysis is complete. The analysis target control unit 25, having been notified of the completion of the regression analysis, notifies the main routine unit 21 that the regression analysis is complete.

[0080] Moving from Figure 12 to Figure 13, the main routine unit 21, notified of the completion of the regression analysis, instructs the generation AI control unit 30 to summarize the long sentences identified in the analysis results by the generation AI as long sentence summaries. Upon receiving the instruction to summarize long sentences, the generation AI control unit 30 requests the generation AI service to summarize the long sentences identified in the result management database 45, obtains the summaries, and notifies the main routine unit 21 of the obtained summaries. Upon receiving the notification, the main routine unit 21 records the notified summaries in the result display database 46 of the storage unit 12. The summarization of long sentences is repeated for each long sentence.

[0081] Once the summarization of all long texts is complete, the main routine unit 21 requests output information, retrieves data from the results display database 46 as display information, and displays it on the user 92's screen via the user interface unit 13.

[0082] Figure 15 is a conceptual diagram illustrating the analysis of design documents by a design document review support system. Figure 15 shows how the system extracts and classifies defects using a generation AI, and then identifies weakness trends through regression analysis. In the example in Figure 15, it is identified that the extracted defects tend to have a weakness where the lower limit is easily missed.

[0083] Figure 16 shows an example of the output screen of a design document review support system.

[0084] On screen 60, area 61 shows the name of the user currently using the design document review support system 10. Area 62 shows the design document to be processed. Area 63 shows the generation AI service used for processing. Area 64 shows information from the previous execution. Area 65 shows a list of issues. The issues include deficiencies extracted by the generation AI and weakness trends identified by multiple regression analysis. Issue List No. 2 is the output of the multiple regression analysis AI, which points out potential problems based on past information. The issues in Issue List No. 2 are an example of how the regression analysis AI effectively addressed the deficiencies occurring in Issue List No. 1.

[0085] The embodiments described above are illustrative for explaining the present invention and are not intended to limit the scope of the invention to those embodiments only. Those skilled in the art can implement the present invention in various other forms without departing from the scope of the invention.

[0086] Furthermore, the embodiments described above include the following items. However, the items included in the embodiments described above are not limited to those shown below.

[0087] (Item 1)

[0088] A design document review support system assists in the review of design documents and includes a deficiency extraction unit that checks whether the descriptions in the first design document created in the first process and the second design document created in the second process, which is a downstream process of the first process, are appropriate, and whether the first design document and the second design document are consistent with each other, and extracts any deficiencies.

[0089] The system includes a weakness prediction unit that analyzes the extracted defects and identifies weakness trends indicating what types of defects are likely to occur. This makes it possible to support the extraction of defects and identification of trends in design documents.

[0090] (Item 2)

[0091] In the design document review support system described in item 1, the defect extraction unit inputs the first design document and the second design document as input information to the generative artificial intelligence and obtains output information indicating defects from the generative artificial intelligence. This enables the generative artificial intelligence to perform a thorough check of the design document.

[0092] (Item 3)

[0093] In the design document review support system described in item 2, the deficiency extraction unit causes the generative artificial intelligence to inspect the consistency between the first and second design documents based on a first check criterion: whether the description in the second design document is appropriate and consistent with the description in the first design document, and a second check criterion: whether the description in the first design document is appropriate and consistent with the description in the second design document, and obtains the output information from the generative artificial intelligence. As a result, the validity and consistency checks are performed bidirectionally between the upstream and downstream design documents, so that any parts of each design document that are not appropriate and any parts that are missing or incomplete can be checked effectively.

[0094] (Item 4)

[0095] In the design document review support system described in item 3, the weakness prediction unit identifies the weakness trend by performing regression analysis on the deficiencies shown in the output information. This allows for the accurate identification of the trend of deficiencies in a series of design documents through regression analysis.

[0096] (Item 5)

[0097] In the design document review support system described in item 4, the defect extraction unit, upon extracting defects, classifies the defects by location classification, which classifies the location of the defects, and by content classification, which classifies the content of the defects. The weakness prediction unit includes which design document the defects were extracted from as an explanatory variable, and includes the location classification and content classification as dependent variables, and identifies the weakness tendency based on the strength of the relationship between the explanatory variable and the dependent variable. This makes it possible to effectively identify what kinds of defects are likely to occur in what kinds of design documents.

[0098] (Item 6)

[0099] In the design document review support system described in item 5, the weakness prediction unit further includes as an explanatory variable which of the first and second check perspectives the deficiency was extracted from, and identifies the weakness trend based on the strength of the relationship between the explanatory variable and the objective variable. This makes it possible to effectively identify which perspectives and in which design documents deficiencies are likely to be extracted.

[0100] (Item 7)

[0101] In the design document review support system described in item 1, the deficiency extraction unit further checks whether the description in the first design document satisfies the internal specific rules which are information described in the first design document regarding the design specifications, and whether the description in the second design document satisfies the internal specific rules which are information described in the second design document regarding the design specifications.

[0102] (Item 8)

[0103] The design document review support system described in item 1 further includes a parameter management unit that accepts information regarding the input design specifications as external specific rules, and the deficiency extraction unit further checks whether the first design document and the second design document satisfy the external specific rules.

[0104] (Item 9)

[0105] The design document review support system described in item 1 further includes an output organization unit that presents a screen showing the weakness trends based on the data indicating the weakness trends.

[0106] (Item 10)

[0107] In the design document review support system described in item 9, the defect extraction unit generates a statement sentence explaining the defect, the output organization unit stores the statement sentence for the defect, and displays a screen showing the defect and the statement sentence or a summary of the statement sentence.

[0108] (Item 11)

[0109] A design document review support method for supporting the review of design documents, wherein a computer inspects whether the descriptions in the first design document created in the first process and the second design document created in the second process, which is a downstream process of the first process, are appropriate, and whether the first design document and the second design document are consistent with each other, extracts deficiencies, analyzes the extracted deficiencies, and identifies weakness trends indicating what kinds of deficiencies are likely to occur.

[0110] (Item 12)

[0111] A design document review support program for supporting the review of design documents, which uses a computer to check whether the descriptions in the first design document created in the first process and the second design document created in the second process, which is a downstream process of the first process, are appropriate, and whether the first design document and the second design document are consistent with each other, and to extract any deficiencies,

[0112] The extracted deficiencies are analyzed, and weakness trends indicating what types of deficiencies are likely to occur are identified. [Explanation of Symbols]

[0113] 10...Design document review support system, 11...Processing unit, 12...Storage unit, 13...User interface unit, 21...Main routine unit, 22...Parameter management unit, 23...Document management unit, 24...Token amount check unit, 25...Analysis target control unit, 26...Prompt token amount, 27...Document token amount, 28...Maximum token amount, 29...Processing order setting, 30...Generation AI control unit, 31...Generation AI call unit, 32...Result control unit, 33...Regression analysis AI unit, 34...Preprocessing unit, 35...Learning control unit, 36...Regression analysis AI processing unit, 37...Result control unit, 38...Output organization unit, 41...Prompt management database, 42...Processing target management database, 43...Processing sequence database, 44...Setting management database, 45...Result management database, 46...Result display database, 50...Computer, 51...Processor, 52...Memory, 53...Auxiliary storage device, 54...Communication interface, 55...Media interface, 56...Input / output device, 60...Screen, 91...Expert, 92...User, 93...Generating AI service

Claims

1. A design document review support system that assists in the review of design documents, A deficiency detection unit inspects whether the descriptions in the first design document created in the first process and the second design document created in the second process, which is a downstream process of the first process, are appropriate, and whether the first design document and the second design document are consistent with each other, and extracts any deficiencies. A weakness prediction unit analyzes the extracted defects and identifies weakness trends that indicate what kinds of defects are likely to occur, A design document review support system that has the following features.

2. The defect detection unit inputs the first design document and the second design document as input information to the generative artificial intelligence, and obtains output information indicating defects from the generative artificial intelligence. The design document review support system according to claim 1.

3. The defect extraction unit causes the generative artificial intelligence to inspect the consistency between the first and second design documents based on a first check criterion: whether the description in the second design document is appropriate and consistent with the description in the first design document, and a second check criterion: whether the description in the first design document is appropriate and consistent with the description in the second design document, and then obtains the output information from the generative artificial intelligence. The design document review support system according to claim 2.

4. The weakness prediction unit identifies the weakness trend by performing regression analysis on the deficiencies shown in the output information. The design document review support system according to claim 3.

5. When the defect extraction unit extracts defects, it classifies the defects by location classification, which classifies the location where the defects are located, and by content classification, which classifies the content of the defects. The weakness prediction unit includes, as an explanatory variable, which design document the deficiency was extracted from, and as dependent variables, the location classification and the content classification, and identifies the weakness trend based on the strength of the relationship between the explanatory variable and the dependent variable. The design document review support system according to claim 4.

6. The weakness prediction unit further includes, as an explanatory variable, which of the first and second check perspectives the deficiency was extracted from, and identifies the weakness trend based on the strength of the relationship between the explanatory variable and the objective variable. The design document review support system according to claim 5.

7. The defect detection unit further checks whether the description in the first design document satisfies the internally specific rules which are information described in the first design document regarding the design specifications, and whether the description in the second design document satisfies the internally specific rules which are information described in the second design document regarding the design specifications. The design document review support system according to claim 1.

8. It further includes a parameter management unit that accepts information regarding the input design specifications as externally specific rules, The defect detection unit further checks whether the first design document and the second design document satisfy the external specific rules. The design document review support system according to claim 1.

9. The system further includes an output processing unit that displays a screen showing the weakness trends based on the data indicating the weakness trends. The design document review support system according to claim 1.

10. The aforementioned defect extraction unit generates a sentence that explains the defect, The output processing unit stores the sentences that point out the deficiencies and displays a screen showing the deficiencies and the sentences that point out the deficiencies, or a summary of the sentences that point out the deficiencies. The design document review support system according to claim 9.

11. A method for supporting the review of design documents, Computers The first design document, created in the first process, and the second design document, created in the second process which is a downstream process of the first process, are examined to determine whether their descriptions are appropriate and whether the first and second design documents are consistent with each other, and any deficiencies are identified. The extracted deficiencies are analyzed to identify weaknesses that indicate what types of deficiencies are likely to occur. A method for supporting design document reviews to execute the task.

12. A design document review support program to assist in the review of design documents, On the computer, The first design document, created in the first process, and the second design document, created in the second process which is a downstream process of the first process, are examined to determine whether their descriptions are appropriate and whether the first and second design documents are consistent with each other, and any deficiencies are identified. The extracted deficiencies are analyzed to identify weaknesses that indicate what types of deficiencies are likely to occur. A design document review support program that enables the execution of this task.

Images