Drawing generation program and drawing generation device

Abstract

The system generates drawings related to the description in the drawing. [Solution] The drawing generation device 1 is characterized by comprising: an explanatory text extraction unit 12 that extracts explanatory text for a specified drawing from input natural language text; a Markdown notation conversion unit 13 that inputs a prompt to a generation AI system 3 that includes an instruction to generate Mermaid notation text from the explanatory text for the drawing, and obtains a response including Mermaid notation text from the generation AI system 3; and a drawing drawing unit 15 that draws a drawing from this Mermaid notation text.

Description

Technical Field

[0001] The present invention relates to a drawing generation program and a drawing generation device.

Background Art

[0002] With the recent development of AI technology, it has been proposed to support the generation of various contents with the assistance of generative AI. For example, Patent Document 1 describes an invention that suppresses variations in quality during claim creation or reduces the time required for preparing a patent application manuscript including claims. Thus, many techniques for humans to create sentences have been implemented with the support of generative AI.

[0003] A technique for creating drawings with the support of generative AI is described in Non-Patent Document 1. In Non-Patent Document 1, by telling ChatGPT (GPT-4) the requirements of the drawing to be created, the Mermaid syntax is output, and attempts are made to create various drawings such as sequences, Gantt charts, and flowcharts.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Non-Patent Documents

[0005]

Non-Patent Document 1

[0006] Non-patent document 1 generates Mermaid notation text from a bulleted list of requirements. However, in reality, drawing descriptions are often written in natural language. Furthermore, the drawing type may not be specified in the drawing description. In such cases, the drawing type may not be correctly recognized, and the Mermaid notation text may not be generated properly.

[0007] Furthermore, generative AIs such as OpenAI's ChatGPT (registered trademark) and Google's Gemini have the ability to accept text prompts and generate images as instructed by those prompts. However, this image generation function cannot draw flowcharts, mode transition diagrams, or sequence diagrams. Therefore, the object of the present invention is to generate a drawing related to a description from the description of the drawing. [Means for solving the problem]

[0008] In other words, the above-mentioned problems of the present invention are solved by the following configuration. The present invention provides a drawing generation program for causing a computer to perform the following steps: extract a descriptive text relating to a specified drawing from input natural language text; input a first prompt to a generation AI that includes an instruction to generate a predetermined Markdown notation text from the descriptive text, and obtain a response from the generation AI that includes the predetermined Markdown notation text; and draw a drawing from the predetermined Markdown notation text.

[0009] The present invention provides a drawing generation device comprising: an explanatory text extraction unit that extracts explanatory text for a specified drawing from input natural language text; a Markdown notation conversion unit that inputs a prompt to a generation AI that includes an instruction to generate predetermined Markdown notation text from the explanatory text for the drawing, and obtains a response containing predetermined Markdown notation text from the generation AI; and a drawing drawing unit that draws a drawing from the predetermined Markdown notation text. Other configurations will be described in the embodiments section. [Effects of the Invention]

[0010] According to the present invention, it is possible to generate a drawing related to a description from the description of the drawing. [Brief explanation of the drawing]

[0011] [Figure 1] This is a logical configuration diagram showing a drawing generation device according to the first embodiment. [Figure 2] This is a hardware configuration diagram showing the drawing generation device. [Figure 3] This is the screen displayed by the drawing generation device. [Figure 4] This is a flowchart of the process for generating explanatory text. [Figure 5] This diagram shows the request sent to the generation AI system. [Figure 6] This is a diagram explaining the content of the prompt. [Figure 7] This diagram shows chat completion objects received from the generation AI system. [Figure 8] This is a diagram illustrating the content of the responses included in the chat completion object. [Figure 9] This diagram shows the request sent to the generation AI system. [Figure 10] This is a diagram explaining the content of the prompt. [Figure 11] This diagram shows chat completion objects received from the generation AI system. [Figure 12]This is a diagram for explaining the content of the answer included in the chat completion object. [Figure 13] This is a flowchart drawn based on Mermaid format text. [Figure 14] This is a logical configuration diagram showing a drawing generation device according to the second embodiment. [Figure 15] This is a flowchart of drawing generation processing. [Figure 16] This is Mermaid format text generated from the explanatory text of the mode transition diagram. [Figure 17] This is an example of a mode transition diagram. [Figure 18] This is Mermaid format text generated from the explanatory text of the sequence diagram. [Figure 19] This is an example of a sequence diagram.

Embodiments for Implementing the Invention

[0012] Hereinafter, embodiments for implementing the present invention will be described in detail with reference to each drawing. FIG. 1 is a logical configuration diagram showing a drawing generation device 1 according to the first embodiment. The drawing generation device 1 includes an explanatory text specifying unit 11, an explanatory text extraction unit 12, a Markdown notation conversion unit 13, a Markdown notation extraction unit 14, and a drawing drawing unit 15. The drawing generation device 1 is a device that collaborates with the generation AI system 3 to generate a drawing 26 specified by a drawing number 22 from a natural text 21.

[0013] When the natural text 21 and the drawing number 22 are input, the explanatory text specifying unit 11 instructs the generation AI system 3 to specify an explanatory text from this natural text 21 by a request body 51 including a first prompt. The natural text 21 includes an explanatory text of the drawing specified by the drawing number 22. The drawings are a flowchart, a sequence diagram, a mode transition diagram, and the like.

[0014] This request 51 is written in JSON format. The request 51 is executed by the generation AI system 3, for example, by calling the API (Application Programming Interface) of the generation AI system 3, and a chat completion object 52 is generated. Then, the generation AI system 3 sends the chat completion object 52 in JSON format to the description identification unit 11. This chat completion object 52 contains the description 23 of the drawing specified by drawing number 22.

[0015] The description extraction unit 12 extracts the description 23 of the drawing specified by drawing number 22 from the chat completion object 52. When the Markdown notation conversion unit 13 receives the description text 23 and the drawing type 24 as input, it instructs the generation AI system 3 via a request body 53, which includes a second prompt, to convert the description text 23 into Mermaid notation text 25. The drawing type 24 includes flowcharts, sequence diagrams, mode transition diagrams, etc.

[0016] The request 53 is written in JSON format. This request 53 is executed by the generation AI system 3, for example, by calling the API of the generation AI system 3, and a chat completion object 54 is generated. The generation AI system 3 then returns the chat completion object 54 in JSON format to the Markdown notation conversion unit 13. This chat completion object 54 contains Mermaid notation text 25.

[0017] The Markdown notation extraction unit 14 extracts Mermaid notation text 25 from the chat completion object 54. Then, the drawing unit 15 draws a drawing based on the Mermaid notation text 25.

[0018] Figure 2 is a hardware configuration diagram showing the drawing generation device 1. The drawing generation device 1 is equipped with a CPU (Central Processing Unit) 101, RAM (Random Access Memory) 102, and ROM (Read Only Memory) 103.

[0019] The CPU 101 is a processing unit that executes computer programs and provides overall control to the drawing generation device 1. The RAM 102 is a volatile read / write memory used by the CPU 101 as a workspace for computer programs. The ROM 103 is a non-volatile read-only memory that stores, for example, the BIOS (Basic I / O System).

[0020] The drawing generation device 1 further includes a display unit 104, an input unit 105, a communication unit 106, and a storage unit 107. The display unit 104 is, for example, a liquid crystal display, which displays characters, figures, images, etc. The input unit 105 is, for example, a mouse, keyboard, or touch panel, which the user inputs information into. The communication unit 106 is, for example, a network interface card, which communicates information with, for example, the generation AI system 3.

[0021] The memory unit 107 is a large-capacity storage device such as a hard disk or an SSD (Solid State Drive). The drawing generation program 1071 is stored in this memory unit 107. When the CPU 101 executes the drawing generation program 1071, each of the functional units in Figure 1 is realized.

[0022] Figure 3 shows the drawing generation screen 41 displayed on the display unit 104 by the drawing generation device 1. The drawing generation screen 41 consists of a drawing number combo box 411, a natural language text area 413, a Mermaid format text area 414, and a drawing area 415. This drawing generation screen 41 is a screen for generating a drawing by inputting natural language text and a drawing number.

[0023] The drawing number combo box 411 is a combo box for selecting a drawing number. The natural language text area 413 is an area for displaying entered natural language text. For example, a user can register natural language text in the natural language text area 413 by dragging and dropping a natural language text file into the natural language text area 413.

[0024] The Mermaid format text area 414 is an area that displays Mermaid format text generated from the drawing displayed in the natural language area 413. The text in the Mermaid format text area 414 is editable. This allows for manual correction of any errors in the Mermaid format text generated by the generation AI system 3, resulting in a correct drawing.

[0025] The drawing area 415 is an area that displays a drawing generated from the Mermaid format text displayed in the Mermaid format text area 414.

[0026] Figures 4A and 4B are flowcharts of the explanatory text generation process. First, the description identification unit 11 accepts natural language text and a drawing number as input (step S10). Then, the description identification unit 11 creates a prompt to identify the description text by specifying the natural language text and the drawing number (step S11). The description identification unit 11 sends the request body containing the generated prompt to the generation AI system 3 via API (step S12), and receives a chat completion object from the generation AI system 3 (step S13).

[0027] Next, the description extraction unit 12 extracts the description from the chat completion object (step S14). The Markdown notation conversion unit 13 determines whether or not there is a description of the type of drawing in the description (step S15). A description of the type of drawing includes, for example, flowcharts, mode transition diagrams, sequence diagrams, etc. If there is a description of the type of drawing in the description (Yes), proceed to step S16. If there is no description of the type of drawing in the description (No), proceed to step S17.

[0028] In step S16, the Markdown notation conversion unit 13 creates a prompt to convert the description and the drawing type specified in the description into Mermaid notation text, and then proceeds to step S18. In step S17, the Markdown notation conversion unit 13 creates a prompt to convert the description and the drawing type specified by the user into Mermaid notation text, and then proceeds to step S18.

[0029] In step S18, the Markdown notation conversion unit 13 sends the request containing the generated prompt to the generation AI system 3 via API. The Markdown notation conversion unit 13 then receives a chat completion object from the generation AI system 3 via API (step S17).

[0030] Next, the Markdown notation extraction unit 14 extracts Mermaid notation text from the chat completion object (step S20). Then, the drawing unit 15 draws the drawing based on the Mermaid notation text (step S21), and the process shown in Figure 4 is completed.

[0031] Figure 5 shows a request body 51 containing a prompt to be sent to the generating AI system 3. This request 51 consists of a model item and a messages item. The Model item specifies the ID of the generating AI model, followed by a semicolon. The Messages item stores a list of messages that make up the conversation so far, and various message types such as text, images, and audio can be set. The Messages item contains a role item and a content item in curly braces, followed by a semicolon.

[0032] For the role field, enter either system or user. For the contents field, enter text for the type field and enter prompt 31 from Figure 6 for the text field.

[0033] Figure 6 is a diagram illustrating the content of prompt 31. Prompt 31 contains the instruction text "Output the description for {drawing number} from natural language," and "{drawing number} = Figure 2." Following this, the target natural language is written after "#natural language." As a result, the generation AI system 3 outputs the description for Figure 2.

[0034] Figure 7 shows a chat completion object 52 received from the generation AI system 3. This chat completion object 52 consists of the following fields: id, object, created, model, choices, and usage.

[0035] The `id` field is an identifier that uniquely identifies the chat completion object 52. The `object` field indicates the object type, which is always `chat.completion`. The `created` field shows the Unix timestamp in seconds when the chat completion was created. The `model` field is the model used for the chat completion. The `choices` field is a list of choices for the chat completion, which may be multiple. The `usage` field is the usage statistics for the completion request.

[0036] The `choices` field consists of the `index` field, the `role` and `content` fields included in the `message` field, and the `finish_reason` field. The `index` field is the number of the option to complete the chat. The `role` field within the `message` field stores the `assistant`. The `content` field within the `message` field stores the response 32 generated by the AI ​​generation system.

[0037] Figure 8 is a diagram illustrating the content of the response 32 included in the chat completion object 52. This answer, number 32, contains a description of the drawing specified by the drawing number. This description is shown below. "Figure 2 illustrates the terminal's processing operation. First, the terminal processes the data. Next, the terminal sends it to the server. Then, the terminal determines whether the transmission was successful or not. If the transmission was successful, the terminal terminates processing. If the transmission was unsuccessful, the terminal processes the data again, and this process is repeated."

[0038] Figure 9 shows a request body 53 containing a prompt to be sent to the generating AI system 3. This request 53 consists of a model item and a messages item. The Model item specifies the ID of the generating AI model, followed by a semicolon. The Messages item stores a list of messages that make up the conversation so far, and various message types such as text, images, and audio can be set. The Messages item contains a role item and a content item in curly braces, followed by a semicolon.

[0039] For the role field, enter either system or user. For the contents field, enter text for the type field and enter prompt 33 from Figure 10 for the text field.

[0040] Figure 10 is a diagram illustrating the content of prompt 33. Prompt 33 contains the following instruction text on the first line: "The attached description explains {Drawing Type}. Remove the symbols from this description and then convert it to Mermaid notation text." The second line contains "{Drawing Type} = Flowchart." The third line contains "#Description," indicating the description. The description is written from the fourth line onward. The generating AI system 3 then converts the description from the fourth line onward into Mermaid notation text.

[0041] Figure 11 shows a chat completion object 54 received from the generation AI system 3. This chat completion object 54 consists of the following fields: id, object, created, model, choices, and usage.

[0042] The `id` field is an identifier that uniquely identifies the chat completion object 54. The `object` field indicates the object type, which is always `chat.completion`. The `created` field shows the Unix timestamp in seconds when the chat completion was created. The `model` field is the model used for the chat completion. The `choices` field is a list of choices for the chat completion, which may be multiple. The `usage` field is the usage statistics for the completion request.

[0043] The `choices` field consists of the `index` field, the `role` and `content` fields included in the `message` field, and the `finish_reason` field. The `index` field is the number of the option to complete the chat. The `role` field within the `message` field stores the `assistant`. The `content` field within the `message` field stores the response 34 generated by the AI ​​generation system.

[0044] Figure 12 is a diagram illustrating the content of the response included in the chat completion object 54. Answer 34 contains Mermaid notation text. This Mermaid notation text is a converted version of the drawing's description. The Mermaid notation text is transcribed below. ```mermaid flowchart TD A [Data processing] --> B [Send to server] B --> C {Was the transmission successful?} C -->|Success| D[Processing complete] C -->|Failure|A ```

[0045] Figure 13 is an example of a diagram drawn based on Mermaid notation text. This diagram shows the "data processing," "sending to server" processes, and the determination of whether the transmission was successful or not. Initially, the terminal performs data processing and sends the data to the server. Then, it determines whether the transmission was successful or not. If the transmission is successful (success), the process ends. If the transmission fails, it returns to data processing and repeats.

[0046] Figure 14 is a logic diagram showing the drawing generation device 1A according to the second embodiment. The drawing generation device 1A comprises a rule-based explanatory text extraction unit 16, a Markdown notation conversion unit 13, a Markdown notation extraction unit 14, and a drawing drawing unit 15. The drawing generation device 1 works in cooperation with the generation AI system 3 to generate a drawing 26 specified by drawing number 22 from natural language text 21.

[0047] When the rule-based description extraction unit 16 receives natural language text 21 and drawing number 22 as input, it uses rule-based processing to identify the description text 23 for the drawing specified by drawing number 22. The rule-based description extraction unit 16 searches the natural language text 21 for drawing number 22 and identifies the sentences from the sentence in which drawing number 22 is mentioned up to the sentence immediately preceding the sentence in which another drawing number is mentioned as the description text 23.

[0048] When the Markdown notation conversion unit 13 receives the description text 23 and the drawing type 24 as input, it instructs the generation AI system 3 via a request body 53, which includes a second prompt, to convert the description text 23 into Mermaid notation text 25. The drawing type 24 includes flowcharts, sequence diagrams, mode transition diagrams, etc.

[0049] The request 53 is written in JSON format. This request 53 is executed by the generation AI system 3, for example, by calling the API of the generation AI system 3, and a chat completion object 54 is generated. The generation AI system 3 then returns the chat completion object 54 in JSON format to the Markdown notation conversion unit 13. This chat completion object 54 contains Mermaid notation text 25.

[0050] The Markdown notation extraction unit 14 extracts Mermaid notation text 25 from the chat completion object 54. Then, the drawing unit 15 draws a drawing based on the Mermaid notation text 25.

[0051] Figure 15 is a flowchart of the explanatory text generation process. First, the rule-based description extraction unit 16 accepts natural language text and a drawing number as input (step S30). Then, the rule-based description extraction unit 16 identifies the description text by specifying the natural language text and the drawing number (step S31).

[0052] The Markdown notation conversion unit 13 determines whether or not there is a description of the type of drawing in the description (step S32). A description of the type of drawing includes, for example, a flowchart, a mode transition diagram, or a sequence diagram. If there is a description of the type of drawing in the description (Yes), proceed to step S33. If there is no description of the type of drawing in the description (No), proceed to step S34.

[0053] In step S33, the Markdown notation conversion unit 13 creates a prompt to convert the description and the drawing type specified in the description into Mermaid notation text, and then proceeds to step S35. In step S34, the Markdown notation conversion unit 13 creates a prompt to convert the description and the drawing type specified by the user into Mermaid notation text, and then proceeds to step S35.

[0054] In step S35, the Markdown notation conversion unit 13 sends the request containing the generated prompt to the generation AI system 3 via API. The Markdown notation conversion unit 13 then receives a chat completion object from the generation AI system 3 via API (step S36).

[0055] Next, the Markdown notation extraction unit 14 extracts Mermaid notation text from the chat completion object (step S37). Then, the drawing unit 15 draws the drawing based on the Mermaid notation text (step S38), and the process of Figure 15 is completed.

[0056] Figure 16 is a Mermaid-format text generated from the explanation of the mode transition diagram. The following is a transcription of this Mermaid-formatted text. stateDiagram [*] --> Battery operation Battery operation --> Main power supply operation: Power ON Main power supply operation --> Battery operation: Mating error First main power supply operation --> Second main power supply operation: for a certain period of time Second main power supply operation --> Battery operation: Power OFF

[0057] Figure 17 is an example of a mode transition diagram. The device starts in battery operation mode. When the power is turned on, the device transitions to the first main power operation mode. If an attendance error occurs while in the first main power operation mode, the device returns to battery operation mode. After a certain period of time, the device transitions from the first main power operation mode to the second main power operation mode. If the power is turned off while in the second main power operation mode, the device returns to battery operation mode.

[0058] Figure 18 is Mermaid-formatted text generated from the sequence diagram's explanatory text. The following is a transcription of this Mermaid-formatted text. sequenceDiagram participant user terminal Application Server Subject Processing Result Cache DB User terminal ->> Server: Processing request Server ->> Processing Result Cache DB: Processing Request Processing result cache DB -->> Server: Cache not found Server ->> Server: Processing Execution Server ->> Processing Result Cache DB: Processing Result Server -->> User Terminal: Processing Result Processing result cache DB ->> Processing result cache DB: Record

[0059] Figure 19 is an example of a sequence diagram. The sequence diagram according to the present invention illustrates a series of steps in which a server responds to a processing request from a user terminal and stores the processing result in a cache database. The details of each step are described below.

[0060] First, the user terminal sends a processing request to the server. Upon receiving this processing request, the server sends a processing request to the cache database to check if the corresponding processing result exists in the cache.

[0061] Next, if the processing result cache database does not exist in the relevant cache, it returns a "cache not found" response to the server. Upon receiving this response, the server executes the processing required for the request and generates the processing result.

[0062] Once processing is complete, the server saves the generated processing results to a processing result cache database, allowing it to quickly respond to similar future requests using the cache. The server then returns the processing results to the user's terminal, allowing the user to verify the results.

[0063] Finally, the processing result cache database stores the processing results received from the server as a record. This reduces response time to future processing requests and improves system efficiency. This invention, by utilizing a caching function, achieves a reduction in processing time for similar requests, resulting in improved overall system performance.

[0064] (modified version) The present invention is not limited to the embodiments described above, and can be modified without departing from the spirit of the invention, for example, (a) to (d) below.

[0065] (a) The text converted by the generating AI is not limited to Mermaid notation text, but can be any Markdown notation that uniquely represents the nodes and topology of the drawing. (b) The generation AI system is not limited to a cloud environment; it may also operate in an on-premises environment. (c) The descriptions of the drawings are not limited to those for patent specifications, but may also be descriptions for software specifications or user manuals. (d) The generative AI systems that send each prompt may be the same or different, and are not limited to such systems. [Explanation of Symbols]

[0066] 1. Drawing generation device 11. Description section 16 Rule-based explanatory text extraction unit 12. Description Extraction Unit 13 Markdown Notation Conversion Section 14. Markdown notation extraction section 15 Drawing Section 3. Generative AI System 21 Natural sentences 22 Drawing number 26 Drawings 51 Request body 52 Chat Completed Objects 23 Description 24 Types of Drawings 53 Request body 25 Mermaid Notation Text 54 Chat Completed Objects 101 CPU 102 RAM 103 ROM 104 Display section 105 Input section 106 Communications Department 107 Storage section 1071 Drawing Generation Program 41 Drawing generation screen 411 Drawing Number Combo Box 413 Natural sentence area 414 Mermaid format text area 415 Drawing Area 31 Prompt 32 answers 33 Prompts 34 answers 1A Drawing generation device

Claims

1. On the computer, Procedure for extracting descriptive text about a specified drawing from input natural language text. A procedure for inputting a first prompt to a generation AI that includes an instruction to generate a predetermined Markdown notation text from the description text, and obtaining a response containing the predetermined Markdown notation text from the generation AI. A procedure for drawing a diagram from the aforementioned predetermined Markdown notation text, A drawing generation program to implement this.

2. The type of drawing mentioned above is one of the following: a flowchart, a sequence diagram, or a mode transition diagram. The drawing generation program according to claim 1.

3. The first prompt includes information indicating the type of drawing. The drawing generation program according to claim 1.

4. The first prompt includes an instruction to remove symbols from the description and an instruction to generate a predetermined Markdown notation text from the description from which the symbols have been removed. The drawing generation program according to claim 1.

5. A description extraction unit that extracts a description of a specified drawing from the input natural language text, A Markdown notation conversion unit inputs a prompt to a generating AI that includes an instruction to generate a predetermined Markdown notation text from the description text of the drawing, and obtains a response containing the predetermined Markdown notation text from the generating AI. A drawing unit that draws a drawing from the predetermined Markdown notation text, A drawing generation device characterized by comprising the following features.

6. The explanatory text extraction unit inputs a second prompt to the generating AI, which includes the input natural language text and an instruction to extract an explanatory text for a specified drawing from the text, and extracts an explanatory text related to the drawing. The drawing generation apparatus according to feature 5.

7. The aforementioned descriptive text extraction unit extracts descriptive text for the drawing specified by the drawing number from the input natural language text using a rule-based method. The drawing generation apparatus according to feature 5.

Images