Drawing generation program and drawing generation device

Abstract

From the description of a drawing, a drawing relating to the description is generated. [Solution] The drawing generation device 1 is characterized by comprising: a description extraction unit 12 that extracts a description of the specified drawing from input natural language text; a Markdown notation conversion unit 13 that inputs a prompt including an instruction to generate Mermaid notation text from the drawing description to a generation AI system 3 and obtains an answer including Mermaid notation text from the generation AI system 3; and a drawing drawing unit 15 that draws a drawing from this Mermaid notation text.

Description

[Technical Field]

[0001] The present invention relates to a drawing generation program and a drawing generation device. [Background technology]

[0002] With the recent development of AI technology, it has been proposed to support the generation of various content with the assistance of generative AI. For example, Patent Document 1 describes an invention that reduces the variation in quality when drafting claims or reduces the time required to prepare a patent application manuscript including claims. In this way, many technologies are being implemented in which humans create text with the assistance of generative AI.

[0003] A technology for creating diagrams with the assistance of generative AI is described in Non-Patent Document 1. In Non-Patent Document 1, the requirements for the diagram to be created are conveyed to ChatGPT (GPT-4) to output Mermaid syntax, and attempts are made to create various diagrams such as sequences, Gantt charts, and flowcharts. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-095653 [Non-patent literature]

[0005] [Non-Patent Document 1] Ryuta Wakatsuki, "I tried to output Mermaid syntax with ChatGPT (practical examples of creating sequence diagrams, Gantt charts, and flowcharts)," [online], https: / / dev.classmethod.jp / articles / chatgpt-mermaid-diagram / , search date [searched November 10, 2024], source (Internet:<URL:https: / / dev.classmethod.jp / articles / chatgpt-mermaid-diagram / > ) Summary of the Invention [Problem to be solved by the invention]

[0006] In Non-Patent Document 1, a Mermaid text is generated from a bulleted list of requirements. However, in reality, the description of a drawing is often written in natural language. Also, the description of a drawing may not specify the type of drawing. In this case, the type of drawing may not be correctly recognized, and the Mermaid text may not be generated appropriately.

[0007] Additionally, generative AI such as OpenAI's ChatGPT (registered trademark) and GOOGLE's Gemini have the ability to accept text prompts and generate images according to the prompts. However, this image generation function cannot draw flowcharts, mode transition diagrams, or sequence diagrams. Therefore, an object of the present invention is to generate a drawing related to a description from the description of the drawing. [Means for solving the problem]

[0008] That is, the above-mentioned problems of the present invention are solved by the following configuration. The drawing generation program of the present invention includes a procedure for extracting explanatory text about a designated drawing from input natural language text in a computer, and a generation AI for: instructions for removing symbols from the description; and instructions for generating predetermined Markdown notation text from the description with the symbols removed. and obtaining an answer including a predetermined Markdown notation text from the generation AI; and drawing a drawing from the predetermined Markdown notation text.

[0009] The drawing generation device of the present invention includes a description extraction unit that extracts a description of a designated drawing from an input natural language text, and a generation AI that extracts a description of the designated drawing from the description of the drawing. Instructions to remove the symbols and the explanation from the above with the symbols removedThe system is characterized by comprising a Markdown notation conversion unit that inputs a prompt including an instruction to generate a predetermined Markdown notation text and obtains an answer including the predetermined Markdown notation text from the generation AI, and a drawing drawing unit that draws a drawing from the predetermined Markdown notation text. The other configurations will be explained in the embodiments. [Effects of the Invention]

[0010] According to the present invention, it is possible to generate a drawing relating to a description from the description of the drawing. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a logical configuration diagram showing a drawing generation device according to a first embodiment. [Figure 2] FIG. 1 is a hardware configuration diagram illustrating a drawing generating device. [Figure 3] 1 is a screen displayed by a drawing generating device. [Figure 4A] 10 is a flowchart of an explanation generation process. [Figure 4B] 10 is a flowchart of an explanation generation process. [Figure 5] FIG. 10 is a diagram showing a request body to be sent to the generation AI system. [Figure 6] FIG. 10 is a diagram illustrating the contents of a prompt. [Figure 7] A diagram showing a chat completion object received from the generation AI system. [Figure 8] FIG. 10 is a diagram illustrating the content of a reply included in a chat completion object. [Figure 9] FIG. 10 is a diagram showing a request body to be sent to the generation AI system. [Figure 10] FIG. 10 is a diagram illustrating the contents of a prompt. [Figure 11] A diagram showing a chat completion object received from the generation AI system. [Figure 12] FIG. 10 is a diagram illustrating the content of a reply included in a chat completion object. [Figure 13] This is a flowchart based on Mermaid format text. [Figure 14] FIG. 10 is a logical configuration diagram showing a drawing generating device according to a second embodiment. [Figure 15] 10 is a flowchart of a drawing generation process. [Figure 16] This is Mermaid format text generated from the description of the mode transition diagram. [Figure 17] 1 is an example of a mode transition diagram. [Figure 18] This is Mermaid format text generated from the description of the sequence diagram. [Figure 19] 1 is an example of a sequence diagram. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is a logical configuration diagram showing a drawing generating device 1 according to the first embodiment. The drawing generation device 1 includes an explanatory text identification unit 11, an explanatory text extraction unit 12, a Markdown notation conversion unit 13, a Markdown notation extraction unit 14, and a drawing drawing unit 15. The drawing generation device 1 is a device that cooperates with a generation AI system 3 to generate a drawing 26 specified by a drawing number 22 from natural language 21.

[0013] When the natural language sentence 21 and the drawing number 22 are input, the explanatory sentence identification unit 11 instructs the generative AI system 3 to identify explanatory sentences from the natural language sentence 21 by a request body 51 including a first prompt. The natural language sentence 21 includes explanatory sentences for the drawing specified by the drawing number 22. The drawing may be a flowchart, a sequence diagram, a mode transition diagram, or the like.

[0014] This request body 51 is written in JSON format. The request body 51 is executed by the generation AI system 3, for example, by calling the API (Application Programming Interface) of the generation AI system 3, and a chat completion object 52 is generated. Then, the chat completion object 52 in JSON format is sent from the generation AI system 3 to the explanation specification unit 11. This chat completion object 52 includes the explanation 23 of the drawing specified by the drawing number 22.

[0015] The description extracting unit 12 extracts the description 23 of the drawing designated by the drawing number 22 from the chat completion object 52 . When the Markdown notation conversion unit 13 receives the explanatory text 23 and the drawing type 24, it instructs the generative AI system 3 to convert the explanatory text 23 into Mermaid notation text 25 using a request body 53 including a second prompt. The drawing type 24 is a flowchart, a sequence diagram, a mode transition diagram, etc.

[0016] The request body 53 is written in JSON format. This request body 53 is executed by the generation AI system 3, for example, by calling the API of the generation AI system 3, and a chat completion object 54 is generated. The generation AI system 3 then returns the chat completion object 54 in JSON format to the Markdown notation conversion unit 13. This chat completion object 54 includes Mermaid notation text 25.

[0017] The Markdown notation extraction unit 14 extracts the Mermaid notation text 25 from the chat completion object 54. Then, the drawing drawing unit 15 draws a drawing based on the Mermaid notation text 25.

[0018] FIG. 2 is a hardware configuration diagram showing the drawing generation device 1. As shown in FIG. The drawing generating device 1 includes a CPU (Central Processing Unit) 101, a RAM (Random Access Memory) 102, and a ROM (Read Only Memory) 103.

[0019] The CPU 101 is a processing device that executes computer programs and controls the drawing generation device 1. The RAM 102 is a volatile readable / writable memory that is used as a work area for the computer programs by the CPU 101. The ROM 103 is a non-volatile readable-only memory that stores, for example, a BIOS (Basic I / O System).

[0020] The drawing generation device 1 further includes a display unit 104, an input unit 105, a communication unit 106, and a storage unit 107. The display unit 104 is, for example, a liquid crystal display, and displays characters, figures, images, etc. The input unit 105 is, for example, a mouse, keyboard, or touch panel, and is used by the user to input information. The communication unit 106 is, for example, a network interface card, and communicates information with, for example, the generation AI system 3.

[0021] The storage unit 107 is a large-capacity storage device such as a hard disk or a solid-state drive (SSD). A drawing generation program 1071 is stored in the storage unit 107. When the CPU 101 executes the drawing generation program 1071, the respective functional units in FIG. 1 are realized.

[0022] FIG. 3 shows a drawing generation screen 41 displayed on the display unit 104 by the drawing generation device 1. As shown in FIG. The drawing generation screen 41 includes a drawing number combo box 411, a natural language area 413, a Mermaid format text area 414, and a drawing area 415. This drawing generation screen 41 is a screen for generating a drawing by inputting natural language and a drawing number.

[0023] The drawing number combo box 411 is a combo box for selecting a drawing number. The natural sentence area 413 is an area for displaying the input natural sentence. For example, the user can drag and drop a natural sentence file into the natural sentence area 413, whereby the natural sentence is registered in the natural sentence area 413.

[0024] The Mermaid text area 414 is an area that displays Mermaid text generated from the drawing displayed in the natural language area 413. The Mermaid text area 414 displays text that is editable. This allows the user to manually correct the Mermaid text generated by the generation AI system 3 to obtain a correct drawing, even if there is an error in the Mermaid text.

[0025] The drawing area 415 is an area for displaying a drawing generated from the Mermaid format text displayed in the Mermaid format text area 414 .

[0026] 4A and 4B are flowcharts of the explanation generation process. First, the explanatory text identification unit 11 accepts input of natural language and drawing number (step S10). Then, the explanatory text identification unit 11 creates a prompt that specifies the natural language and drawing number to identify the explanatory text (step S11). The explanatory text identification unit 11 sends a request body including the generated prompt to the generation AI system 3 via API (step S12), and receives a chat completion object from the generation AI system 3 (step S13).

[0027] Next, the explanation extraction unit 12 extracts an explanation from the chat completion object (step S14). The Markdown notation conversion unit 13 determines whether or not the description contains a description of the drawing type (step S15). The description of the drawing type is, for example, a description of a flowchart, a mode transition diagram, a sequence diagram, etc. If the description contains a description of the drawing type (Yes), proceed to step S16. If the description does not contain a description of the drawing type (No), proceed to step S17.

[0028] In step S16, the Markdown notation conversion unit 13 creates a prompt for converting the explanatory text and the drawing type described in the explanatory text into Mermaid notation text, and then proceeds to step S 18. In step S17, the Markdown notation conversion unit 13 creates a prompt for converting the explanatory text and the drawing type specified by the user into Mermaid notation text, and then proceeds to step S18.

[0029] In step S18, the Markdown notation conversion unit 13 sends a request body including the generated prompt to the generation AI system 3 via the API. Then, the Markdown notation conversion unit 13 receives a chat completion object from the generation AI system 3 via the API (step S17).

[0030] Next, the Markdown notation extraction unit 14 extracts Mermaid notation text from the chat completion object (step S20). Then, the drawing drawing unit 15 draws a drawing based on the Mermaid notation text (step S21), and the processing of FIG. 4 ends.

[0031] FIG. 5 shows a request body 51 including a prompt to be sent to the generation AI system 3. As shown in FIG. This request body 51 is composed of a model item and a messages item. In the Model item, the ID of the generation AI model is specified after a semicolon. The Messages item stores a list of messages that make up the conversation so far, and various message types such as text, image, and audio can be set. In the Messages item, the role item and content item are written in curly brackets after a semicolon.

[0032] In the role field, enter system or user. In the contents field, enter text in the type field and enter the prompt 31 in Figure 6 in the text field.

[0033] FIG. 6 is a diagram for explaining the contents of the prompt 31. As shown in FIG. The prompt 31 contains the instruction text "Please output the description of {drawing number} from natural language." and "{drawing number} = Figure 2." The target natural language is then written after "#natural language." This causes the generative AI system 3 to output the description of Figure 2.

[0034] FIG. 7 is a diagram showing the chat completion object 52 received from the generation AI system 3. The chat completion object 52 includes an id item, an object item, a created item, a model item, a choices item, and a usage item.

[0035] The id field is an identifier that uniquely identifies the chat completion object 52. The object field indicates the object type, which is always chat.completion. The created field is the Unix timestamp in seconds when the chat completion was created. The model field is the model used for the chat completion. The choices field is a list of options for the chat completion, which may be multiple. The usage field is usage statistics for the completion request.

[0036] The choices item is composed of an index item, a role item and a content item contained in a message item, and a finish_reason item. The index field is the number of the option for completing the chat. The role field included in the message field stores assistant. The content field included in the message field stores the answer 32 generated by the AI ​​generation system.

[0037] FIG. 8 is a diagram illustrating the content of the answer 32 included in the chat completion object 52. As shown in FIG. This reply 32 contains a description of the drawing designated by the drawing number. This description is shown below. "Figure 2 is a diagram showing the processing operation of a terminal. First, the terminal processes data. Next, the terminal transmits the data to the server. Then, the terminal determines whether the transmission was successful or not. If the transmission was successful, the terminal ends the processing. If the transmission was not successful, the terminal processes the data again, and this process is repeated."

[0038] FIG. 9 shows a request body 53 including a prompt to be sent to the generation AI system 3. As shown in FIG. This request body 53 is composed of a model item and a messages item. In the Model item, the ID of the generation AI model is specified after a semicolon. In the Messages item, a list of messages that make up the conversation so far is stored, and various message types such as text, image, and audio can be set. In the Messages item, the role item and content item are written in curly brackets after a semicolon.

[0039] In the role field, enter system or user. In the contents field, enter text in the type field and enter the prompt 33 in Fig. 10 in the text field.

[0040] FIG. 10 is a diagram for explaining the contents of the prompt 33. As shown in FIG. The prompt 33 has the following instruction text on the first line: "The attached description explains {Drawing type}. Please remove symbols from this description and convert it to Mermaid notation text." The second line states "{Drawing type} = Flowchart." The third line states "#description," which indicates the description. The descriptions are written from the fourth line onwards. As a result, the generation AI system 3 converts the descriptions from the fourth line onwards into Mermaid notation text.

[0041] FIG. 11 is a diagram showing the chat completion object 54 received from the generation AI system 3. The chat completion object 54 includes an id item, an object item, a created item, a model item, a choices item, and a usage item.

[0042] The id field is an identifier that uniquely identifies the chat completion object 54. The object field indicates the object type, which is always chat.completion. The created field is the Unix timestamp in seconds when the chat completion was created. The model field is the model used for the chat completion. The choices field is a list of options for the chat completion, which may be multiple. The usage field is usage statistics for the completion request.

[0043] The choices item is composed of an index item, a role item and a content item contained in a message item, and a finish_reason item. The index field is the number of the option for completing the chat. The role field included in the message field stores assistant. The content field included in the message field stores the answer 34 generated by the AI ​​generation system.

[0044] FIG. 12 is a diagram illustrating the content of the reply included in the chat completion object 54. As shown in FIG. This answer 34 contains Mermaid text. This Mermaid text is a converted version of the drawing description. The Mermaid text is transcribed below. mermaid flowchart TD A [Data processing] --> B [Send to server] B --> C{Successful?} C --> | Success | D [Processing completed] C -->|Failure|A ```

[0045] FIG. 13 is an example of a drawing drawn based on Mermaid text. This diagram shows "data processing," "send to server" processing, and determining whether the transmission was successful. First, the terminal performs data processing and sends it to the server. It then determines whether the transmission was successful. If the transmission was successful (success), the processing ends. If the transmission failed, it returns to data processing and repeats.

[0046] FIG. 14 is a logical configuration diagram showing a drawing generating device 1A according to the second embodiment. The drawing generation device 1A includes a rule-based description extraction unit 16, a Markdown notation conversion unit 13, a Markdown notation extraction unit 14, and a drawing drawing unit 15. The drawing generation device 1 cooperates with a generation AI system 3 to generate a drawing 26 specified by a drawing number 22 from natural language 21.

[0047] When natural language 21 and drawing number 22 are input, rule-based description extraction unit 16 uses rule-based processing to identify description 23 of the drawing specified by drawing number 22. Rule-based description extraction unit 16 searches natural language 21 for drawing number 22 and identifies, as description 23, the sentence from the sentence in which drawing number 22 is written to the sentence immediately before the sentence in which another drawing number is written.

[0048] When the Markdown notation conversion unit 13 receives the explanatory text 23 and the drawing type 24, it instructs the generative AI system 3 to convert the explanatory text 23 into Mermaid notation text 25 using a request body 53 including a second prompt. The drawing type 24 is a flowchart, a sequence diagram, a mode transition diagram, etc.

[0049] The request body 53 is written in JSON format. This request body 53 is executed by the generation AI system 3, for example, by calling the API of the generation AI system 3, and a chat completion object 54 is generated. The generation AI system 3 then returns the chat completion object 54 in JSON format to the Markdown notation conversion unit 13. This chat completion object 54 includes Mermaid notation text 25.

[0050] The Markdown notation extraction unit 14 extracts the Mermaid notation text 25 from the chat completion object 54. Then, the drawing drawing unit 15 draws a drawing based on the Mermaid notation text 25.

[0051] FIG. 15 is a flowchart of the explanation generation process. First, the rule-based explanation extraction unit 16 receives input of natural language and drawing number (step S30), and then specifies the explanation by specifying the natural language and drawing number (step S31).

[0052] The Markdown notation conversion unit 13 determines whether or not the description contains a description of the drawing type (step S32). The description of the drawing type is, for example, a description of a flowchart, a mode transition diagram, a sequence diagram, etc. If the description contains a description of the drawing type (Yes), proceed to step S33. If the description does not contain a description of the drawing type (No), proceed to step S34.

[0053] In step S33, the Markdown notation conversion unit 13 creates a prompt for converting the explanatory text and the drawing type described in the explanatory text into Mermaid notation text, and then proceeds to step S35. In step S34, the Markdown notation conversion unit 13 creates a prompt for converting the explanatory text and the drawing type specified by the user into Mermaid notation text, and then proceeds to step S35.

[0054] In step S35, the Markdown notation conversion unit 13 sends a request body including the generated prompt to the generation AI system 3 via the API. Then, the Markdown notation conversion unit 13 receives a chat completion object from the generation AI system 3 via the API (step S36).

[0055] Next, the Markdown notation extraction unit 14 extracts Mermaid notation text from the chat completion object (step S37). Then, the drawing drawing unit 15 draws a drawing based on the Mermaid notation text (step S38), and the processing of FIG. 15 ends.

[0056] FIG. 16 shows Mermaid format text generated from the explanation of the mode transition diagram. Below is a transcription of this Mermaid-formatted text. stateDiagram [*] --> Battery operation Battery operation --> 1st main power supply operation: Power ON 1st main power supply operation --> Battery operation: Mating error First main power supply operation --> Second main power supply operation: for a certain period Secondary main power supply operation --> Battery operation: Power OFF

[0057] FIG. 17 is an example of a mode transition diagram. The device starts in battery operation mode. When the power is turned on, the device transitions to the first main power operation mode. If an attendance error occurs while in the first main power operation mode, the device returns to battery operation mode. After a certain period of time has passed, the device transitions from the first main power operation mode to the second main power operation mode. If the power is turned off while in the second main power operation mode, the device returns to battery operation mode.

[0058] Figure 18 shows Mermaid format text generated from the explanatory text of the sequence diagram. Below is a transcription of this Mermaid-formatted text. sequenceDiagram participant user terminal Participant Server Participant processing result cache DB User terminal ->> Server: Processing request Server ->> Processing result cache DB: Processing request Processing result cache DB -->> Server: Cache not found Server ->> Server: Process execution Server ->> Processing result cache DB: Processing results Server -->> User device: Processing result Processing result cache DB ->> Processing result cache DB: Record

[0059] FIG. 19 is an example of a sequence diagram. The sequence diagram according to the present invention shows a series of steps in which a server responds to a processing request from a user terminal and stores the processing results in a cache database. Each step is described in detail below.

[0060] First, the user terminal sends a processing request to the server. Upon receiving this processing request, the server sends the processing request to the cache database to check whether the corresponding processing result exists in the cache.

[0061] Next, if the processing result cache database does not have the corresponding cache, it returns a "cache not found" response to the server. Upon receiving this response, the server executes the processing according to the request and generates the processing result.

[0062] Once the processing is complete, the server stores the generated processing results in a processing result cache database so that future similar requests can be responded to quickly using the cache.The server then returns the processing results to the user's terminal so that the user can check the results.

[0063] Finally, the processing results cache database stores a record of processing results received from the server, thereby improving response times for future processing requests and increasing system efficiency. According to the present invention, by using a cache function, the processing time for similar requests can be reduced, and the performance of the entire system can be improved.

[0064] (Variation) The present invention is not limited to the above-described embodiment, and modifications can be made without departing from the spirit of the present invention, for example, the following (a) to (d).

[0065] (a) The text into which the generation AI converts the description is not limited to Mermaid notation text, but can be any Markdown notation that uniquely indicates the nodes and topology of the drawing. (b) The generative AI system is not limited to one that operates in a cloud environment, but may also operate in an on-premise environment. (c) The description of the drawings is not limited to that of a patent specification, but may be that of a software specification or a user manual. (d) The generating AI systems that send each prompt may be the same or different, and are not limited thereto. [Explanation of symbols]

[0066] 1. Drawing generation device 11 Description specification section 16 Rule-based description extraction unit 12 Description extraction section 13 Markdown conversion section 14 Markdown syntax extractor 15 Drawing Department 3. Generative AI System 21 Natural sentences 22 Drawing number 26 Drawings 51 Request body 52 Chat Completed Object 23 Description 24 Drawing Types 53 Request body 25 Mermaid text 54 Chat Completed Object 101 CPU 102 RAM 103 ROM 104 Display section 105 Input section 106 Communications Department 107 Storage section 1071 Drawing Generation Program 41 Drawing generation screen 411 Drawing Number Combo Box 413 Natural sentence area 414 Mermaid-style text fields 415 Drawing Area 31 prompts 32 answers 33 prompts 34 answers 1A Drawing generation device

Claims

1. On the computer, A step of extracting a description of a designated drawing from the input natural language text; a step of inputting a first prompt to a generating AI, the first prompt including an instruction to remove symbols from the explanatory text and an instruction to generate a predetermined Markdown notation text from the explanatory text with the symbols removed, and obtaining an answer including the predetermined Markdown notation text from the generating AI; drawing a drawing from said predetermined Markdown notation text; A drawing generation program for carrying out the above.

2. The type of the drawing is one of a flowchart, a sequence diagram, and a mode transition diagram. The drawing generation program according to claim 1 .

3. the first prompt includes information indicating the type of the drawing; The drawing generation program according to claim 1 .

4. a description extraction unit that extracts a description of the designated drawing from the input natural language text; a Markdown notation conversion unit that inputs a prompt including an instruction to remove symbols from the description of the drawing and an instruction to generate a predetermined Markdown notation text from the description with the symbols removed to the generation AI, and obtains an answer including the predetermined Markdown notation text from the generation AI; a drawing drawing unit that draws a drawing from the predetermined Markdown notation text; A drawing generation device comprising:

5. The description extraction unit inputs a second prompt including the input natural language text and an instruction to extract a description of the designated drawing from the text to the generation AI, thereby extracting the description related to the drawing.

5. The drawing generating device according to claim 4.

6. the description extraction unit extracts, from the input natural language text, a description of the drawing specified by the drawing number on a rule-based basis; 5. The drawing generating device according to claim 4.

Images