Information processing device, control method, and program

Abstract

It provides suitable support for the preparation of documents related to patent applications. [Solution] The information processing device includes a first drawing description model which includes a vision encoder that converts an input drawing into an image feature vector, and a drawing description text encoder that describes the drawing and converts drawing description text written in a descriptive modeling language into a drawing description text feature vector, and a processing unit which causes the first drawing description model to learn such that the drawing description text feature vector and the drawing description text feature vector are similar when the drawing description text is drawing description text that describes an input drawing.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, a control method, and a program.

Background Art

[0002] As a technology for assisting in the creation of documents related to patent applications, such as patent specifications, a technology for efficiently enabling a user to grasp the content of sentences has been disclosed (see, for example, Patent Document 1). In addition, research has been conducted on using a multimodal model for patent drawings (see, for example, Non-Patent Document 1).

Prior Art Documents

Patent Documents

[0003] [[ID=X]] [[ID=X]]

Patent Document 1

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Non-limiting embodiments of the present disclosure contribute to providing an information processing apparatus, a control method, and a program that suitably assist in the creation of documents related to patent applications.

Means for Solving the Problems

[0006] An information processing device according to one embodiment of the present disclosure includes a first drawing description model that includes a vision encoder that converts an input drawing into an image feature vector, and a drawing description text encoder that describes the drawing and converts drawing description text written in a descriptive modeling language into a drawing description text feature vector, and a processing unit that causes the first drawing description model to learn such that the drawing description text feature vector and the drawing description text feature vector are similar when the drawing description text is drawing description text that describes the input drawing.

[0007] A control method for an information processing device according to one embodiment of the present disclosure includes a vision encoder that converts an input drawing into an image feature vector, and a drawing description text encoder that describes the drawing and converts drawing description text written in a descriptive modeling language into a drawing description text feature vector. The first drawing description model is trained so that when the drawing description text is drawing description text that describes the input drawing, the drawing description text feature vector and the drawing description text feature vector are similar.

[0008] A program according to one embodiment of the present disclosure causes a computer constituting an information processing device to learn a first drawing description model, which includes a vision encoder that converts an input drawing into an image feature vector, and a drawing description text encoder that describes the drawing and converts drawing description text written in a descriptive modeling language into a drawing description text feature vector, so that when the drawing description text is drawing description text that describes the input drawing, the drawing description text feature vector and the drawing description text feature vector become similar.

[0009] These comprehensive or specific embodiments may be implemented as systems, devices, methods, integrated circuits, computer programs, or recording media, or as any combination of systems, devices, methods, integrated circuits, computer programs, and recording media. [Effects of the Invention]

[0010] Non-limiting embodiments of this disclosure suitably assist in the preparation of documents relating to a patent application.

[0011] Further advantages and effects of one embodiment of this disclosure will be made apparent from the specification and drawings. Such advantages and / or effects are provided by several embodiments and features described in the specification and drawings, but not all of them are necessarily provided in order to obtain one or more identical features. [Brief explanation of the drawing]

[0012] [Figure 1] Figure 1 shows an example of a multimodal model configuration. [Figure 2] A diagram illustrating an example of the correspondence between descriptive languages ​​and various technical fields of patents. [Figure 3] A diagram demonstrating that drawings and drawing description text can be converted. [Figure 4] Diagram showing an example of the configuration of an information processing device during learning. [Figure 5] This figure shows an example configuration of an information processing device that generates drawing description text from drawings. [Figure 6] A diagram demonstrating that drawings and drawing description text can be converted. [Figure 7] Diagram showing an example of the configuration of an information processing device during learning. [Figure 8] This diagram shows an example configuration of an information processing device that generates explanatory text for drawings from drawings. [Figure 9] This diagram shows an example configuration of an information processing device that generates drawings from drawing description text. [Figure 10] This figure shows an example configuration combining the first drawing description model, the drawing unit, and the second drawing description model. [Figure 11] This figure shows an example of drawing description text written in Mermaid notation. [Figure 12] Diagram showing an example drawing. [Figure 13] Figure showing an example of drawing description text. [Figure 14] Flowchart showing the processing flow of the drawing unit. [Figure 15] Figure showing an example of the shape of a lead line [Figure 16] Figure showing an example of a drawing [Figure 17] Figure showing a configuration example of a drawing drawing section [Figure 18] Figure showing a configuration with an expert switching section added [Figure 19] Figure showing the first combined feature vector generation device [Figure 20] Figure showing the second combined feature vector generation device [Figure 21] Block diagram showing the basic configuration related to the registration and search of combined feature vectors [Figure 22] Figure showing that the claims and claim support disclosure can be converted [Figure 23] Figure showing a configuration example of an information processing device during learning [Figure 24A] Flowchart showing the processing when BOS is detected [Figure 24B] Flowchart showing the processing when BOF is detected [Figure 24C] Flowchart showing the processing when BOI is detected [Figure 24D] Flowchart showing the processing when REF is detected [Figure 25] Figure showing a configuration example of an information processing device that generates claim support disclosure from claims [Figure 26] Figure showing a configuration example of an information processing device that generates claims from claim support disclosure [Figure 27] Figure showing the third combined feature vector generation device [Figure 28] Figure showing the configuration of a similar location display device [Figure 29] Figure showing an example of the display of claim support disclosure [Figure 30] Figure showing an example of a prompt [Figure 31] Flowchart showing the flow of the exclusion process for only "postscript" [Figure 32] Flowchart showing the flow of the "postscript" proposal process [Figure 33]A flowchart illustrating the process of training a model to generate "notes." [Figure 34] A flowchart illustrating the process of expanding search queries to perform a search. [Figure 35] This figure shows an example of the hardware configuration of an information processing device, a knowledge server, a generation device, and a tagged question response server. [Modes for carrying out the invention]

[0013] Preferred embodiments of this disclosure will be described in detail below with reference to the attached drawings. In this specification and the drawings, components having substantially the same function are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0014] Before describing the embodiments, we will explain the multimodal model and description language used in these embodiments.

[0015] <Multimodal Model> In this embodiment, a multimodal model is used. This multimodal model will be described below.

[0016] A multimodal model is a model that can handle drawings and text simultaneously. CLIP, released by OpenAI, is a well-known example of a multimodal model. Figure 1 shows an example configuration of a multimodal model 1. In configuration 1, pairs of text and drawings are trained in the CLIP model 7. The text of one pair is converted into a text feature vector through the text encoder 5 in the CLIP model 7, and the drawing of the other pair is converted into an image feature vector through the vision encoder 4 in the CLIP model 7. There is also an extended model called the LLaVA model, which replaces the text encoder 5 with a Large-Scale Language Model (LLM). By using an LLM, LLaVA enables more detailed and context-dependent text generation.

[0017] The similarity calculation unit 6 calculates the cosine similarity between the text feature vector and the image feature vector. In doing so, it updates the parameters of the text encoder 5 and the vision encoder 4 so that the similarity of correct pairs (positive examples) is high and the similarity of incorrect pairs (negative examples) is low. Specifically, it employs contrast learning using cross-entropy loss and optimizes the CLIP model 7 by adjusting the similarity score according to the matching or mismatch of pairs. This type of learning is called contrast learning of the text encoder and the vision encoder.

[0018] Specifically, the similarity calculation unit 6 updates its parameters during the learning phase to increase the similarity of correct pairs (positive examples) within a mini-batch (a portion of the dataset). It also adjusts the parameters for incorrect pairs (negative examples) randomly generated within the mini-batch to reduce their similarity. Techniques such as contrast loss are used for this. By performing this iterative learning on a mini-batch basis across the entire dataset, the model learns to output feature vectors that are similar to those of paired texts and drawings.

[0019] By repeating this mini-batch training across the entire dataset, the model is optimized to output feature vectors with high similarity between paired images and text. CLIP Model 7 is trained on a dataset of approximately 400 million pairs of text and images and has shown good performance in many zero-shot classification, image search, and caption generation tasks. Furthermore, by adding new datasets and performing fine-tuning, performance improvements in specific tasks and fields can be expected. Details are described in the following papers, etc. Learning Transferable Visual Models From Natural Language Supervision https: / / arxiv.org / abs / 2103.00020 arXiv release date:2021 / 2 / 26

[0020] Non-Patent Document 2 discloses research on using a CLIP model on patent drawings. For example, Non-Patent Document 2 describes training a CLIP model with pairs of drawings and their descriptive texts filed in Japan over the past year, aiming for further accuracy improvements. The technology disclosed in Non-Patent Document 2 uses regular expressions to extract the descriptive texts from the patent specification.

[0021] In Non-Patent Document 2, "sentences describing the drawings" refers to sentences relating to the drawings within the specification (sentences included in the brief description of the drawings or sentences describing the drawings in embodiments).

[0022] There are several types of CLIP models, differing in version and parameter size. In the commonly used CLIP model, the text encoder and vision encoder each have 63 million parameters. Large-scale language models are intended for advanced natural language understanding and generation, and generally have hundreds of millions to tens of billions of parameters. Compared to these, the text handling capabilities of CLIP models are somewhat limited.

[0023] Furthermore, the CLIP model has a limited number of tokens it can handle (maximum text length), approximately 80 tokens. While text is converted into tokens for processing, the system is statistically designed to assign shorter tokens to frequently occurring characters and words. Therefore, tokenizers designed based on global text data tend to require fewer tokens for English words and numbers, while strings in languages ​​like Japanese, which are less prevalent in the training data, tend to require more tokens.

[0024] The LLaVA model is based on the CLIP model, but replaces the text encoder with Llama. The number of parameters for the text encoder can be selected from 7 billion to 65 billion, with 700 million or 1.3 billion commonly used. The maximum text length that can be handled can be selected from approximately 2000 or 4000 tokens.

[0025] <Descriptive language> In this embodiment, "drawing description text" is one of the texts used to describe the drawings. "Drawing description text" is an example of a descriptive modeling language. In addition to "descriptive modeling language," it may also be referred to as a "text-based modeling language." A descriptive modeling language is a language that abstractly describes systems, processes, and structures in text and generates visual diagrams and simulations based on that description. It differs from general documents such as newspaper articles and patents, which aim to convey information to the reader by explaining facts and technical details. Descriptive modeling languages ​​aim to efficiently model complex systems and processes through quantitative and logical descriptions and to organize and present information in a visually understandable form.

[0026] In this embodiment, the drawing description text can be classified into three types (A, B, C) as follows:

[0027] (A) Text written in Mermaid notation, PlantUML notation, Graphviz notation, etc. These notations allow for a simple description of the relationships between elements without specifying specific location or size information, making them particularly suitable for representing highly abstract structures and processes.

[0028] (B) Text written in DRAWIO format or SVG (Scalable Vector Graphics) format While the DRAWIO format is primarily edited using dedicated tools, the SVG format is a versatile vector graphics format usable with a wide range of tools and environments. Both formats are XML-based, offering high extensibility and the ability to describe positional and size information, allowing for the representation of relatively complex drawings and shapes. A graphical user interface allows users to fine-tune the representation. However, the format itself is complex. Furthermore, the DRAWIO format is often saved in a compressed format, making direct human editing difficult.

[0029] (C) Text written in accordance with CAD formats such as STEP and IGES. These formats allow for the accurate representation of complex shapes and product structures, such as those of parts. They are edited using graphical user interfaces like CAD tools, and are rarely edited directly by humans.

[0030] In this embodiment, the above-mentioned formats and methods of representation are conveniently referred to as "descriptive languages." As described above, each descriptive language has diagrams that are suitable and unsuitable when expressed using that language. Therefore, in this embodiment, each descriptive language is associated with the respective technical fields of the patent. For example, PlantUML is suitable for describing relationships such as flowcharts and class diagrams, while SVG is suitable for representing graphs and geometric figures.

[0031] Figure 2 shows an example of the correspondence between a description language and the various technical fields of a patent. In Figure 2, "Field" indicates the technical field of the patent. "Description Language" indicates the description language corresponding to that field. "Representation Target" indicates the content of the figure to be represented by the description language. By using an appropriate description language in each technical field, the system is configured to efficiently represent drawings and data according to the characteristics of that field. For example, SPICE and Verilog are often used in the field of electrical and electronic engineering and are suitable for circuit design and simulation.

[0032] In Figure 2, one of the "fields" is "General." This refers to descriptive languages ​​and representational objects that are not dependent on a specific technical field and can be used in common across various fields. CSV and Markdown, included in "General," are widely used for easily describing and managing structured data and tabular information. SVG is a standard format for generating scalable vector graphics, enabling efficient representation of visualization elements such as pie charts and bar graphs. Furthermore, all of the descriptive languages ​​shown in Figure 2 are languages ​​that allow for the creation of drawings from text written according to their respective formats, utilizing existing publicly known technologies. This enables efficient creation of drawings and data visualizations that meet the specialized requirements of each field.

[0033] The drawing description text can be edited using a text editor or dedicated tool, depending on the description language. Due to the characteristics of the description language, a relatively freely customizable format is available, allowing users to create a variety of drawing description texts efficiently and at low cost. For example, using PlantUML or Mermaid notation allows for the quick generation of simple structural diagrams and flowcharts, while using SVG or BIM formats enables the description of more precise graphics and structural details. The created drawing description text can be used to obtain image data or object-level drawing images according to dedicated drawing tools and methods. By creating numerous drawings paired with the drawing description text, training data for the drawing description model in the first embodiment described later can be obtained. Of course, it can also be created manually by a human. When given a target drawing, a human can manually create the drawing description text using tools to obtain a drawing similar to it.

[0034] Based on the above explanation, each embodiment will now be described. <First Embodiment> In the first embodiment, we will describe an information processing device 1000 that can convert drawings into drawing description text and back into drawings, as shown in Figure 3, using a multimodal model similar to the CLIP model or LLaVA. The difference from the technology disclosed in Non-Patent Literature 1 is that drawing description text is input to the multimodal model instead of "text describing the drawing". First, we will describe the configuration of the multimodal model during training.

[0035] Figure 4 shows an example configuration of the information processing device 1000 during learning according to the first embodiment. The information processing device 1000 includes a first drawing description model 1100, a similarity calculation unit 1200, and a drawing drawing unit 1300. The first drawing description model 1100 is a multimodal model and includes a vision encoder 1110 and a drawing description text encoder 1120.

[0036] The drawing unit 1300 analyzes the input drawing description text and either draws the entire drawing as a single image or draws individual elements as objects. In this process, the user may be allowed to select and edit objects using mouse clicks or screen touch operations.

[0037] The first drawing description model 1100 receives a drawing drawn by the drawing drawing unit 1300 and drawing description text as input. The vision encoder 1110 converts the input drawing into an image feature vector and outputs the image feature vector to the similarity calculation unit 1200. The drawing description text encoder 1120 converts the input drawing description text into a drawing description text feature vector and outputs the drawing description text feature vector to the similarity calculation unit 1200.

[0038] As described above, the drawing input to the first drawing description model 1100 is a drawing generated by the drawing drawing unit 1300 from the drawing description text. In other words, since the drawing description text is a drawing description text that explains the input drawing, the image feature vector and the drawing description text feature vector are trained to be similar vectors.

[0039] Therefore, the similarity calculation unit 1200 calculates the similarity between the image feature vector and the drawing description text feature vector, and updates the parameters used by the vision encoder 1110 for conversion or the parameters used by the drawing description text encoder 1120 for conversion using a symmetric learning method so that the similarity value is higher. The similarity calculation unit 1200 is an example of a processing unit.

[0040] Furthermore, as training data, the drawings from the patent publication and the explanatory text describing those drawings may be converted into drawing description text. In addition, to obtain drawing description text with higher accuracy, design drawings or specifications that served as the basis for the drawings in the patent publication may be used instead of the drawings from the patent publication.

[0041] As shown in Figure 4, the first drawing description model 1100, which has been appropriately trained from diverse datasets, exhibits high generalization performance and shows relatively high performance not only for drawings published in patent publications but also for drawings other than those published in patent publications. Of course, it is also possible to extract pairs of drawings and texts from internal design drawings and specifications, not just patent publications, and use them as training datasets to develop a model enhanced for the internal design information of a specific company.

[0042] Figure 5 shows an example configuration of an information processing device 1000 that generates drawing description text from a drawing after training the first drawing description model 1100. In Figure 5, the vision encoder 1110 converts the drawing into image feature vectors. The image feature vectors become the initial values ​​for the other drawing description text encoder 1120, which has been trained in comparison. Normally, the drawing description text encoder 1120 operates as an encoder that converts input drawing description text into feature vectors, but in this case, it functions as a decoder to generate drawing description text from image feature vectors.

[0043] Image feature vectors are input to the drawing description text encoder 1120 as drawing description text feature vectors. The drawing description text encoder 1120 uses the input drawing description text feature vectors to convert the content of the drawing into drawing description text by repeatedly predicting the next word based on the understanding of the content of the drawing by its large-scale language model. Drawing description text that accurately represents the content of the drawing while maintaining the consistency of the entire sentence is output.

[0044] As described above, according to the first embodiment, drawings can be converted into drawing description text. Furthermore, the drawing unit 1300 can convert the drawing description text into drawings. This effectively supports the creation of documents related to patent applications.

[0045] <Second Embodiment> In the second embodiment, we will describe an information processing device 1000 that can convert drawings into drawing description text and back into drawings, as shown in Figure 6, using a multimodal model similar to the CLIP model or LLaVA. The difference from the first embodiment is that the drawing description text in the first embodiment has become drawing description text. Unlike the "drawing description text," which is an abstract and structural description, the drawing description text aims to explain the details of the drawing in natural language. Another main difference from the technology disclosed in Non-Patent Document 1 is that the drawing goes through a drawing description model, which will be described later. First, we will describe the configuration during the learning of the multimodal model.

[0046] Figure 7 shows an example configuration of the information processing device 2000 during learning according to the second embodiment. The information processing device 2000 includes a second drawing description model 2100, a similarity calculation unit 2200, and a drawing description model 2400. The second drawing description model 2100 is a multimodal model and includes a drawing description text encoder 2110 and a drawing description text encoder 1120. The drawing description text encoder 2110 and the drawing description text encoder 2120 include a large-scale language model and are responsible for understanding the meaning of the drawing description text and the drawing description text and generating feature vectors.

[0047] The drawing is input to the drawing description model 2400. The drawing here may be a publicly available drawing, a drawing printed in a catalog, or various drawings photographed with a smartphone or other device. The drawing description model 2400 analyzes the input drawing, performs OCR (optical character recognition) and image correction as necessary, generates drawing description text, and outputs it to the drawing description text encoder 2110. The drawing description text encoder 2110 converts the drawing description text into a drawing description text feature vector and outputs the drawing description text feature vector to the similarity calculation unit 2200.

[0048] The drawing description text is natural language text that describes the purpose, design intent, and details of the components related to the drawing. The drawing description text is input to the drawing description text encoder 2120, which converts the drawing description text into a drawing description text feature vector and outputs the drawing description text feature vector to the similarity calculation unit 2200.

[0049] Since the drawing description text is a drawing description text that describes the input drawing, the similarity calculation unit 2200 calculates the cosine similarity between the drawing description text feature vector and the drawing description text feature vector, and updates the parameters of encoders 2110 and 2120 using a comparative learning method to increase this value. Through this learning, both encoders are adjusted to generate consistent feature representations, enabling a unified understanding of multimodal data. The similarity calculation unit 2200 is an example of a processing unit.

[0050] Figure 8 shows an example configuration of an information processing device 2000 that generates drawing description text from a drawing after training the second drawing description model 2100. In Figure 8, the drawing description model 2400 converts the drawing into drawing description text and outputs the drawing description text to the drawing description text encoder 2110. The drawing description text encoder 2110 converts the drawing description text into drawing description text feature vectors. The drawing description text feature vectors become the initial values ​​for the other drawing description text encoder 2120, which has been trained in a comparative manner.

[0051] The drawing description text feature vector is input to the drawing description text encoder 2120 as the drawing description text feature vector. At this time, the drawing description text encoder 2120 functions as a decoder when generating drawing description text from the generated feature vector. The drawing description text encoder 2120 understands the input drawing description text feature vector as context, and generates drawing description text by predicting the next word based on the understanding of the drawing content by its large-scale language model. This results in the output of detailed explanations in natural language regarding the drawing's purpose (e.g., an overview of the electrical circuit or the role of mechanical parts), design intent (e.g., reasons for structural selection or points of efficiency improvement), and components (e.g., descriptions of main parts and materials).

[0052] As described above, according to the second embodiment, drawings can be converted into drawing description text. This can suitably assist in the creation of documents related to patent applications.

[0053] Figure 9 shows an example configuration of an information processing device 2000 that generates drawings from drawing description text after training the second drawing description model 2100. The main difference from the configurations shown in Figures 7 and 8 is that a drawing unit 2300 is provided instead of the drawing description model 2400. In Figure 9, the drawing description text encoder 2120 converts the drawing description text into a drawing description text feature vector. The drawing description text feature vector becomes the initial value of the other drawing description text encoder 2110, which has been trained in comparison.

[0054] The drawing description text feature vector is input to the drawing description text encoder 2110 as the drawing description text feature vector. At this time, the drawing description text encoder 2110 functions as a decoder when generating drawing description text from the generated feature vector. Using the input drawing description text feature vector, the drawing description text encoder 2110 repeatedly predicts the next word based on the understanding of the drawing content of its large-scale language model, converts it into drawing description text, and outputs the drawing description text to the drawing unit 2300.

[0055] The drawing unit 2300 draws a drawing from the drawing description text. Here, the drawing unit 2300 may draw the entire drawing as a single image, or it may draw it object by object so that it can be selected and manipulated by mouse clicks or screen touch operations.

[0056] As described above, according to the second embodiment, the drawing description text can be converted into a drawing. This can suitably support the creation of documents related to patent applications.

[0057] Figure 10 shows an example configuration combining the first drawing description model 1100 described in the first embodiment above, the drawing unit 1300, and the second drawing description model 2100 described in the second embodiment. In Figure 10, the first drawing description model 1100 converts the drawing into drawing description text, and the drawing unit 1300 regenerates the drawing using the drawing description text. It also shows a configuration in which the second drawing description model 2100 converts the drawing description text into drawing explanation text.

[0058] As shown in Figure 10, this combined configuration allows for the conversion of drawings to drawing description text via drawing description text, and vice versa. This enables bidirectional conversion between drawings and description text, streamlining the creation of drawings and the automatic generation of descriptions in patent applications.

[0059] <Extension of drawing description text> Next, we will describe the extensions to the drawing description text used in the first and second embodiments described above.

[0060] Of the three types (A, B, C) classified in the description of the descriptive language, (A) cannot directly represent the leader lines and symbols necessary for drawings. Furthermore, because (A) is a simple language that does not explicitly hold positional or size information, including numerical values ​​and symbols for the symbols results in an awkward drawing.

[0061] Let's explain this using diagrams. Figure 11 shows an example of a diagram description text written in mermaid notation. Figure 12 shows an example of a diagram when the diagram drawing unit draws the diagram description text shown in Figure 11.

[0062] In Figure 11, the left column shows an example of drawing description text, and the right column shows an explanation of the drawing description text. Reference numerals 71, 72, and 73, as well as the first, second, and third processing units shown in Figure 12, are represented by the same type (box). Furthermore, in mermaid notation, as shown in Figure 11, the reference numerals and positional information for each processing unit are not included, resulting in an irregular arrangement where the horizontal positions of the reference numerals and processing units are not aligned in the drawing result.

[0063] The reason for this type of drawing is that the symbols and the first, second, and third processing units are not distinguished. The reason why the symbols and processing units are not distinguished is that Mermaid notation does not clearly define a notation for identifying symbols or a leader line connecting symbols to processing units.

[0064] Therefore, in this embodiment, a notation is introduced to indicate that something is a code. Figure 13 shows an example of drawing description text when written using the notation that indicates something is a code. In Figure 13, a description element whose name begins with "REF" indicates that it is a code element. By defining a description element whose name begins with "REF" as a code element, it becomes possible to distinguish between code elements and processing units and apply an appropriate drawing format to each. By introducing such a notation, it becomes possible to distinguish between code elements and non-code elements (hereinafter also referred to as "original description elements").

[0065] Figure 14 is a flowchart showing the processing flow of the drawing unit. When the drawing unit receives drawing description text, it analyzes the text and sorts each descriptive element contained in the text into a symbol element and an actual description element (step S101). Specifically, the drawing unit sorts descriptive elements whose names begin with REF into symbol elements, and sorts all other descriptive elements into actual description elements.

[0066] Next, the drawing unit determines the display position of each descriptive element. For the original descriptive elements, the drawing unit uses only the original descriptive elements (excluding the symbol elements) and, following the same logic as existing drawing description text, determines the display position of the original descriptive elements within the group of original descriptive elements (step S102), and then terminates processing related to the original descriptive elements. For example, the position determination of the original descriptive elements is performed considering the relative relationships of each element and predetermined placement rules (e.g., centering or alignment).

[0067] Regarding symbol elements, the drawing unit determines the display position of the symbol elements within a group of symbol elements in accordance with the drawing requirements (step S103). For example, the drawing unit places the symbols in a position where leader lines do not intersect. When the descriptive elements are originally located in the center, such as in a part appearance drawing, it is easier to see if the symbol elements are arranged concentrically from the center. The optimal arrangement method is selected according to the type of drawing, such as flowcharts, block diagrams, and sequence diagrams. Generally, for drawings where the descriptive elements are expected to be arranged neatly, it is easier to see if the symbol elements are arranged in a relative position within the drawing, such as to the right or above the original descriptive elements.

[0068] The drawing unit determines the shape and display label of the symbol element (step S104) and finishes processing related to the symbol element. Figure 15 shows an example of the shape of a leader line. The shape of the leader line is curved (Curved Leader) to make it easy to distinguish it from the links between the elements that are actually described. In particular, an S-shaped leader (S-shaped Leader: see Figure 15) is drawn in this style because it provides better visual distinction from the elements that are actually described. Because the S-shaped leader has a different curve shape from the links between the elements that are actually described, it improves visibility and allows for an intuitive distinction between the symbol element and the elements that are actually described. The tip of the leader line (the end on the side of the elements that are actually described) may or may not have an arrow. If the symbol is enclosed in a square or rectangle, the visual distinction from the elements that are actually described decreases, so as shown in Figure 15, the numbers and letters of the symbol are displayed directly without any decoration such as being enclosed.

[0069] The drawing unit determines the position and shape of each element using the procedure described above and draws it for display on the support disclosure display device. The drawing unit may convert the drawing to an image format, or it may draw on an object-by-object basis so that selection and editing of elements can be performed using a mouse or touch operation.

[0070] Figure 16 shows an example of a drawing that is generated when the processing shown in Figure 14 is applied to the drawing description text shown in Figure 13. The drawing shown in Figure 16 may be a single image as a whole, or each element included in the drawing may be selectable and editable using mouse or touch operations. For example, "First Processing Unit," "Second Processing Unit," and "Third Processing Unit" may be selectable and manipulated using mouse or touch. Similarly, the symbols "71," "72," and "73" may also be selectable and manipulated using mouse or touch. This allows users to select symbols and processing unit elements by clicking or tapping, change their position, and edit their text, enabling interactive adjustment of the drawing.

[0071] Figure 17 shows an example of the configuration of the drawing unit. In Figure 17, the drawing unit 300 includes a processing unit 310 and a layout determination model 320. The processing unit 310 analyzes the drawing description text, separates it into symbol elements and original elements, and determines the layout of the original elements. The layout determination model 320 determines the layout of the symbol elements and modifies the layout of the original elements based on the layout of the original elements determined by the processing unit 310. Based on human visual characteristics, the layout determination model 320 considers appropriate spacing and alignment patterns between elements and outputs a harmonious overall layout.

[0072] Layout refers to the size, position, and shape of elements such as lines and boxes. It also includes specifying the alignment of multiple elements (such as horizontal centering or vertical left-alignment). To create a layout that looks natural to humans, alignment is crucial, in addition to position and size.

[0073] The layout determination model 320 can be trained by taking the drawing description text and the layout determined by the processing unit 310 as input, and updating the parameters within the model to minimize the difference from a layout that looks natural to a human. The layout determined by the processing unit 310 here is the layout output by following the existing layout determination process. However, the layout determined by the processing unit 310 is not mandatory. The layout determination model 320 may also be trained to receive the drawing description text and output a layout that looks natural to a human without referring to the layout determined by the processing unit 310.

[0074] Preparing the training dataset for the layout determination model 320 is relatively easy. This can be done by following existing layout determination processes and recording the process of humans adjusting the position and size of elements in the output layouts through a graphical UI or chat UI to create a layout that looks natural to humans. The target data can be drawings included in patent publications or internal company documents. The drawing description text can be created manually from scratch or created by referencing data obtained using a large-scale language model capable of image recognition.

[0075] As shown in Figure 2, there are many types of descriptive languages ​​for describing drawings and operations. Up to this point, we have mainly used block diagrams as examples, but throughout this specification, we are not limited to any particular descriptive language. As shown in Figure 2, electrical circuit diagrams can be described using SPICE, mechanical parts can be described using STEP or IGES, and chemical structural formulas and molecular structures can be described using IUPAC or SMILES. Although not limited to any particular field, tabular structures can be described using CSV or Markdown, and pie charts and bar graphs can be described using SVG. Anything that can be expressed using any of the descriptive languages ​​shown in Figure 2 can be effectively handled in the first and second embodiments described above.

[0076] To create a general-purpose multimodal model that covers all types of drawings, it is necessary to have a sufficiently large number of parameters and prepare a sufficiently large amount of training data, which presents challenges in terms of implementation costs.

[0077] Therefore, for the model related to the drawings in this embodiment, we introduce MoE (Mixture of Experts). MoE combines relatively small, specialized large-scale language models (expert models). An appropriate expert model is selected according to the characteristics of the input text.

[0078] In the text of a specification, identifying the specific field and subject matter can be achieved through language processing that detects characteristic keywords. For example, in electrical and electronic engineering, characteristic keywords include circuit configuration, elements, voltage, and current. In mechanical engineering, characteristic keywords include shape, dimensions, relative position, and material. In software, characteristic keywords include processing procedures, data structures, and functional descriptions. In chemistry and materials science, characteristic keywords include chemical structure, molecules, bonds, and substituents.

[0079] By inputting patent classification codes (such as FI terms and IPC classifications) and drawing description text, a language model can be created that is trained to output the above-mentioned fields and appropriate descriptive languages. To prepare the training data, the above-mentioned characteristic keywords are extracted and used as training data. After that, human editing can be used to improve accuracy.

[0080] Using a language model that outputs an appropriate descriptive language, an appropriate expert model is selected from the text of the specification. Then, using the selected expert model, various processing described herein can be performed with high accuracy on the drawings, drawing description text, and drawing description text.

[0081] Figure 18 shows a configuration in which an expert switching unit 1130, which is a language model that outputs an appropriate description language, is added to the configuration of the first embodiment. The expert switching unit 1130 receives input such as IPC, FI terms, and drawing description text. For example, IPC, FI terms, and drawing description text may be obtained from the drawings in the patent publication and the IPC and FI terms described in the patent publication. The expert switching unit 1130 analyzes the input classification symbols and drawing description text and selects a description language, for example, SPICE for electrical circuit diagrams and STEP for mechanical parts. This selection is based on the extraction of characteristic keywords and may be performed manually by a human to improve accuracy. The first drawing description model 1100 outputs drawing description text using the description language output from the expert switching unit 1130. The output drawing description text can be directly edited with general drawing editing software, streamlining the creation and modification of drawings in patent publications.

[0082] Instead of the keywords and classification symbols mentioned above, a model can be created to identify the field from the drawing. Humans can easily classify a drawing as a circuit diagram, chemical structure diagram, or flowchart. By preparing training data for identifying fields from drawings and using a machine learning-based field identification model, high-accuracy classification can be achieved. Since this involves only a few dozen types of classification problems and is a relatively easy task, even a small-scale field identification model can achieve sufficient performance.

[0083] Using this domain-specific model, an appropriate expert model is selected from the drawings. Then, using the selected expert model, the various processes described herein can be performed with high precision on the drawings, drawing description text, and drawing commentary text.

[0084] <Search> Next, we will describe an embodiment for searching for drawings using drawings and text. In this embodiment, we first generate a combined feature vector to be used in the search. We will now describe how to generate this combined feature vector.

[0085] There are two configurations for generating combined feature vectors. The first configuration will be described. Figure 19 shows a first combined feature vector generation device 11 for obtaining combined feature vectors. The first combined feature vector generation device 11 consists of a search model 7 and a combining processing unit 8. The search model 7 consists of the vision encoder 4 and text encoder 5 described above. The search model 7 is a learned model that has been trained so that the text feature vector and the image feature vector are similar when the drawing description text is drawing description text that describes the input drawing.

[0086] Before inputting a drawing into the vision encoder 4, it undergoes preprocessing such as resizing to an appropriate size and normalizing pixel values. The preprocessed drawing is then input into the vision encoder 4, and an image feature vector (a fixed-length vector (e.g., 512 dimensions)) is output. The image feature vector represents the visual features of the drawing in a high-dimensional space.

[0087] Similarly, text data representing the text describing the drawing is also vectorized. Each text is first tokenized and preprocessed by padding to a fixed length as needed. This preprocessed text is input to the text encoder 5, which outputs a text feature vector with the same number of dimensions as the image feature vector. This text feature vector represents the semantic content of the text in a similarly high-dimensional space.

[0088] In Figure 19, the drawing is input to the vision encoder 4. The vision encoder 4 converts the drawing into an image feature vector and outputs the image feature vector to the merging processing unit 8. The text is input to the text encoder 5. The text encoder 5 converts the text into a text feature vector and outputs the text feature vector to the merging processing unit 8.

[0089] Ideally, if the data is similar to the trained drawing-text pairs, the text feature vectors and image feature vectors will be similar, but sometimes the resulting vectors may not be very similar. If the data is far removed from the training dataset, the text feature vectors and image feature vectors may not be similar. Therefore, instead of using only one of the feature vectors, the combining processing unit 8 combines these two feature vectors to generate a single multimodal representation (i.e., a combined feature vector). For example, the two feature vectors may be concatenated or averaged. The final combined feature vector represents the features of the original drawing-text pairs, enabling drawing retrieval.

[0090] In the configuration shown in Figure 19, both drawings and text are input, but either one or the other may be input. In this case, the combining processing unit 8 outputs either of the feature vectors output from the search model 7 as the combined vector.

[0091] Next, the second configuration will be described. Figure 20 shows the second combined feature vector generation device 3000 for obtaining a combined feature vector using the second drawing description model 2100 described above. The second combined feature vector generation device 3000 consists of the drawing description model 2400, the second drawing description model 2100, and the combined processing unit 8. The second drawing description model 2100 consists of the drawing description text encoder 2110 and the drawing description text encoder 2120 described above.

[0092] In Figure 20, the drawing is input to the drawing description model 2400. The drawing description model 2400 converts the drawing into drawing description text and outputs it to the drawing description text encoder 2110. The drawing description text encoder 2110 converts the drawing description text into drawing description feature vectors and outputs the drawing description feature vectors to the merging processing unit 8. Note that, as shown in Figure 20, even if there is no drawing, if there is drawing description text, that drawing description text may be input to the drawing description text encoder 2110.

[0093] The drawing description text is input to the drawing description text encoder 2120. The drawing description text encoder 2120 converts the drawing description text into a drawing description text feature vector and outputs the drawing description text feature vector to the merging processing unit 8. During the vector conversion of the drawing description text and drawing description text, the text is first tokenized and preprocessed by padding to a fixed length as needed. Note that if the specification is for the user themselves or the user's company, the original drawing description text that was separately created to create the drawings for that specification may be used.

[0094] Ideally, if the data is similar to a pre-trained pair of drawings and texts, the drawing description text feature vector and the drawing explanation feature vector will be similar. On the other hand, if the data is far removed from the training dataset, the drawing description text feature vector and the drawing explanation text feature vector may not be similar. Therefore, instead of using either one of the feature vectors, the combining processing unit 8 combines these two feature vectors to generate a single representation, i.e., a combined feature vector. This process may involve, for example, concatenating the two feature vectors or averaging them. The finally obtained combined feature vector represents the features of the original drawing and text pair, enabling drawing retrieval.

[0095] In the configuration shown in Figure 20, both drawing description text and drawing explanation text are input, but either one or the other may be input. In this case, the combining processing unit 8 outputs either of the feature vectors output from the search model 7 as the combined vector. This function ensures the flexibility to perform a search using only one of the data sets.

[0096] Figure 21 is a block diagram showing the basic configuration for registering and searching for combined feature vectors. The explanatory models 2100A and 2100B in Figure 21 include the configuration of the first combined feature vector generation device 11 or the second combined feature vector generation device 3000. These configurations are shown in a simplified manner, and both are capable of outputting combined feature vectors. See Figures 19 and 20 for details.

[0097] The search target database stores search target data such as drawings, drawing description text, and drawing explanation text, as well as the corresponding combined feature vectors (generated by the drawing generation model 2100A). The drawing explanation model 2100A retrieves search target data (drawings, drawing description text, drawing explanation text, etc.) from the search target database, generates combined feature vectors, and stores them in the search target database.

[0098] The drawing-illustrated model 2100B generates a search query feature vector from a user search query that indicates what the user wants to search for, and obtains search target data as high-similarity data by associating it with a combined feature vector similar to the generated search query feature vector.

[0099] In the search shown in Figure 21, the system may be constructed as follows: First, to store and search the data converted into combined feature vectors, a vector database specialized in the efficient processing of high-dimensional vectors is employed. Specifically, a database system that provides high-speed nearest neighbor search functionality for large datasets, such as FAISS, is used. FAISS, through its internal algorithm, can achieve search responses in milliseconds even for vector data on the scale of billions.

[0100] Next, a FAISS index is constructed using the generated combined feature vectors. Alternatively, an IVFFlat index can be created to enable high-speed searching of the vector space. This index divides the vector space into clusters and performs linear search within each cluster, dramatically improving search efficiency for large datasets.

[0101] In the search process, the search query entered by the user (drawings, text, or both) is first vectorized using the same settings as the model and method used during database construction. Next, a nearest neighbor search is performed on the FAISS index using the resulting query vector. During the search, distance metrics such as cosine similarity and L2 distance are applied to identify the database vector that is most similar to the query vector. Cosine similarity measures similarity based on the angle between two vectors, with a value closer to 1 indicating higher similarity. This is effective for checking content similarity because it does not depend on the magnitude of the vectors and only considers the direction of the vectors. On the other hand, L2 distance (Euclidean distance) measures the straight-line distance between two vectors, and a smaller distance indicates higher similarity.

[0102] Furthermore, when query vectors are evaluated using cosine similarity, normalizing each vector and setting its length to 1 allows the similarity to be calculated using only the dot product, improving computational efficiency. At this stage, the FAISS IVFFlat index is used to efficiently narrow down the clusters that are most likely to contain the query vectors. This eliminates the need to calculate similarity with all data vectors, and by performing detailed similarity calculations only within the narrowed-down clusters, the amount of computation is significantly reduced, improving search speed.

[0103] Next, calculations based on cosine similarity or L2 distance are performed between the query vector and the target vectors within the narrowed cluster. For cosine similarity, the dot product is used; for L2 distance, the Euclidean distance formula is followed to measure similarity. Finally, candidates are ranked based on the calculated similarity and presented as search results in order of highest similarity. This entire process allows for the efficient acquisition of high-similarity data for the user's search queries.

[0104] In the information processing device using the first combined feature vector generation device 11, the types of search data that the user can input are the following three types: 1. Search by drawing 2. Search by drawing description text 3. Search using drawings and drawing description text.

[0105] In the information processing device using the combined feature vector generation second device 3000, the types of search data that the search user can input are the following seven types: 1. Search by drawing 2. Search by drawing description text 3. Search using drawings and drawing description text. 4. Search using drawing description text. 5. Search using drawing description text and drawing explanation text. 6. Search using drawings and drawing description text. 7. Search using drawings, drawing description text, and drawing explanation text.

[0106] In the combined feature vector generation second device 3000, drawing description text can be input as a substitute for the drawing, making the searches described in 3 and 4 above possible. Furthermore, since the drawing description text converted from the drawing using a drawing description model can be used simultaneously with the original drawing description text by concatenation, the searches described in 6 and 7 above become even more possible.

[0107] <Third Embodiment> In the third embodiment, an information processing device 3000 that can convert claims to claim-supporting disclosures and claim-supporting disclosures to claims, as shown in Figure 22, using a multimodal model similar to the CLIP model or LLaVA, will be described. "Claim-supporting disclosures" refer to descriptions and drawings in the specification that support the claims. Descriptions in the specification that support the claims refer to elements (text, tables, mathematical formulas, chemical formulas, hereinafter also referred to as "supporting text") in the detailed description of the invention that supports the claims.

[0108] Generally, in patent law, "support" refers to an invention claimed in one patent application being described as an invention in the detailed description of an invention in the same patent application. However, in this third embodiment, in addition to the general support in patent law described above, support may also include an invention claimed in one patent application being described as an invention in the detailed description of an invention in another patent application.

[0109] Claims describe the defining features of the invention, and these are described in the specification together with specific descriptions that support the claims (such as the problem the invention aims to solve, the means for solving the problem, a description of the drawings, and a detailed description). Drawings are part of the specific description, and it is desirable to be able to search not only individual drawings, but also patent publications and individual claims while taking the information from the drawings into consideration.

[0110] A single patent publication can have a considerable number of characters. When attempting to input the entire text of a patent publication into a large-scale language model, it is not uncommon for the entire text to be uninputtable because it exceeds 128K tokens. For example, the maximum input token size for ChatGPT around 2023 varied from 4K to 16K depending on the model and version, while the maximum input token size for ChatGPT around 2024 was 128K.

[0111] The same applies to Llama within LLaVA; when it first appeared around 2023, the maximum input token size was 4K tokens, and as a result of technological competition with models such as ChatGPT and Claude, it expanded to 128K tokens around 2024. As of October 2024, large-scale language models have finally expanded to over 100K tokens, but patent publications with more characters than this are not uncommon. Furthermore, even if the maximum token size continues to expand, inputting a large number of tokens increases processing time and power consumption in proportion to the number of tokens, so a mechanism that efficiently processes patent information with a small number of tokens is desirable.

[0112] It is said that a single patent application typically contains 10 claims. We introduce a data unit that associates an entire patent publication with individual claims and their corresponding supporting text, figures, tables, mathematical formulas, or chemical formulas. In the following explanation, this unit will be referred to as a CSU (Claim Support Unit). For a patent document with 10 claims, 10 CSUs can be defined. Even if the supporting text is duplicated, the amount of data created by dividing the CSUs is significantly reduced. As a result, processing time and power consumption when processing with a large-scale language model can be greatly reduced. Dividing into smaller CSUs also facilitates distributed parallel processing using multiple servers.

[0113] Since the supporting text corresponding to a claim can generally be found in multiple consecutive or non-consecutive paragraphs, there may be multiple supporting texts corresponding to a single claim in the above CSU. Similarly, since there may be multiple figures, tables, mathematical formulas, or chemical formulas corresponding to a claim, there may be multiple figures, tables, mathematical formulas, or chemical formulas corresponding to a single claim in the CSU. Note that drawings showing figures, tables, mathematical formulas, or chemical formulas may be converted into drawing description text.

[0114] Figure 23 shows an example configuration of the information processing device 3000 during learning according to the third embodiment. The information processing device 3000 includes a CSU model 3100, a similarity calculation unit 3200, and a text conversion unit 3500. The CSU model 3100 is a multimodal model and includes a claimed text encoder 3110 and a support text encoder 3120.

[0115] The claim text encoder 3110 and the support text encoder 3120 may be any large-scale language model, for example, Llama. Also, since the length of the claim and the length of the claim support disclosure differ significantly, large-scale language models with different numbers of parameters and performance may be used.

[0116] Comparing the claims and the claim-supporting disclosures, the claim-supporting disclosures are longer. Taking the Llama3.1 model as an example, the claimed text encoder 3110 may use either Llama3.1 8B with 8 billion parameters or Llama3.1 70B with 70 billion parameters. On the other hand, the support text encoder 3120, which will handle relatively long data, may use Llama3.1 405B with 405 billion parameters. When combining models with different numbers of parameters, the number of dimensions of the vectors may differ. In this case, the vector with the lower number of dimensions is linearly projected onto the vector space of the model with the higher number of dimensions to make the dimensions the same.

[0117] In Figure 23, the claim text representing the claims is input to the claim text encoder 3110. The claim text encoder 3110 converts the claim text into a claim text feature vector and outputs the claim text feature vector to the similarity calculation unit 3200.

[0118] The text conversion unit 3500 converts the figures, tables, mathematical formulas, and chemical formulas in the claim support disclosure into text and outputs them to the support text encoder 3120 as support text. The text conversion unit 3500 may be located before the CSU model 3100, as shown in Figure 23, or it may be included inside the CSU model 3100.

[0119] The support text is input to the support text encoder 3120. The support text encoder 3120 converts the support text into a support text feature vector and outputs the support text feature vector to the similarity calculation unit 3200.

[0120] Since the support text is the support text that supports the input claim, the similarity calculation unit 3200 updates the parameters used by the claim text encoder 3110 for conversion, or updates the parameters used by the support text encoder 3120 for conversion, so that the claim text feature vector and the support text feature vector are similar. As a result, the parameters of the claim text encoder 3110 and the support text encoder 3120 are learned in comparison. The similarity calculation unit 3200 is an example of a processing unit.

[0121] This section explains the training data used in this learning process. Claim support disclosures are found by searching for parts with high semantic similarity within the entire lengthy patent document, using characteristic keywords as clues. After setting up the initial training data, human correction is performed to prepare pairs of claims and claim support disclosures. By preparing the training data limited to a specific patent field, a CSU model that operates with high accuracy for that field can be created with relatively little effort. After achieving high performance in a certain field, applying the CSU model to adjacent fields allows the model to input patent documents and output claim support disclosures with relatively high accuracy, which can then be corrected and completed by a human. Using the method of marking parts with high similarity scores (described later), humans can create correct claim support disclosures by referring to the output of the CSU model. In this way, by repeatedly outputting reference data for training data from the CSU model and correcting it manually, it is possible to create a CSU model that operates with high accuracy for all patent documents.

[0122] Next, we will describe the data structure of the claim-supporting disclosure. The claim-supporting disclosure consists of multiple parts, such as the text of the specification and drawings. When converting the claim-supporting disclosure to text format, special delimiters or tokens may be included, as shown below.

[0123] A description of text information. (X) Sentence 1 [SEP] Sentence 2 [SEP] Sentence 3... (Y) [BOS] Sentence 1 [EOS] [BOS] Sentence 2 [EOS] [BOS] Sentence 3 [EOS] (Z) [BOS] Sentence 1 [SEP] Sentence 2 [SEP] Sentence 3 [EOS]

[0124] The text information in the specification body is described using one of the methods (X), (Y), or (Z) described above. [SEP] is a token that indicates the separation of paragraphs or sentences. When dealing with patent documents, remove the paragraph number and insert the [SEP] token each time a paragraph number appears. Paragraph numbers are information used to identify a location and are unrelated to the content of the invention. In CSU models that deal with claim text, which is the content of the invention, and claim supporting disclosures, which are specific descriptions that support the claims, excluding paragraph numbers improves the accuracy of CSU generation.

[0125] On the other hand, in a device that receives and processes claim-supporting disclosures generated by a CSU model, the [SEP] token indicates that a paragraph number originally existed, so the [SEP] token may be replaced with the paragraph number for display.

[0126] The description format (Y) above encloses the text with a [BOS] token, which signifies the start of the text, and an [EOS] token, which signifies the end of the text. When dealing with the main text of a patent document, [BOS] is placed where the paragraph number was located, and [EOS] is placed where the next paragraph number exists or where the text ends. In a device that receives and processes data in this format, paragraph number generation and automatic numbering may be performed in accordance with the [BOS] token, similar to the description format (X). The format (Z) above is a combination of the formats (X) and (Y).

[0127] Description for the drawing description text (including image description text) of an independent drawing. [BOF] Drawing Description Text [EOF]

[0128] In cases where drawings are grouped together at the end of a PDF, such as in official gazettes in Japan, the drawing description text containing the drawing information should be written in the format described above. The drawing description text should be enclosed by [BOF], which indicates the start position of the independent drawing description text, and [EOF], which indicates the end position of the independent drawing description text.

[0129] Description of the image description text for small images (such as mathematical formulas) embedded in the main text of the specification. [BOI] Drawing Description Text [EOI]

[0130] The image description text for small images (such as mathematical formulas) embedded in the main text of the specification should be enclosed by [BOI], indicating the start position, and [EOI], indicating the end position.

[0131] Description of the reference numerals in the main text of the specification. Replace the sign with [REF].

[0132] Let's explain the substitution examples. First, the text within the following quotation marks is an example of text from the specification. "The first processing unit 71 has the function of receiving emails from external sources and obtaining the email body and related information. It extracts only the emails with relevant content and passes them to the second processing unit 72. The second processing unit 72 automatically generates multiple reply options according to the content and passes them to the third processing unit 73."

[0133] The following text in brackets is the result of replacing the symbols in the example text above. "The first processing unit [REF] has the function of receiving emails from external sources and retrieving the email body and related information. It extracts only the emails with relevant content and passes them to the second processing unit [REF]. The second processing unit [REF] automatically generates multiple reply options according to the content and passes them to the third processing unit [REF]."

[0134] Reference numerals are information that links descriptive elements in the text and drawings, but they do not provide information that specifically describes the operation of the invention. When storing them in a CSU model, replacing reference numerals with [REF] tokens improves the ability to generate claims and claim-supporting disclosures.

[0135] Figures 24A, 24B, 24C, and 24D are flowcharts illustrating processing examples for a support disclosure display device that shows a claim support disclosure including a special token. Figure 24A is a flowchart showing the processing when a BOS is detected in a claim support disclosure including a special token. The support disclosure display device increments the paragraph number PI (step S201). This paragraph number PI is initialized to 0. Next, the support disclosure display device displays the paragraph number PI in square brackets, similar to a patent publication (step S202), and terminates the processing when a BOS is detected.

[0136] Figure 24B is a flowchart showing the processing when a BOF (Block on Final) is detected in a claim support disclosure including a special token. The support disclosure display device increments the figure number FI (step S301). This figure number FI is initialized to 0. Next, the support disclosure display device displays the figures from BOF to EOF (step S302), displays the figure number FI in corner brackets, similar to a patent publication, and terminates the processing when BOF is detected.

[0137] Figure 24C is a flowchart showing the processing when a BOI is detected in a claim support disclosure including a special token. The support disclosure display device processes the BOI to EOI as an inline image (step S401). The support disclosure display device displays the image from the BOI to the EOI that has been processed as an inline image (step S402), and terminates the processing for BOI detection. When processing the image from the BOI to the EOI inline, it is possible to select a format such as rendering in SVG format or outputting in PNG format.

[0138] Figure 24D is a flowchart showing the processing when a REF is detected in a claim support disclosure that includes a special token. The support disclosure display device obtains the noun immediately preceding the REF (step S501) and obtains the code corresponding to the obtained noun (step S502). The code corresponding to the noun is assumed to be stored in advance. The support disclosure display device displays the code at the position of the REF (step S503) and terminates the processing when a REF is detected.

[0139] Figure 25 shows an example configuration of an information processing device 4000 that generates claim support disclosures from claims after training the CLU model 4100. In Figure 25, the claim text encoder 4110 converts the claim text into a claim text feature vector. The claim text feature vector becomes the initial value of the other support text encoder 4120, which has been trained in comparison.

[0140] The support text feature vector is input to the support text encoder 4120 as the drawing description text feature vector. At this time, the support text encoder 4120 functions as a decoder when generating claim support disclosures from the generated feature vector. The support text encoder 4120 uses the input support text feature vector to repeatedly predict the next word based on the large-scale language model within the support text encoder 4120's understanding of the drawing content, converts it into support text, and outputs the support text to the text inverse conversion unit 4600. The text inverse conversion unit 4600 converts the support text into claim support disclosures.

[0141] The support text includes the text to be included in the main body of the specification and image description text. Image description text is text that describes image information such as drawings, mathematical formulas, chemical formulas, tables, and graphs. Image information such as drawings, mathematical formulas, chemical formulas, tables, and graphs can be generated from the image description text in the same manner as described in the first embodiment.

[0142] The configuration shown in Figure 25 allows users to obtain claim-supporting disclosures from the claims. In particular, with the configuration shown in Figure 25, information that may be represented by drawings in the specification, such as diagrams, mathematical formulas, chemical structural formulas, tables, and graphs, can be generated seamlessly without any special operations, just like the text describing the claims. Therefore, it offers high convenience for users when writing.

[0143] Figure 26 shows an example configuration of an information processing device 5000 that generates claims from claim support disclosures after training the CSU model 5100. In Figure 26, the text conversion unit 5500 automatically determines the type of drawing (such as drawings, mathematical formulas, chemical structural formulas, tables, and graphs) included in the claim support disclosure, selects an appropriate image description language format, and converts the image information into image description text. The text originally included in the claim support disclosure is not converted. The text conversion unit 5500 concatenates the originally included text and the converted image description text and outputs it as support text to the support text encoder 5120. The support text encoder 5120 converts the support text into a support text feature vector. The support text feature vector becomes the initial value of the other drawing description text encoder 2110, which has been learned in comparison.

[0144] The support text feature vector is input to the claim text encoder 5110 as the claim feature vector. In this case, the claim text encoder 5110 performs the following: It functions as a decoder when generating claim text from the generated feature vectors. The claim text encoder 5110 uses the input claim text feature vectors to convert the large language model within the claim text encoder 5110 into claim text, which iteratively predicts the next word based on the understanding of the content of the drawing, and outputs the claim text.

[0145] It is often difficult to accurately describe inventions characterized by two-dimensional or three-dimensional structures, such as electrical circuit diagrams, chemical structural formulas, and external view drawings, solely through the text of the specification. According to the third embodiment, these complex structural diagrams, or the original drawing description text that generated the diagrams, can be handled seamlessly. This is because the information such as diagrams is converted into text format by the text conversion unit, and the model can process it in text information format, just like the main text. The CSU model can understand the content of the diagrams and generate accurate claim text.

[0146] Next, the information processing apparatus in the third embodiment will be described. First, the generation of a combined feature vector using the CSU model will be described. Figure 27 shows the third combined feature vector generation apparatus 6000 for obtaining a combined feature vector using the CSU model 3100 described above. The third combined feature vector generation apparatus 6000 consists of a text conversion unit 3500, a CSU model 3100, and a combined processing unit 8. The CSU model 3100 consists of the claimed text encoder 3110 and the support text encoder 3120 described above.

[0147] In Figure 27, the claim text is input to the claim text encoder 3110. The claim text encoder 3110 converts the claim text into a claim text feature vector and outputs the claim text feature vector to the merging processing unit 8. The text conversion unit 3500 converts the figures, tables, mathematical formulas, and chemical formulas in the claim supporting disclosure into text and outputs them as support text to the support text encoder 3120. The support text is input to the support text encoder 3120. The support text encoder 3120 converts the support text into a support text feature vector and outputs the support text feature vector to the merging processing unit 8.

[0148] During the vector transformation of each text, the text is first tokenized and padded to a fixed length as needed. This pre-processed text is then input into the model's text encoder to generate a vector representation.

[0149] Furthermore, image description text may be included in the claim support disclosure instead of drawings. The text conversion unit 3500 outputs the image description text without performing any conversion processing. If the specification is not from another company's patent publication but from the user's own company or the company the user works for, the original drawing description text that was separately created to produce the drawings for that specification may be used as the image description text.

[0150] Ideally, if the data is similar to a trained pair of claims and claim supporting disclosures (claim support units), the claim text feature vector and the support text feature vector will be similar. However, sometimes vectors that are not very similar may be obtained. If the data is far from the training dataset, the claim text feature vector and the support text feature vector may not be similar. Therefore, the combining processing unit 8 does not use either feature vector alone, but combines these two feature vectors to create a single representation (i.e., a combined feature vector). For example, the two feature vectors may be concatenated or averaged. The finally obtained combined feature vector represents the features of the claim support unit and enables similarity search.

[0151] In the configuration shown in Figure 27, the claim text and the claim supporting disclosure are input, but either one or the other may be input. In this case, the coupling processing unit 8 outputs either of the feature vectors output from the search model 7 as the coupling vector. The coupling processing unit 8 is an example of a coupling unit.

[0152] For searching, a search query feature vector is generated from the user search query that indicates what the user wants to search for, and search target data (claims, claim supporting disclosures, etc.) associated with a combined feature vector similar to the generated search query feature vector is obtained as high-similarity data.

[0153] Next, the process for displaying claim-supporting disclosures will be described. Figure 28 shows the configuration of the similarity display device 3300 for displaying claim-supporting disclosures. The similarity display device 3300 includes a receiving unit 3130, a CSU model 3100, a similarity extraction unit 3140, and a similarity display unit 3150.

[0154] In Figure 28, the receiving unit 3130 receives the patent publication. The receiving unit 3130 outputs the claim text corresponding to the claim specified by the user (for example, claim 1) to the CSU model 3100. The CSU model 3100 converts the claim text into support text, the reverse text conversion unit 4600 converts the support text into claim support disclosure, and outputs the claim support disclosure to the similarity extraction unit 3140.

[0155] Due to limitations in the performance of the CSU model 3100 and the fact that the writing style is not similar to the training data of the CSU model, it is highly unlikely that the generated claim support disclosure will exist exactly as it is in existing patent documents, but it can be expected that similar expressions will be found. The procedure of the similarity extraction unit 3140, which extracts the scope of claim support disclosure that is likely to correspond to the claims, is shown below.

[0156] First, the similarity extraction unit 3140 divides the claim support disclosure into units such as a fixed number of characters or punctuation marks, and finds similar parts in the specification and figures to be extracted. A similarity score is used to determine the similarity of the text. As this similarity score, cosine similarity or BLUE score after vector representation is used.

[0157] Next, the similarity extraction unit 3140 outputs the location of the similarity section and the claim similarity score to the similarity display unit 3150. The claim similarity score, as determined by the similarity extraction unit 3140, is the similarity between each part of the claim supporting disclosure and the entire specification.

[0158] The similarity display unit 3150, when displaying a portion similar to a claim supporting disclosure, colors the entire specification according to the aforementioned similarity score. If the cosine similarity is normalized from 0 to 1, the text and figures showing high similarity may be marked in a bright yellow or similar color to make them stand out, while the text and figures showing moderate similarity may be marked in a lighter color with slightly reduced saturation. In other words, the saturation and brightness are changed according to the similarity.

[0159] When displaying multiple similar areas simultaneously, the similarity display unit 3150 may mark text and figures that show a high claim similarity score with claim 1 in yellow, and text and figures that show a high claim similarity score with claim 2 in green, for example. In other words, the colors may be changed according to the claim. Furthermore, bright colors may be used for areas showing high similarity, and light colors for areas showing moderate similarity, so that the degree of similarity can be grasped intuitively.

[0160] Figure 29 shows an example of how to display claim-supporting disclosures. In the example in Figure 29, the claim-supporting disclosures corresponding to claim 1 are clearly indicated in the specification and drawings using dotted lines. In the example in Figure 29, the color of the area within the dotted lines will change according to the claim similarity score.

[0161] Next, we will explain a search system that utilizes the CSU model. Assuming that the claim similarity score is normalized from 0 to 1, users can extract data by setting score conditions. For example, a high score condition of 0.9 or higher can extract text and figures that are strongly related. On the other hand, setting a condition of 0.6 or higher can include a wide range of related parts. This function enables efficient searching in initial screening and when dealing with large amounts of data. By limiting the search to parts with high claim similarity scores, the problem of losing motivation due to a large number of patent documents being hit while adjusting the search formula can be mitigated. In addition, since users can adjust the amount of data according to the research phase (e.g., initial research and detailed analysis) and the available time, efficient and flexible searching is achieved.

[0162] By using the search system described above, users can search only the parts of text and drawings (a small amount of data) that are strongly related to a particular claim, according to their search. This search system allows users to limit their search to the parts most relevant to their query. As a result, irrelevant patent search results are reduced, saving users time reviewing unnecessary data.

[0163] Furthermore, the database may be pre-stored for each of the multiple claim similarity scores. For example, the text and images extracted based on a claim similarity score of 0.9 or higher may be stored in the database, or for users who want a broader overview, the text and images extracted based on a claim similarity score of 0.6 or higher may be stored in the database. This allows users to make selections according to the screening phase and the total amount of time they can dedicate. By configuring the database separately for each claim similarity score, user access is distributed, and the response performance of the search system is improved.

[0164] Furthermore, the claim support score, described later, may be stored in a database, and filtering based on this score may be provided. For example, when searching for claims, the search scope may be limited to claims with a claim support score above a certain level. Alternatively, when searching the entire specification, the search may be limited to claims with a claim support score above a certain level, and then the search may be limited to the portion of text or figures with a similarity score above a certain level to those claims.

[0165] Next, we will explain how to extract unsupported claim portions. The claim is entered into the CSU model, and the generated claim support disclosure is divided into units based on a certain number of characters, punctuation marks, paragraphs, or phrases. Then, the parts with the highest similarity within the entire specification are searched for within these units. In some cases, no similar descriptions can be found in any part of the specification. In this case, the support requirements may not be met.

[0166] By dividing the claim support disclosure and comparing each unit with the entire specification, the parts where a corresponding description does not exist can be identified. This clearly indicates areas that may not meet the support requirements. Furthermore, by highlighting the unsupported areas with a warning color such as red, users can quickly identify the problematic areas and take appropriate action, such as revising the specification.

[0167] The claim support score is defined as the degree to which descriptions similar to the claim-supporting disclosures are found in the specification. This claim support score is defined as a numerical value that quantitatively evaluates the supportability of the claims in the specification. First, the claim-supporting disclosures in the patent document are divided into units of a certain number of characters, punctuation marks, paragraphs, or phrases. This division clarifies how each component of the claim-supporting disclosure is distributed within the specification. Next, each divided unit is compared with the entire specification to confirm whether corresponding content exists. To evaluate this similarity, the CSU model, which quantifies the content similarity between documents, is used to calculate the degree of agreement between the divided unit and the specification.

[0168] Next, the similarity to the specification is calculated for each divided unit. The similarity is normalized to a range of 0 to 1, and a value closer to 1 indicates that the portion contains sufficient explanation for the claim-supporting disclosure. This similarity serves as a basic evaluation criterion for determining whether the claims are adequately supported within the specification.

[0169] Furthermore, weights are assigned to the similarity of each unit. These weights are adjusted based on the number of characters or the length of phrases in each segment, with longer parts or parts containing detailed descriptions receiving higher weights. This takes into account the importance of each unit and reflects its contribution to the claim support score. Finally, these weighted similarities are summed up to obtain the overall weighted average, which derives the claim support score. This claim support score ranges from 0 to 1, with values ​​close to 1 indicating that the claim is adequately supported in the specification. Conversely, a low claim support score suggests that the description of the claim may be insufficient.

[0170] A ranking (sorting) based on this claim support score may be performed. In other words, claims or CSUs with low claim support scores may be considered less important. Therefore, patent search results may be ranked according to the claim support score.

[0171] Next, we will explain citation maps in CSU units. Examiners and inventors sometimes cite patent documents as prior art documents, and a map displaying these citation-cited relationships is called a citation map. The cited sections are often very narrow within the patent documents. Therefore, it is easier to handle if the unit of the citation map is smaller than the patent document unit. By creating a database that stores information in CSU units and a database that converts the citation-cited link information into CSU units, a citation map in CSU units can be created. When cited in a notice of reasons for rejection, the paragraph number of the cited patent document is listed. By extracting CSUs with a high degree of similarity to the paragraph indicated by this paragraph number, a citation map in CSU units can be created. Similarly, in IP landscape, the display may also be in CSU units instead of patent document units.

[0172] When viewing overviews such as citation maps or IP landscapes, summaries are displayed, but there are times when users want to view the actual specifications to confirm the content. In the conventional method of displaying information on a patent document basis, the amount of information contained in each patent document is often too large to fit into a sufficient summary, rendering the summary ineffective. On the other hand, by using a CSU (Certified Specification Unit)-based overview, the amount of information is reduced, allowing for accurate summary display. Even when viewing CSUs to confirm the details of the content, the small amount of data allows for quick and easy content review.

[0173] <Examples of "Notes"> In this embodiment, "notes" refers to a special form of description used to provide supplementary information in the specification. By including these "notes," various effects (hereinafter also referred to as "note effects") can be obtained, such as supplementing the main description, presenting exceptional circumstances or conditions, describing alternative embodiments, clarifying definitions and interpretations of terms, excluding later applications, or being usable as a basis for amendments. In this embodiment, "notes" are sentences beginning with "Furthermore," "Also," or "Moreover," but are not limited to these. For example, only sentences beginning with "Furthermore," or sentences beginning with at least one of "Furthermore," "Also," and "Moreover," may be used. Also, since general "notes" or short paragraphs often appear and are placed at the end of embodiments or at the end of each description, "notes" may also be defined as having these characteristics.

[0174] However, it has been pointed out that the "notes" in many patent documents are insufficiently written. Specifically, when a large number of patent documents containing ambiguous "notes" are included in the search results, a problem arises in that a great deal of time and effort is required to scrutinize them. This situation significantly reduces the efficiency of patent searches and creates the problem of requiring a great deal of time and effort to scrutinize their contents.

[0175] Therefore, in this embodiment, we will describe a function that automatically detects "notes" and evaluates the completeness of their content. Specifically, natural language processing technology is used to analyze the context of the "notes," and the relationship with other descriptions in the specification and the level of detail of the explanation are quantified. Based on this evaluation result, the user can exclude patents containing "notes" with insufficient explanations from the search results, or conversely, prioritize the display of "notes" with sufficient explanations.

[0176] First, we will explain the method using natural language processing techniques. If a relatively short sentence beginning with a conjunction such as "Furthermore," "Also," or "Moreover" contains the search keyword or key phrase, that sentence will be ignored. This process is applied to the entire text of a patent document, and if there are no more search-matching parts in the main text of the patent document, that patent document is considered to have been hit only by the footnotes and is excluded from the search results. The document length threshold may be set by the user.

[0177] Next, we will explain a method using a large-scale language model. Figure 30 shows an example of a prompt when using a large-scale language model. This prompt is used to determine whether a document contains a "note," and as an example, it shows the process of determining whether a hit patent document contains a "note" using the search keyword "lithium ion."

[0178] The prompt shown in Figure 30 clearly defines the characteristics and criteria for determining whether a document is a "note." This prompt includes the input format of the target text, the criteria for determining whether or not it is a "note," and the format for displaying the determination result. This allows for efficient and accurate "note" detection using a large-scale language model.

[0179] This prompt allows for a high rating to be given to "notes" that are closely related to the technical elements mentioned in the main text and include specific examples or numerical ranges. Conversely, "notes" that are not very relevant to the main text and lack specific explanations will receive a low rating. This rating can be used as a filtering criterion for search results, allowing users to efficiently access highly relevant patent information.

[0180] For example, if a particular keyword is found only in the "notes" section of a searched patent specification, the patent document can be excluded from the search results. This process is shown in Figure 31. Figure 31 is a flowchart of the "notes" determination and exclusion process performed by the information processing device, illustrating the procedure for efficiently narrowing down patent documents.

[0181] The information processing device obtains search results indicating one or more patent documents searched using keywords (step S601). Next, it obtains the specification of one patent document from the search results (hereinafter referred to as the "specification to be judged") (step S602), and performs a note determination on the specification to be judged (step S603). This "note determination" determines whether the specified keyword is included only in the "note".

[0182] If it is determined that the specified keyword is included only in the "notes" of the specification to be judged (Step S604: YES), the information processing device excludes the patent document corresponding to the specification to be judged from the search results (Step S605) and proceeds to Step S606. On the other hand, if it is determined that the keyword is included in something other than the "notes" (Step S604: NO), the information processing device proceeds directly to Step S606. The information processing device determines whether or not all patent documents shown in the search results have been determined to be "notes" (Step S606). If all patent documents have been determined to be "notes" (Step S606: YES), the information processing device terminates the process. On the other hand, if none of the patent documents have been determined to be "notes" (Step S606: NO), the information processing device obtains the specifications of the patent documents that have not been determined to be "notes" (Step S602) and proceeds to Step S603.

[0183] Next, we will explain the support for creating "notes." As mentioned above, evaluating the completeness of the notes can reduce the number of noisy patents found in patent searches, but it can also be used as a support function when drafting the specification. While "notes" often become noise in patent searches, the "note effect" mentioned above can be obtained, making the support for creating "notes" when drafting the specification very effective.

[0184] Therefore, in this embodiment, the above-described method for determining "notes" is applied to provide a specification drafting support function. Specifically, a large number of existing patent documents are analyzed, and major concepts and terms in the main text are extracted as pairs with typical corresponding "notes." By training a machine learning model or a large-scale language model with these pairs, efficient and highly accurate support for creating "notes" is achieved.

[0185] By using this trained model, it becomes possible to automatically suggest highly relevant "notes" when the specification drafter enters the main text. For example, if the main text describes "lithium-ion batteries," according to this embodiment, the following "notes" candidates will be suggested.

[0186] Candidate 1. "Furthermore, the lithium-ion battery of the present invention may use any of LiCoO2, LiNiO2, or LiMn2O4 as the positive electrode material." Option 2: "Furthermore, as the electrolyte, a solution can be obtained by dissolving solutes such as LiPF6, LiBF4, and LiClO4 in an organic solvent such as ethylene carbonate or dimethyl carbonate." Candidate 3: "Furthermore, the battery of the present invention can be implemented in various shapes, such as cylindrical, prismatic, and laminated."

[0187] By making such suggestions, specification drafters can easily add "notes," resulting in the "note effect." Furthermore, in this embodiment, it is possible to learn the trends of "notes" for each company and technology field, and to propose "notes" that are specific to that company or field. This supports the creation of more appropriate "notes" that are in line with each company's patent strategy.

[0188] Figure 32 is a flowchart showing the process flow for suggesting "notes" using a trained model. This process provides a function that automatically suggests highly relevant "notes" based on the target sentence entered by the user. This process is implemented using a model structure that combines an encoder and a decoder, and has a mechanism for generating appropriate "notes" based on the vector representation obtained from the target sentence.

[0189] The information processing device first obtains the target sentence provided by the user (step S701) and inputs the target sentence into the encoder (step S702). The information processing device obtains the vector representation output by the encoder according to the target sentence (step S703). The information processing device inputs the obtained vector representation into the decoder (step S704). The information processing device obtains the "notes" output by the decoder according to the vector representation (step S705). The information processing device outputs the obtained "notes" as a candidate (step S706) and terminates the process.

[0190] It is desirable that the "notes" be automatically generated according to the main theme of the specification. Therefore, in this embodiment, when keywords such as AUTOMORE are embedded, the notes may be automatically generated in accordance with the changes to the main text. Alternatively, the "notes" may be generated at a timing specified by the user, similar to the reference string update function of word processing software. Alternatively, a history of the past few "notes" may be kept for each AUTOMORE embedding location, and the user may select the one that best matches their intention from among the past few.

[0191] Next, we will explain the details of the model and training method for the proposed "note-taking" feature. As the base model for training, you can use a Transformer model, BERT, or a large-scale language model (such as Llama). Here, we will explain how to implement this using an encoder / decoder mechanism with a Transformer, and how to train it.

[0192] When implementing the "notes" model using an encoder-decoder mechanism, a process is employed in which the specification is encoded and appropriate "notes" are decoded and generated based on its content. In this approach, the encoder is responsible for extracting the content of the specification as a vector representation, and the decoder is responsible for generating supplementary information as "notes" based on that vector representation. This approach enables the generation of high-quality "notes" that reflect the technical context of the entire specification.

[0193] Figure 33 is a flowchart showing the process flow for training a model that generates "notes." The information processing device acquires a specification to be used for training (step S801), and acquires the main text of the specification and the corresponding "notes" section described in the same specification as training data, and labels them (step S802). It identifies supplementary explanatory sections in the specification that begin with conjunctions such as "Furthermore," "Also," and "Moreover," as "notes," and clarifies which part each "note" relates to. Document data from different technical fields and companies is also collected to add diversity to the dataset, enabling the generation of "notes" that can handle a wide range of contexts.

[0194] The information processing device inputs the text into the encoder (step S803). In the first stage of the learning process, the text of the specification is input into the encoder, and the encoder encodes the main concepts, technical elements, and related terms in the text. This encoder uses a Transformer-based model to convert the semantic features of the input text into a high-dimensional vector representation (step S804). This vector representation contains the main points and points that need to be supplemented with information in the text and serves as the basis for transmitting information related to "notes" to the decoder.

[0195] Next, the information processing device inputs the vector representation generated by the encoder into the decoder (step S805) and generates the "notes" by performing the task of generating "notes" (step S806). The decoder incorporates supplementary conjunctions and expressions specific to "notes," and is learned to generate supplementary information appropriate to the content of the patent document using conjunctions such as "furthermore" and "also." Based on the information extracted by the encoder, it generates specific "notes" phrases that are in line with the main content of the main text.

[0196] For learning, a typical supervised learning method for encoder-decoder models is used. Specifically, the text is input to the encoder, and the corresponding correct data's "notes" are output to the decoder for training. During training, the information processing device uses cross-entropy as the loss function, compares the generated "notes" with the correct "notes" (step S807), calculates the loss, and updates the model to minimize the error (step S808).

[0197] The model is trained by repeatedly performing steps S803 to S808 described above. Additionally, data augmentation using masking may be used to reinforce the learning process by masking some words in the "notes" text and adding a task to predict those parts, thereby ensuring that the specific terminology and conjunctions used in "notes" are used appropriately.

[0198] Furthermore, to accommodate different patent fields, fine-tuning is performed for each field to enable the generation of "notes" specific to particular technical fields. For example, in the chemical field, the model is adjusted to generate many "notes" indicating alternative materials or conditions, and the model is adapted to output supplementary information appropriate to each field.

[0199] In the evaluation phase, the quality of the generated "notes" is verified from the perspectives of relevance to the main text, specificity, and appropriateness of the expression within the context. The content of the generated "notes" is checked to see if it is actually useful as supplementary information to the patent, and if the generation quality is low, appropriate correction data or additional training is performed. By continuously incorporating user feedback and new patent documents, the model's generation capabilities are updated and improved to keep it up-to-date with the latest technologies and terminology.

[0200] This encoder-decoder approach makes the "notes" model a useful system as a support tool for generating appropriate "notes" for the specification and achieving the benefits of notes.

[0201] Next, we will describe an embodiment that supports searching using "notes". In this embodiment, a function is provided to extend search queries using a language model trained on "notes". This function makes it possible to extend the search query entered by the user to be more comprehensive and accurate, thereby improving search accuracy.

[0202] Figure 34 is a flowchart showing the process flow for expanding and searching a search query for patent documents. The information processing device receives a search query from the user (step S901) and inputs the input query into the "notes" learning model (step S902). The "notes" learning model predicts and generates the content of "notes" that may be related to the input query, and the information processing device retrieves the predicted and generated "notes" (step S903).

[0203] The information processing device extracts relevant keywords, synonyms, and related terms from the predicted and generated "notes" (step S904). The generated "notes" may be in the form of sentences (phrases). If it is a sentence (phrase), it can be used as is as a search phrase (key phrase), but since keywords are often easier for users to use, natural language processing is used to extract nouns and verbs and create a list of keywords.

[0204] The information processing device presents the user with a list of extracted keywords. The sentences (phrases) generated by the "notes" model may also be added directly to the list presented to the user. The information processing device determines the keywords and key phrases selected by the user from the list as an extended query (step S905). When adding keywords, OR operators can be used to increase the number of hits, or AND or DIFF operators can be used to prevent too many irrelevant patents from being found. The user may also be allowed to delete their original search query.

[0205] The information processing device, triggered by user instructions such as pressing a search execution button, executes a search using an extended query (step S906), displays the search results (step S907), and then terminates the process.

[0206] For example, if a user enters the query "lithium-ion battery," the information processing device will perform the following actions: User input query: "Lithium-ion battery" The following related notes were predicted and generated by the "Notes" learning model: "Furthermore, the lithium-ion battery of the present invention may use any of LiCoO2, LiNiO2, or LiMn2O4 as the positive electrode material. In addition, the electrolyte can be a solution of a solute such as LiPF6, LiBF4, or LiClO4 dissolved in an organic solvent such as ethylene carbonate or dimethyl carbonate." List of keywords to present to users: “Cathode material”, “LiCoO2”, “LiNiO2”, “LiMn2O4”, “Electrolyte”, “LiPF6”, “LiBF4”, “LiClO4”, “Solute”, “Ethylene carbonate”, “Dimethyl carbonate”, “Organic solvent”

[0207] By suggesting search keywords and key phrases in this way, users can become aware of related technologies, materials, and situations that they did not explicitly specify, reducing the likelihood of forgetting to search for them and enabling them to obtain more comprehensive search results.

[0208] Furthermore, this embodiment provides a function that allows the user to adjust the degree of query expansion. The user can select the strength of the expansion from radio buttons such as "weak," "medium," and "strong" located on the screen or in the settings menu, thereby optimizing the balance between the scope and accuracy of the search to suit their own needs.

[0209] For example, if the intensity is set to "strong," the information processing device instructs the "notes" learning model to "create 10 note phrases," and if the intensity is set to "weak," it instructs the "notes" model to "create 2 note phrases." In other words, the intensity can be controlled by changing the instruction for the number of phrases to generate according to the intensity.

[0210] <Example Hardware Configuration> The information processing devices 1000, 2000, 3000, 4000, 5000, and 6000 in one embodiment of the present disclosure may function as computers that perform the processing of the present disclosure. Figure 35 is a diagram showing an example of the hardware configuration of an information processing device according to one embodiment of the present disclosure. The above-described information processing device may be physically configured as a computer device including a processor 1001, memory 1002, storage 1003, communication device 1004, input device 1005, output device 1006, bus 1007, etc.

[0211] In the following explanation, the term "device" can be replaced with "circuit," "device," "unit," etc. The hardware configuration of the information processing device 100, the knowledge server 200, and the generation device 300 may include one or more of the devices shown in the figure, or it may be configured to omit some of the devices.

[0212] Each function in the information processing device is realized by loading predetermined software (programs) onto hardware such as the processor 1001 and memory 1002, which allows the processor 1001 to perform calculations, control communication by the communication device 1004, and control at least one of data reading and writing in the memory 1002 and storage 1003.

[0213] The processor 1001 controls the entire computer, for example, by running an operating system. The processor 1001 may consist of a central processing unit (CPU) that includes interfaces with peripheral devices, control units, arithmetic units, registers, and so on.

[0214] Furthermore, the processor 1001 reads programs (program code), software modules, data, etc., from at least one of the storage 1003 and the communication device 1004 into the memory 1002 and executes various processes accordingly. The program used is one that causes the computer to execute at least a part of the operations described in the above embodiment. For example, each part of the information processing device may be stored in the memory 1002 and implemented by a control program that runs on the processor 1001, and other functional blocks may be implemented similarly. The above-described various processes have been explained as being executed by one processor 1001, but they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The program may also be transmitted from a network via a telecommunications line.

[0215] Memory 1002 is a computer-readable recording medium and may consist of at least one of the following: ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. Memory 1002 may also be called a register, cache, main memory, etc. Memory 1002 can store executable programs (program code), software modules, etc., for carrying out a wireless communication method according to one embodiment of the present disclosure.

[0216] Storage 1003 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disc, a digital multipurpose disc, a Blu-ray® disc), a smart card, flash memory (e.g., a card, a stick, a key drive), a floppy® disk, a magnetic strip, etc. Storage 1003 may also be called an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, server, or other suitable medium including at least one of memory 1002 and storage 1003.

[0217] The communication device 1004 is hardware (transceiver / receiver device) for communicating between computers via at least one of a wired network and a wireless network, and is also called a network device, network controller, network card, communication module, etc.

[0218] The input device 1005 is an input device that accepts input from an external source (e.g., a keyboard, mouse, microphone, switch, button, sensor, etc.). The output device 1006 is an output device that outputs to an external source (e.g., a display, speaker, LED lamp, etc.). The input device 1005 and the output device 1006 may be configured as an integrated unit (e.g., a touch panel).

[0219] Furthermore, each device, such as the processor 1001 and memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or different buses may be configured for each device.

[0220] Furthermore, the information processing device may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array), and some or all of each functional block may be realized by such hardware. For example, processor 1001 may be implemented using at least one of these hardware components.

[0221] This disclosure can be implemented using software, hardware, or software integrated with hardware.

[0222] Each functional block used in the description of the above embodiments may be implemented partially or entirely as an integrated circuit (LSI), and each process described in the above embodiments may be controlled partially or entirely by a single LSI or a combination of LSIs. An LSI may consist of individual chips, or it may consist of a single chip that includes some or all of the functional blocks. An LSI may have data inputs and outputs. Depending on the degree of integration, LSIs may be referred to as ICs, system LSIs, super LSIs, or ultra LSIs.

[0223] The method of integration is not limited to LSIs; it may also be implemented using dedicated circuits, general-purpose processors, or dedicated processors. Furthermore, FPGAs (Field Programmable Gate Arrays) that can be programmed after LSI manufacturing, or reconfigurable processors that allow for the reconfiguration of the connections and settings of circuit cells within the LSI, may also be used. This disclosure may be implemented as digital or analog processing.

[0224] Furthermore, if advancements in semiconductor technology or related technologies lead to the emergence of integrated circuit technologies that replace LSIs, then naturally, these technologies can be used to integrate functional blocks. The application of biotechnology, for example, is a possible possibility.

[0225] Although various embodiments have been described above with reference to the drawings, it goes without saying that this disclosure is not limited to such examples. It is clear to those skilled in the art that various modifications or alterations can be conceived within the scope of the claims, and these will naturally also fall within the technical scope of this disclosure. Furthermore, the components in the above embodiments may be combined in any way without departing from the spirit of the disclosure.

[0226] The specific examples of this disclosure have been described in detail above, but these are merely illustrative and do not limit the scope of the claims. The technologies described in the claims include various modifications and changes to the specific examples described above. [Industrial applicability]

[0227] One embodiment of the present disclosure is suitable for an information processing device that appropriately assists in the preparation of documents related to a patent application. [Explanation of Symbols]

[0228] 7 Search Models 8. Coupling Section 300, 1300 Drawing section 1000, 2000, 3000, 4000, 5000, 6000 Information Processing Devices 1001 Processor 1002 memory 1003 Storage 1004 Communication device 1005 Input device 1006 Output device 1100 First Drawing Description Model 1130 Expert Switching Unit 2100 Second Drawing Description Model 2400 Drawing Description Model 3100 Claims Support Unit Model 3140 Similarity extraction section 3150 Similarity display section

Claims

1. A first drawing description model includes a vision encoder that converts an input drawing into an image feature vector, and a drawing description text encoder that describes the drawing and converts drawing description text written in a descriptive modeling language into a drawing description text feature vector. When the drawing description text is a drawing description text that describes the input drawing, the processing unit trains the first drawing description model so that the drawing description text feature vector and the drawing description text feature vector are similar. Equipped with an information processing device.

2. The information processing device described in claim 1 comprises a first drawing description model learned by the information processing device described in claim 1, The vision encoder converts the drawing into the image feature vector, The drawing description text encoder, which receives the image feature vector as the drawing description text feature vector, converts the input drawing description text feature vector into the drawing description text. Information processing device.

3. A first drawing description model learned by the information processing device described in claim 1, The unit includes a drawing unit that outputs a drawing from the aforementioned drawing description text, The drawing unit sorts the drawing description text into the symbol elements described in the patent drawing and the original description elements other than the symbol elements, and determines the display position of the original description elements. Information processing device.

4. The descriptive modeling language is different depending on the content of the drawing.

5. A drawing unit that converts the input drawing into a drawing description text written in a descriptive modeling language, which also explains the drawing. A second drawing description model includes a drawing description text encoder that converts the aforementioned drawing description text into a drawing description text feature vector, and a drawing description text encoder that converts drawing description text that describes a drawing into a drawing description text feature vector, When the drawing description text is a drawing description text that describes the input drawing, the processing unit trains the second drawing description model so that the drawing description text feature vector and the drawing description text feature vector are similar. Equipped with an information processing device.

6. The information processing device described in claim 5 comprises a second drawing description model learned by the information processing device, The drawing description text encoder converts the drawing description text into the drawing description text feature vector, The drawing description text encoder, which receives the drawing description text feature vector as the drawing description text feature vector, converts the input drawing description text feature vector into the drawing description text. Information processing device.

7. The information processing device described in claim 5 comprises a second drawing description model learned by the information processing device, The drawing description text encoder converts the drawing description text into a drawing description text feature vector. The drawing description text encoder, which receives the drawing description text feature vector as the drawing description text feature vector, converts the input drawing description text feature vector into the drawing description text. The drawing unit that converts the drawing description text into a drawing converts the drawing description text output by the drawing description text encoder into a drawing. Information processing device.

8. A search model is provided which includes a vision encoder that converts an input drawing into an image feature vector, and a text encoder that converts a drawing description text describing the drawing into a text feature vector, wherein, when the drawing description text is a drawing description text describing the input drawing, the text feature vector and the image feature vector are similar. A combining unit outputs a combined feature vector obtained by combining the image feature vector output by the vision encoder and the text feature vector output by the text encoder. Equipped with an information processing device.

9. The search target data, which includes the aforementioned drawing or the aforementioned drawing description text, is associated with the combined feature vector corresponding to the search target data. The information processing apparatus according to claim 8, which generates a search query feature vector from a user search query that indicates what the user wants to search for, and searches for a combined feature vector that is similar to the search query feature vector.

10. A second drawing description model learned by the information processing device described in claim 5, A combining unit outputs a combined feature vector obtained by combining the drawing description text feature vector output by the drawing description text encoder and the drawing explanation text feature vector output by the drawing explanation text encoder. Equipped with an information processing device.

11. The search target data, which includes the aforementioned drawing or the aforementioned drawing description text, is associated with the combined feature vector corresponding to the search target data. The information processing apparatus according to claim 10, which generates a search query feature vector from a user search query that indicates what the user wants to search for, and searches for a combined feature vector that is similar to the search query feature vector.

12. A claim support unit model includes a claim text encoder that converts the input claim text into a claim text feature vector, and a support text encoder that converts the supporting text that represents the description and drawings in the specification that support the claim into a support text feature vector, A processing unit that trains the claim support unit model such that the claim text feature vector and the support text feature vector are similar, when the support text is the support text that supports the input claim. Equipped with an information processing device.

13. The information processing device described in claim 12 comprises a claim support unit model learned by the device, The claimed text encoder converts the claimed text into a claimed text feature vector, The support text encoder, which receives the claimed text feature vector as the support text feature vector, converts the input support text feature vector into the support text. A text conversion unit converts the aforementioned support text into descriptions and drawings in the specification that support the claims, converts the support text output by the support text encoder into descriptions and drawings in the specification. Information processing device.

14. The information processing device described in claim 12 comprises a claim support unit model learned by the device, The support text encoder converts the support text into the support text feature vector, A claim text encoder, which receives the support text feature vector as the claim text feature vector, converts the input claim text feature vector into the claim text. Information processing device.

15. The information processing apparatus according to claim 14, wherein the supporting text includes special tokens, the special tokens indicating paragraphs in the specification, figure numbers, mathematical formulas, tables, or chemical formulas shown in images, or symbols.

16. The information processing device described in claim 12 comprises a claim support unit model learned by the device, From the claim text, the support text is output using the claim support unit model, the support text is converted into a claim support disclosure including descriptions and drawings in the specification, portions similar to the claim support disclosure are extracted, and a score indicating the degree of similarity is displayed. Information processing device.

17. A claim support unit model learned by the information processing device described in claim 12, A coupling unit that outputs a combined feature vector obtained by combining the claimed text feature vector output by the claimed text encoder and the support text feature vector output by the support text encoder, Equipped with an information processing device.

18. The search data including the claim or the claim-supporting disclosure is associated with the combined feature vector corresponding to the search data, The information processing apparatus according to claim 17, which generates a search query feature vector from a user search query that indicates what the user wants to search for, and searches for a combined feature vector that is similar to the search query feature vector.

19. An information processing device that obtains the main text of a specification and any notes contained in the same specification, inputs the main text into a learning model, obtains notes corresponding to the main text from the learning model, and uses the main text of the specification and the same notes contained in the specification as ground truth data to perform supervised learning on the learning model.

20. The learning model described in claim 19 is provided, A search query for searching patent documents is input into a learning model, keywords related to the search are extracted from the notes obtained from the learning model, and the search query is expanded using the extracted keywords. Information processing device.

21. A method for controlling an information processing device, A first drawing description model includes a vision encoder that converts an input drawing into an image feature vector, and a drawing description text encoder that describes the drawing and converts drawing description text written in a descriptive modeling language into a drawing description text feature vector. The model is trained so that when the drawing description text is drawing description text that describes the input drawing, the drawing description text feature vector and the drawing description text feature vector become similar. Control method.

22. A control method for an information processing device comprising a first drawing description model learned by the information processing device described in claim 1, The vision encoder converts the drawing into the image feature vector, The drawing description text encoder, which receives the image feature vector as the drawing description text feature vector, converts the input drawing description text feature vector into the drawing description text. Control method.

23. A control method for an information processing device comprising a first drawing description model learned by the information processing device described in claim 1, The drawing unit outputs a drawing from the drawing description text, The drawing unit sorts the drawing description text into the symbol elements described in the patent drawing and the original description elements other than the symbol elements, and determines the display position of the original description elements. Control method.

24. A method for controlling an information processing device, The drawing unit converts the input drawing into drawing description text written in a descriptive modeling language that explains the drawing. A second drawing description model includes a drawing description text encoder that converts the aforementioned drawing description text into a drawing description text feature vector, and a drawing description text encoder that converts the drawing description text, which contains sentences describing the drawing, into a drawing description text feature vector. The model is trained so that when the drawing description text is drawing description text that describes the input drawing, the drawing description text feature vector and the drawing description text feature vector become similar. Control method.

25. A control method for an information processing device comprising a second drawing description model learned by the information processing device described in claim 5, The drawing description text encoder converts the drawing description text into the drawing description text feature vector, The drawing description text encoder, which receives the drawing description text feature vector as the drawing description text feature vector, converts the input drawing description text feature vector into the drawing description text. Control method.

26. A control method for an information processing device comprising a second drawing description model learned by the information processing device described in claim 5, The drawing description text encoder converts the drawing description text into a drawing description text feature vector. The drawing description text encoder, which receives the drawing description text feature vector as the drawing description text feature vector, converts the input drawing description text feature vector into the drawing description text. The drawing unit that converts the drawing description text into a drawing converts the drawing description text output by the drawing description text encoder into a drawing. Control method.

27. A method for controlling an information processing device, The system includes a vision encoder that converts an input drawing into an image feature vector, and a text encoder that converts a drawing description text into a text feature vector, wherein, when the drawing description text is a drawing description text that describes the input drawing, it outputs a combined feature vector obtained by combining the image feature vector output by the vision encoder and the text feature vector output by the text encoder in a search model that has been trained so that the text feature vector and the image feature vector are similar. Control method.

28. A control method for an information processing device comprising a second drawing description model learned by the information processing device described in claim 5, The system outputs a combined feature vector by combining the drawing description text feature vector output by the drawing description text encoder and the drawing explanation text feature vector output by the drawing explanation text encoder. Control method.

29. A method for controlling an information processing device, A claim support unit model includes a claim text encoder that converts claim text representing an input claim into a claim text feature vector, and a support text encoder that converts supporting text representing descriptions and drawings in the specification that support the claim into a support text feature vector, and is trained so that the claim text feature vector and the support text feature vector are similar when the support text is supporting the input claim. Control method.

30. A control method for an information processing device comprising a claim support unit model learned by the information processing device described in claim 12, The claimed text encoder converts the claimed text into a claimed text feature vector, The support text encoder, which receives the claimed text feature vector as the support text feature vector, converts the input support text feature vector into the support text. A text conversion unit converts the aforementioned support text into descriptions and drawings in the specification that support the claims, converts the support text output by the support text encoder into descriptions and drawings in the specification. Control method.

31. A control method for an information processing device comprising a claim support unit model learned by the information processing device described in claim 12, The support text encoder converts the support text into the support text feature vector, A claim text encoder, which receives the support text feature vector as the claim text feature vector, converts the input claim text feature vector into the claim text. Control method.

32. A control method for an information processing device comprising a claim support unit model learned by the information processing device described in claim 12, From the claim text, the support text is output using the claim support unit model, the support text is converted into a claim support disclosure including descriptions and drawings in the specification, portions similar to the claim support disclosure are extracted, and a score indicating the degree of similarity is displayed. Control method.

33. A control method for an information processing device comprising a claim support unit model learned by the information processing device described in claim 12, The system outputs a combined feature vector obtained by combining the claimed text feature vector output by the claimed text encoder and the support text feature vector output by the support text encoder. Control method.

34. A method for controlling an information processing device, The main text of the specification and any notes included in the same specification are obtained, the main text is input into a learning model, the notes corresponding to the main text are obtained from the learning model, and the learning model is trained in a supervised manner using the main text of the specification and the same notes included in the specification as ground truth data. Control method.

35. The computer that constitutes the information processing device A first drawing description model includes a vision encoder that converts an input drawing into an image feature vector, and a drawing description text encoder that describes the drawing and converts drawing description text written in a descriptive modeling language into a drawing description text feature vector. The model is trained so that when the drawing description text is drawing description text that describes the input drawing, the drawing description text feature vector and the drawing description text feature vector become similar. A program to execute a process.

36. A computer comprising an information processing device equipped with a first drawing description model learned by the information processing device described in claim 1, The vision encoder converts the drawing into the image feature vector, The drawing description text encoder, which receives the image feature vector as the drawing description text feature vector, converts the input drawing description text feature vector into the drawing description text. A program to execute a process.

37. A computer comprising an information processing device equipped with a first drawing description model learned by the information processing device described in claim 1, The drawing unit outputs a drawing from the drawing description text, The drawing unit sorts the drawing description text into the symbol elements described in the patent drawing and the original description elements other than the symbol elements, and determines the display position of the original description elements. A program to execute a process.

38. To the computer that makes up the information processing device. The drawing unit converts the input drawing into drawing description text written in a descriptive modeling language that explains the drawing. A second drawing description model includes a drawing description text encoder that converts the aforementioned drawing description text into a drawing description text feature vector, and a drawing description text encoder that converts the drawing description text, which contains sentences describing the drawing, into a drawing description text feature vector. The model is trained so that when the drawing description text is drawing description text that describes the input drawing, the drawing description text feature vector and the drawing description text feature vector become similar. A program to execute a process.

39. A computer comprising an information processing device equipped with a second drawing description model learned by the information processing device described in claim 5, The drawing description text encoder converts the drawing description text into the drawing description text feature vector, The drawing description text encoder, which receives the drawing description text feature vector as the drawing description text feature vector, converts the input drawing description text feature vector into the drawing description text. A program to execute a process.

40. A computer comprising an information processing device equipped with a second drawing description model learned by the information processing device described in claim 5, The drawing description text encoder converts the drawing description text into a drawing description text feature vector. The drawing description text encoder, which receives the drawing description text feature vector as the drawing description text feature vector, converts the input drawing description text feature vector into the drawing description text. The drawing unit that converts the drawing description text into a drawing converts the drawing description text output by the drawing description text encoder into a drawing. A program to execute a process.

41. The computer that constitutes the information processing device The system includes a vision encoder that converts an input drawing into an image feature vector, and a text encoder that converts a drawing description text into a text feature vector, wherein, when the drawing description text is a drawing description text that describes the input drawing, it outputs a combined feature vector obtained by combining the image feature vector output by the vision encoder and the text feature vector output by the text encoder in a search model that has been trained so that the text feature vector and the image feature vector are similar. A program to execute a process.

42. A computer comprising an information processing device equipped with a second drawing description model learned by the information processing device described in claim 5, The system outputs a combined feature vector by combining the drawing description text feature vector output by the drawing description text encoder and the drawing explanation text feature vector output by the drawing explanation text encoder. A program to execute a process.

43. The computer that constitutes the information processing device A claim support unit model includes a claim text encoder that converts claim text representing an input claim into a claim text feature vector, and a support text encoder that converts supporting text representing descriptions and drawings in the specification that support the claim into a support text feature vector, and is trained so that the claim text feature vector and the support text feature vector are similar when the support text is supporting the input claim. A program to execute a process.

44. A computer comprising an information processing device equipped with a claim support unit model learned by the information processing device described in claim 12, The claimed text encoder converts the claimed text into a claimed text feature vector, The support text encoder, which receives the claimed text feature vector as the support text feature vector, converts the input support text feature vector into the support text. A text conversion unit converts the aforementioned support text into descriptions and drawings in the specification that support the claims, converts the support text output by the support text encoder into descriptions and drawings in the specification. A program to execute a process.

45. A computer comprising an information processing device equipped with a claim support unit model learned by the information processing device described in claim 12, The support text encoder converts the support text into the support text feature vector, A claim text encoder, which receives the support text feature vector as the claim text feature vector, converts the input claim text feature vector into the claim text. A program to execute a process.

46. A computer comprising an information processing device equipped with a claim support unit model learned by the information processing device described in claim 12, From the claim text, the support text is output using the claim support unit model, the support text is converted into a claim support disclosure including descriptions and drawings in the specification, portions similar to the claim support disclosure are extracted, and a score indicating the degree of similarity is displayed. A program to execute a process.

47. A computer comprising an information processing device equipped with a claim support unit model learned by the information processing device described in claim 12, The system outputs a combined feature vector obtained by combining the claimed text feature vector output by the claimed text encoder and the support text feature vector output by the support text encoder. A program to execute a process.

48. The computer that constitutes the information processing device The main text of the specification and any notes included in the same specification are obtained, the main text is input into a learning model, the notes corresponding to the main text are obtained from the learning model, and the learning model is trained in a supervised manner using the main text of the specification and the same notes included in the specification as ground truth data. A program to execute a process.

Images