Programs, methods, information processing devices, and systems

Abstract

To provide a program, method, information processing device, and system that enable efficient investigation. [Solution] The system receives survey conditions including text related to the technology to be investigated, analyzes the received technology by inputting it into a large-scale language model to generate search conditions, combines the generated search conditions to create a search expression in the first database, performs a search in the first database using the created search expression to obtain search results and evaluate the search expression, adjusts the search conditions based on the survey conditions received in the acceptance step to create a search expression in the second database.

Description

Technical Field

[0001] The present disclosure relates to a program, a method, an information processing apparatus, and a system. The technology according to the present disclosure is not limited to patent document searches, and is generally applicable to search expression creation support when searching a database based on specific text information such as academic documents, technical reports, and in-house documents.

Background Art

[0002] Business operators handling technologies conduct patent searches to prevent patent right infringements, properly secure rights for developed technologies, and invalidate patent rights held by competitors and the like. In patent searches, it is necessary to determine a search theme based on the search purpose obtained from the requester, define appropriate search conditions from the determined search theme, and combine them appropriately to create a search expression. Then, a search is performed using various databases with the generated search expression, a list of search results and the like are obtained, then peer review is performed on the database, and the search results such as the degree of match in relation to the search purpose are compiled and reported.

[0003] In such patent searches, each step is important and highly difficult, but creating a search expression is particularly highly difficult and challenging for engineers and the like. For example, Patent Document 1 discloses a method for performing a patent search by presenting similar words and superordinate / subordinate concepts as candidates based on input keywords, removing duplicates, and then logically combining search expressions. A mechanism for excluding or restoring a plurality of related terms is also provided. As a result, many candidates are presented at once, and appropriate conditions can be set from among them.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Thus, while the patent search formula generation device disclosed in Patent Document 1 has the advantage of allowing users to efficiently create search formulas without having to think for themselves due to the abundance of search conditions presented, general users, such as engineers unfamiliar with creating search formulas, may be confused by these selections. Furthermore, although it is equipped with a filtering function that combines with patent classification codes, specialized knowledge is required to select the appropriate patent classification. For this reason, it is difficult for general users to select and discard candidates, and it takes time to reach sufficient search results.

[0006] Furthermore, generating search queries using such automated methods may not create an appropriate search population, making it unclear how many relevant documents will be included in the search results. Therefore, if more results are found than anticipated, it may exceed the estimated work time and budget, preventing proper peer review. On the other hand, if the search query lacks or is too restrictive, fewer documents may be found than expected, or relevant documents may not be extracted, resulting in missed results. In particular, directly experimenting with search queries in the databases that are ultimately the target of the investigation (for example, public databases with high comprehensiveness but no API access, or those with slow search interfaces) presents challenges due to time constraints and operational complexity. There were also technical information processing challenges, such as the extensive trial and error involved in creating search queries, variability in search result quality, and dependence on the skills of the personnel involved.

[0007] Therefore, this disclosure provides a program, method, information processing device, and system that can resolve these technical challenges and enable the creation of search formulas that allow for efficient investigation. [Means for solving the problem]

[0008] To solve the above problems, according to one aspect of this disclosure, a program for operating a computer having a processor and memory is provided, the program causing the processor to execute: a research condition reception step of receiving research conditions including text relating to a technology under investigation; a search condition generation step of generating search conditions by inputting the technology under investigation received in the research condition reception step into a large-scale language model for analysis; a first search expression creation step of creating a search expression in a first database by combining the search conditions generated in the search condition generation step; an acquisition evaluation step of performing a search in the first database using the search expression created in the first search expression creation step to obtain search results and evaluate the search expression; and a second search expression creation step of creating a search expression in a second database by adjusting the search conditions generated in the search condition generation step based on the research conditions received in the research condition reception step.

[0009] According to another aspect of this disclosure, a method may be provided which is performed by a computer having a processor, the processor performing: a research condition reception step of receiving research conditions including text relating to a technology under investigation; a search condition generation step of generating search conditions by inputting the technology under investigation received in the research condition reception step into a large-scale language model for analysis; a first search expression creation step of creating a search expression in a first database by combining the search conditions generated in the search condition generation step; an acquisition evaluation step of performing a search in the first database using the search expression created in the first search expression creation step to obtain search results and evaluate the search expression; and a second search expression creation step of creating a search expression in a second database by adjusting the search conditions generated in the search condition generation step based on the research conditions received in the research condition reception step.

[0010] In yet another aspect of this disclosure, an information processing device is provided, the control unit performing: a research condition reception step of receiving research conditions including text relating to a technology under investigation; a search condition generation step of generating search conditions by inputting the technology under investigation received in the research condition reception step into a large-scale language model for analysis; a first search expression creation step of creating a search expression in a first database by combining the search conditions generated in the search condition generation step; an acquisition evaluation step of performing a search in the first database using the search expression created in the first search expression creation step to obtain search results and evaluate the search expression; and a second search expression creation step of creating a search expression in a second database by adjusting the search conditions generated in the search condition generation step based on the research conditions received in the research condition reception step.

[0011] In yet another aspect of this disclosure, a system may be provided comprising: a survey condition receiving means for receiving survey conditions including text relating to a technology under investigation; a search condition generating means for generating search conditions by inputting the technology under investigation received by the survey condition receiving means into a large-scale language model for analysis; a first search expression creation means for creating a search expression in a first database by combining the search conditions generated by the search condition generating means; an acquisition and evaluation means for performing a search in the first database using the search expression created by the first search expression creation means to obtain search results and evaluate the search expression; and a second search expression creation means for creating a search expression in a second database by adjusting the search conditions generated by the search condition generating means based on the survey conditions received by the survey condition receiving means. [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram showing the functional configuration of System 1. [Figure 2] This is a block diagram showing the functional configuration of the terminal device 10. [Figure 3]This is a block diagram illustrating the functional configuration of the server 20. [Figure 4] This is a diagram illustrating the data structure of an investigation case management table 2021. [Figure 5] This is a diagram illustrating an example of data in the investigation case management table 2021. [Figure 6] This is a diagram illustrating the data structure of an investigation condition table 2022. [Figure 7] This is a diagram illustrating a specific example of data in the investigation condition table 2022. [Figure 8] This is a diagram illustrating the data structure of a search condition table 2023. [Figure 9] This is a diagram illustrating a specific example of data in the search condition table 2023. [Figure 10] This is a diagram illustrating the data structure of a search expression table 2024. [Figure 11] This is a diagram illustrating a specific example of data in the search expression table 2024. [Figure 12] This is a diagram illustrating the data structure of a search execution log table 2025. [Figure 13] This is a diagram illustrating a specific example of data in the search execution log table 2025. [Figure 14] This is a diagram illustrating the data structure of a system evaluation table 2026. [Figure 15] This is a diagram illustrating a specific example of data in the system evaluation table 2026. [Figure 16] This is a diagram illustrating the data structure of a user evaluation table 2027. [Figure 17] This is a diagram illustrating a specific example of data in the user evaluation table 2027. [Figure 18] This is a diagram illustrating the data structure of a user table 2028. [Figure 19] This is a diagram illustrating a specific example of data in the user table 2028. [Figure 20] This is a diagram illustrating the data structure of a patent classification table 2029. [Figure 21] This is a diagram illustrating a specific example of data in the patent classification table 2029. [Figure 22] This is the first part of the flowchart of the search expression creation process. [Figure 23] This is the part of the flowchart of the search expression creation process that follows Figure 22. [Figure 24] This is the part of the flowchart of the search expression creation process that follows Figure 23. [Figure 25] This is a schematic diagram showing an example of an investigation condition input screen. [Figure 26] This is a schematic diagram showing an example of a search expression display and evaluation screen. [Figure 27] This is a schematic diagram showing an example of a search result display screen. [Figure 28] This is a block diagram showing the basic hardware configuration of a computer 90. [Modes for Carrying Out the Invention]

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In all the drawings for describing the embodiments, common constituent elements are denoted by the same reference numerals, and repeated descriptions are omitted. Note that the following embodiments do not unduly limit the content of the present disclosure described in the claims. In addition, not all the constituent elements shown in the embodiments are necessarily essential constituent elements of the present disclosure. Each drawing is a schematic diagram and is not necessarily illustrated strictly.

[0014] In the following description, a "processor" refers to one or more processors. At least one processor is typically a microprocessor such as a CPU (Central Processing Unit), but may be another type of processor such as a GPU (Graphics Processing Unit). The at least one processor may be single-core or multi-core.

[0015] Furthermore, at least one processor may be a broad-sense processor, such as a hardware circuit that performs some or all of the processing (e.g., an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit)).

[0016] Furthermore, in the following explanation, we may use expressions such as "xxx table" to describe information that yields an output for a given input. This information can be data with any structure, or it can be a learning model such as a neural network that generates an output for a given input. Therefore, "xxx table" can be referred to as "xxx information."

[0017] Furthermore, in the following explanation, the structure of each table is just an example; one table may be divided into two or more tables, or all or part of two or more tables may constitute a single table.

[0018] Furthermore, in the following explanation, the subject of the process may sometimes be "program," but since a program is executed by a processor and performs defined processes using the memory and / or interface as appropriate, the subject of the process may also be the processor (or a device such as a controller that has that processor).

[0019] The program may be installed on a device such as a computer, or it may reside on a program distribution server or a computer-readable (e.g., non-temporary) recording medium. Furthermore, in the following description, two or more programs may be implemented as a single program, or one program may be implemented as two or more programs.

[0020] Furthermore, in the following explanation, identification numbers are used as identification information for various objects, but other types of identification information (for example, identifiers including letters or symbols) may also be used.

[0021] Furthermore, in the following explanations, when describing similar elements without distinction, a reference code (or a common code among reference codes) may be used, and when describing similar elements with distinction, the element's identification number (or reference code) may be used.

[0022] Furthermore, in the following explanation, only control lines and information lines deemed necessary for the explanation are shown, and not all control lines and information lines in the product are necessarily shown. All components may be interconnected.

[0023] Each information processing device consists of a computer equipped with an arithmetic unit and a memory device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by said hardware configuration will be described later. For each of the terminal device 10 and the server 20, explanations that overlap with the basic hardware configuration and basic functional configuration of the computer described later will be omitted.

[0024] <Structure> The configuration of the information processing system according to the first embodiment will now be described. Figure 1 is a block diagram illustrating the configuration of the information processing system according to the first embodiment.

[0025] As shown in Figure 1, the information processing system 1 (hereinafter simply referred to as "System 1") includes a terminal device 10, a server 20 which is an information processing device, a large-scale language model 40, a first database 50, and a second database 60. However, the physical implementation is not limited to this, and for example, one terminal device 10 may have all or part of the functions of the other devices, or one server 20 may have all or part of the functions of the other devices excluding the terminal device 10.

[0026] In Figure 1, the number of terminal devices 10 included in System 1 is not limited to one. There may be two or more terminal devices 10 included in System 1.

[0027] Figure 1 shows an example where System 1 includes one server 20, but the number of servers 20 included in System 1 is not limited to one. Server 20 may consist of multiple servers depending on the functions they have. Also, for example, each server 20 may be a collection of multiple devices. The way in which the multiple functions required to implement the server 20 according to this embodiment are distributed to one or more hardware can be appropriately determined in consideration of the processing capacity of each hardware and / or the specifications required for the server 20.

[0028] The terminal device 10 shown in Figure 1 can be implemented, for example, by a stationary PC (Personal Computer) or a laptop PC. Alternatively, the terminal device 10 may be implemented by a mobile device such as a smartphone or tablet that supports a mobile communication system. Furthermore, the terminal device 10 may be implemented by a wearable device such as an HMD (Head Mount Display).

[0029] The terminal device 10 comprises a communication interface 12, an input device 13, an output device 14, memory 15, storage 16, and a processor 19. The input device 13 is a device for receiving input operations from the user (e.g., a touch panel, touchpad, etc.). The output device 14 is a device for presenting information to the user (e.g., a display, speaker, etc.).

[0030] The server 20 is implemented, for example, by an information processing device connected to the network 80. As shown in Figure 1, the server 20 includes a communication IF 22, an I / O IF 23, memory 25, storage 26, and a processor 29. The I / O IF 23 functions as an interface for an input device to receive input operations from the user and an output device to present information to the user.

[0031] The Large Language Model (LLM) 40 is an artificial intelligence model executed by one or more information processing devices connected to the network 80. In this disclosure, "Large Language Model (LLM)" is used as a concept that includes so-called Generative AI, which has the ability to process data such as text, images, and audio and generate new content. The LLM 40 generates output based on user input (prompt text, images, etc.) using a pre-trained model. Examples of LLM 40 include ChatGPT, Claude, LLaMA, Gemini, and Grok. The LLM 40 may also be a model capable of processing multiple types of data such as text, images, audio, and video all at once. In this disclosure, a large language model that primarily outputs text information as output information is described as a type of LLM 40. Furthermore, the LLM 40 may be incorporated into the server 20 as a local LLM. The large-scale language model used in this disclosure may be an external service accessed via network 80 (e.g., OpenAI's GPT series API), or it may be a locally running model (local LLM) built on server 20 or a secure internal network. In the latter case, processing can be performed without transmitting confidential information such as the text to be investigated to an external party.

[0032] This disclosure particularly utilizes the natural language understanding and generation capabilities of the LLM40. Specifically, the LLM40 receives instructions (prompts) and contextual information (e.g., text about the technology under investigation, investigation type, existing search criteria candidates, etc.) as input from the server 20, typically via an API (Application Programming Interface). Based on the prompts, the LLM40 is expected to process the input contextual information, perform some or more of the following functions, and return the results (generated text, score, structured data, etc.) to the server 20. (a) Structural and semantic analysis of the text: Analyze the text under investigation to identify or classify elements such as the technical field, the problem to be solved, the constituent elements of the invention, and the effects. (b) Keyword and concept extraction: From the text, keywords or key phrases are extracted that are considered important in the patent search, such as technical terms, functional features, component names, materials, processes, and standard names. (c) Related word expansion (keyword expansion): For extracted or specified keywords, synonyms, related words, abbreviations, higher-level concepts, lower-level concepts, related concepts, and alternative spellings (e.g., English translations, variations in spelling) are generated to provide a variety of search term candidates to improve the comprehensiveness and accuracy of the search. (d) Patent classification estimation: Based on the content of the input text, the system estimates and proposes patent classification codes such as the International Patent Classification (IPC), FI, and F-terms that are considered to be highly relevant. (e) Relevance and Similarity Assessment: Evaluate and score the extent to which the generated search criteria candidates and the literature obtained as search results are technically related to or semantically similar to the original research criteria (e.g., the text to be researched). (f) Search expression syntax generation (modified): Based on natural language instructions or lists of keywords and classifications, a search expression string is generated that conforms to the syntax rules of a specific database (first database 50 or second database 60), such as search operators and field specifications.

[0033] Because LLM40's behavior can be flexibly controlled through prompt engineering, Server 20 can optimize the output content and format of LLM40 by providing appropriate instructions according to the purpose of the investigation (e.g., investigation type, target number of results) and context. As for the LLM40 model to be used, it is possible to select models that have been fine-tuned for specific tasks (e.g., patent document analysis, understanding scientific and technical literature), or multimodal models that can understand and process information contained not only in text but also in drawings (images). When selecting a model to actually use, it is desirable to comprehensively consider performance (accuracy, creativity, instruction followability, hallucination suppression), response speed, processing cost, ease of API use, provider's terms of service (especially data privacy policy and commercial use conditions), supported languages, and local environment setup and execution feasibility, and to selectively use, or combine, the model that best suits the requirements and operational policies of this system and the characteristics of individual investigation cases (e.g., confidentiality). As of April 2025, examples of available models include OpenAI's GPT series, Google's Gemini series, Anthropic's Claude series, Meta's LLaMA series, Mistral AI's models, and xAI's Grok. However, the available models may change as technology advances, and are not limited to these. Depending on the requirements and confidentiality of the system, it is also possible to select a lightweight open model that can operate in a local environment, such as Google's Gemma series.

[0034] The first database 50 and the second database 60 are databases that store information to be searched, such as patent documents, academic literature, and other technical documents. In the following description, the first database 50 and the second database 60 will be specifically described as patent search databases. As shown in Figure 1, the first database 50 and the second database 60 are configured to be accessible from the server 20 via the network 80. However, this disclosure is not limited to patent document databases; it is also applicable to creating search queries in other types of databases that are subject to text search, such as academic paper databases, technical article databases, or corporate technical document databases. The first database 50 and the second database 60 have the function of searching the stored information based on the search query transmitted from the server 20 and returning the search results (e.g., the number of hits and a list of documents) to the server 20. In this embodiment, the first database 50 contains comprehensive information and is used to perform a preliminary evaluation of the first search formula (such as checking the number of hits), while the second database 60 is a commercial database suitable for a specific research purpose and is used to perform the main search (obtaining a list of literature) using the second search formula created after evaluation and adjustment.

[0035] Furthermore, the first database 50 does not necessarily have to be located outside the server 20. For example, to improve processing efficiency, it may be constructed using tables or databases (not shown) located in the storage unit 202 (see Figure 3) described later within the server 20, and a search module (illustrated) located in the control unit 203 (see Figure 3). The scope of the bibliographic information stored in the first database 50 can also take various forms depending on the purpose. For example, for a specific research purpose (such as checking the clearance of one's own technology), it may be a database containing only patent document data where the user (or their affiliated organization) is the applicant, or a list of the number of patent classifications assigned.

[0036] Alternatively, if the purpose is to analyze the trends of competitors, the database may include patent document data for specific competitors or client companies in addition to the user's own applicants. Furthermore, if the research is limited to a specific business field, the first database 50 may be a database that includes only data for the technical fields related to that business (for example, all documents belonging to a specific related IPC subclass group).

[0037] Furthermore, in this embodiment, the first database 50 and the second database 60 are primarily assumed to be different databases depending on their roles (preliminary evaluation and main search), but they do not necessarily have to be physically or logically separate databases. In other words, the first database 50 and the second database 60 may be the same database. In that case, the server 20 will use different access methods depending on the stage and purpose of the search, such as first performing a preliminary evaluation (checking the number of hits, etc.) using the first search formula on the same database, and then performing the main search (obtaining a list of references) using the second search formula that has been adjusted and created based on the results.

[0038] Each of the information processing devices described above is composed of a computer equipped with an arithmetic unit and a memory device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by said hardware configuration will be described later. For each of the terminal device 10, server 20, LLM 40, first database 50, and second database 60, explanations that overlap with the basic hardware configuration and basic functional configuration of the computer described later will be omitted.

[0039] <Configuration of terminal device 10> Figure 2 is a block diagram showing an example configuration of the terminal device 10 shown in Figure 1. As shown in Figure 2, the terminal device 10 comprises a communication unit 120, an input device 13, an output device 14, a storage unit 180, and a control unit 190. Each block included in the terminal device 10 is electrically connected, for example, by a bus.

[0040] The communication unit 120 performs processing such as modulation and demodulation processing for the terminal device 10 to communicate with other devices. The communication unit 120 performs transmission processing on the signal generated by the control unit 190 and transmits it to an external source (for example, the server 20). The communication unit 120 performs reception processing on the signal received from an external source and outputs it to the control unit 190.

[0041] The input device 13 is a device for a user operating the terminal device 10 to input instructions or information. The input device 13 can be implemented as, for example, a keyboard or mouse. The input device 13 may also be implemented as a touch-sensitive device that inputs instructions by touching an operating surface. The input device 13 converts the instructions input by the user into electrical signals and outputs the electrical signals to the control unit 190. The input device 13 may also include, for example, a receiving port that accepts electrical signals input from an external input device.

[0042] The output device 14 is a device for presenting information to the user operating the terminal device 10. The output device 14 is implemented by, for example, a display or a speaker. The display, which is the output device 14, displays data according to the control of the control unit 190. The output device 14 as a display is implemented by, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

[0043] The storage unit 180 is implemented by, for example, memory 15 and storage 16, and stores data and programs used by the terminal device 10. The storage unit 180 stores, for example, user information 181.

[0044] User information 181 stores information about the user performing the operation. User information may include, for example, a user ID and password for accessing the system provided by server 20, as well as information about the user such as company name, name, and affiliation.

[0045] The control unit 190 is realized when the processor 19 reads a program stored in the memory unit 180 and executes instructions contained in the program. The control unit 190 controls the operation of the terminal device 10. By operating according to the program, the control unit 190 performs the functions of an operation reception unit 191, a transmission / reception unit 192, and a presentation control unit 193.

[0046] The operation reception unit 191 processes instructions or information input from the input device 13. For example, the operation reception unit 191 receives instructions or information input from the input device 13, etc. The transmission / reception unit 192 processes data for the terminal device 10 to send and receive data with an external device such as the server 20 according to a communication protocol. Specifically, for example, the transmission / reception unit 192 sends instructions input by the user to the server 20. The transmission / reception unit 192 receives information provided by the server 20. The presentation control unit 193 controls the output device 14, etc., to present information such as information provided by the server 20 to the user.

[0047] <Functional configuration of Server 20> Figure 3 is a block diagram showing an example of the functional configuration of the server 20 in this embodiment. The server 20 includes a communication unit 201 for communicating with external devices, a storage unit 202 for storing various information, and a control unit 203 for controlling the overall operation of the server 20. First, the configurations of the storage unit 202 and the control unit 203 will be briefly explained.

[0048] The memory unit 202 stores various data necessary for managing and executing the patent search formula creation process in this system. As shown in Figure 3, the memory unit 202 contains, as its main data, a search case management table 2021 for managing individual search cases, a search condition table 2022 for storing search conditions entered by the user, a search condition table 2023 for holding search condition candidates generated by LLM40, etc., a search formula table 2024 for storing created search formula information, a search execution log table 2025 for recording the execution history of database searches, a system evaluation table 2026 for storing the system's search formula evaluation results, and a user evaluation table 2027 for storing user evaluations of search formulas. In addition, the memory unit 202 may optionally include a user table 2028 for managing system user information and a patent classification table 2029 for holding patent classification information. The detailed structure and data examples of these tables will be described later (see Figures 4 to 21).

[0049] The control unit 203 controls the operation of the entire server 20 by executing a program stored in the memory unit 202, thereby realizing a series of information processing operations according to the present invention. As shown in Figure 3, the control unit 203 functions as a communication control module 2031 that manages communication with external devices, a search condition receiving module 2032 that receives search conditions from users, a search condition generation module 2033 that generates search condition candidates in cooperation with LLM 40, a first search formula creation module 2034 that creates search formulas for the first database 50, a first search execution module 2035 that executes searches in the first database, an acquisition evaluation module 2036 that evaluates the first search formula, a search condition adjustment module 2037 that adjusts search conditions based on the evaluation results, a second search formula creation module 2038 that creates search formulas for the second database, a second search execution module 2039 that executes searches in the second database, a display control module 2040 that controls the display to the terminal device 10, an evaluation receiving module 2041 that receives evaluations from users, an overall control module 2042 that oversees the operation order and coordination of these modules, and a data management module 2043 that manages data access within the storage unit 202. The detailed processing contents of each of these modules will be described later.

[0050] (Detailed configuration of the memory unit) The storage unit 202 is composed of storage devices such as RAM (Random Access Memory), ROM (Read Only Memory), HDD (Hard Disk Drive), and SSD (Solid State Drive). The storage unit 202 stores the program related to this disclosure and various data necessary for processing. As shown in Figure 3, the storage unit 202 has an area for storing, for example, the following table.

[0051] (Investigation Case Management Table) The Investigation Case Management Table 2021 is a table used to manage basic information about individual investigation cases (case ID, client information, status, request date and time, etc.).

[0052] Figure 4 shows an example of the data structure of the investigation case management table 2021 in this embodiment. The investigation case management table 2021 is used to manage basic information about individual investigation cases handled by this system.

[0053] As shown in Figure 4, the investigation case management table 2021 has an "investigation case ID" (investigation_id) to uniquely identify each investigation case. This column is of integer type (INTEGER) and is set as the primary key (PK) of the table. The "user ID" (user_id) is a column that stores an ID to identify the user who requested this investigation case, and is of integer type (INTEGER). This column can function as a reference (foreign key, FK) to the separately defined user table 2028 (see Figure 3), and its setting is optional. The "investigation title" (title) is a column that stores a name or title that the user arbitrarily assigns to the investigation case, and is of variable-length string type (VARCHAR(255)). The "request date and time" (request_datetime) is a column that stores timestamp information (DATETIME) indicating the date and time when the investigation conditions for this investigation case were first registered in the system, and NULL values ​​are not allowed (NOT NULL). The "status" (status) is a column that stores a string (VARCHAR(50)) indicating the current progress of the investigation case. For example, values ​​such as "Received," "Processing," "Completed," and "Requires Evaluation" may be stored. "Last Updated" (last_updated) is a column that stores timestamp information (DATETIME) indicating the date and time when this investigation case record was last modified.

[0054] Figure 5 shows a specific example of data stored in the Research Project Management Table 2021. As shown in Figure 5, each record includes a unique research project ID, along with the associated user ID, research title, request date and time, current status, and last updated date and time. For example, a record with research project ID "1" represents a research project titled "Prior Art Document Search for New AI Search" submitted by user ID "101," and its current status is "Requires Evaluation."

[0055] (Survey conditions table) The Survey Conditions Table 2022 is a table for storing detailed survey conditions (target text, survey type, technology field, target hit limit, importance level, etc.) entered by the user for each survey project.

[0056] Figure 6 shows an example of the data structure of the survey conditions table 2022 in this embodiment. The survey conditions table 2022 is used to store and manage the detailed survey conditions entered by the user for each survey case.

[0057] As shown in Figure 6, the survey conditions table 2022 has a "condition ID" (condition_id) as its primary key (PK) to uniquely identify the condition information.

[0058] Furthermore, the Investigation Conditions Table 2022 contains an "Investigation Case ID" (investigation_id) that indicates which investigation case this condition information is associated with. This is a foreign key (FK) that references the Investigation Case Management Table 2021, and NULL values ​​are not allowed (NOT NULL).

[0059] The "target_text" column stores text information entered by the user regarding the technology being investigated, and NULL values ​​are not allowed (NOT NULL). This text serves as the primary input for generating search criteria.

[0060] The "investigation_type" column stores the type of investigation specified by the user (for example, "invalidation search," "infringement prevention search," "prior art document search," etc.).

[0061] The "technical field" column stores the technical field being investigated as specified by the user (for example, the International Patent Classification (IPC), FI, F-term, or a field name entered in free text).

[0062] The "Maximum Target Hits" column stores the upper limit of the target number of hits in a search in the second database. This serves as a guideline for the degree of refinement of the search expression. A "Minimum Target Hits" column may be set in conjunction with, or instead of, the "Maximum Target Hits" column. Furthermore, an initial value (e.g., "300") may be set for the "Maximum Target Hits" column depending on the purpose of the investigation.

[0063] The "Maximum Review Target" column stores an estimate of the maximum number of documents a user intends to ultimately review (screen). For example, the display of search results when searching the second database 60 is adjusted according to the "Maximum Review Target".

[0064] The "Investigation Importance" column stores the importance level of the investigation project set by the user (e.g., "High," "Medium," "Low," etc.). This can be used as an indicator of the business importance of the investigation project. For example, the control unit 203 may enrich or simplify the search conditions by changing the range of search condition candidates generated by the search condition generation module 2033 (e.g., the depth of related word expansion) or the degree of adjustment by the search condition adjustment module 2037 (e.g., the degree of keyword refinement) according to this importance setting. Alternatively, for example, if the importance level is set to "High," the overall control module 2042 may control the system to optimize the search conditions over a longer period of time by increasing the maximum number of iterations of the adjustment loop, which includes the acquisition evaluation step (step S116 in Figure 23) and the search condition adjustment step (step S118 in Figure 23), or by tightening the adjustment threshold.

[0065] The "confidentiality_flag" column stores a boolean value (True / False) indicating settings related to confidentiality when using LLM, such as whether masking or splitting of input information (text to be investigated, etc.) is required.

[0066] The "input_datetime" column stores timestamp information indicating the date and time when this set of investigation conditions was entered by the user, and NULL values ​​are not allowed (NOT NULL).

[0067] Figure 7 shows a specific example of data stored in the Survey Conditions Table 2022. As shown in Figure 7, each record includes the Condition ID, the associated Survey Case ID, the survey target text entered by the user, the survey type, the technology field, various upper limits, importance, confidentiality flag, and input date and time. For example, a record with Condition ID "1" is a survey condition corresponding to Survey Case ID "1," and it indicates that specific conditions such as the text "Technology for automatically extracting keywords for patent searches using generation AI...", the type "Prior Art Document Search", and the target hit limit "200" have been set.

[0068] (Search criteria table) The search condition table 2023 is a table for storing candidate search conditions, such as keywords, synonyms, and patent classifications, which are generated by the search condition generation module 2033 in the control unit 203.

[0069] Figure 8 shows an example of the data structure of the search condition table 2023 in this embodiment. The search condition table 2023 is used to store candidate search conditions that form the basis for creating search expressions, obtained by the search condition generation module 2033 of the control unit 203 by analyzing the target text with LLM40.

[0070] As shown in Figure 8, the search criteria table 2023 has a "candidate ID" (keyword_id) to uniquely identify each search criterion candidate. This column is of integer type (INTEGER) and is set as the primary key (PK) of the table.

[0071] The "Investigation Case ID" (investigation_id) is an ID that indicates which investigation case this search criterion candidate relates to, and is of type integer (INTEGER). This is a foreign key (FK) that references the Investigation Case Management Table 2021, and NULL values ​​are not allowed (NOT NULL).

[0072] The "Source Text Segment (Optional)" column stores information indicating which part of the entered text to be investigated the search criteria candidate was generated based on (for example, the text itself or its location). It is of type TEXT. Storing information in this column is optional.

[0073] The "Keyword Type" column stores a string (VARCHAR(50)) that identifies the type of search criterion candidate represented by this record. NULL values ​​are not allowed (NOT NULL). For example, it may store values ​​such as "keyword," "synonym," "classification," or "concept."

[0074] The "keyword_value" column stores the specific values ​​(keyword strings, classification codes, phrases representing concepts, etc.) of the extracted or generated search criterion candidates. It is of variable-length string type (VARCHAR(255)) and does not allow NULL values ​​(NOT NULL).

[0075] The "Relevance Score (Optional)" column stores a score calculated by LLM indicating the degree of relevance between this search criterion and the original text being investigated. It is of floating-point type (FLOAT). Storing information in this column is optional.

[0076] The "generation_datetime" column stores timestamp information (DATETIME) indicating the date and time this search criterion candidate was generated by LLM, and NULL values ​​are not allowed (NOT NULL).

[0077] Figure 9 shows a specific example of data stored in the search criteria table 2023. As shown in Figure 9, for a single research case (e.g., research case ID "1"), multiple search criterion candidates are generated, and their respective candidate IDs, candidate types, candidate values, relevance scores, generation dates, etc., are recorded. For example, candidate ID "1" is a keyword candidate "patent search", and candidate ID "4" is a classification code candidate "G06F16 / 332 (...Formalization of query [2025.01])".

[0078] (Search formula table) The search expression table 2024 is a table for storing strings, version information, and other data of search expressions (for the first database 50 and the second database 60) created by the first search expression creation module 2034 and the second search expression creation module 2038 in the control unit 203.

[0079] Figure 10 shows an example of the data structure of the search expression table 2024 in this embodiment. The search expression table 2024 is used to store and manage information about search expressions created by the first search expression creation module 2034 and the second search expression creation module 2038 of the control unit 203.

[0080] As shown in Figure 10, the search expression table 2024 has a "search expression ID" (query_id) to uniquely identify each search expression. This column is of integer type (INTEGER) and is set as the primary key (PK) of the table.

[0081] The "investigation_id" is an ID that indicates which investigation case this search expression relates to, and is of integer type (INTEGER). This is a foreign key (FK) that references the investigation case management table 2021, and NULL values ​​are not allowed (NOT NULL).

[0082] The "query_version" column is used to manage the version number of a search query when it is modified or recreated within the same research project. It is of integer type (INTEGER). It can be used to track recreations based on user feedback, and its setting is optional.

[0083] The "Target DB Type" (database_type) is an identifier that indicates which database this search expression targets. It is a variable-length string type (VARCHAR(10)) and does not allow NULL values ​​(NOT NULL). For example, it may store values ​​such as "DB1" (for the first database 50) or "DB2" (for the second database 60).

[0084] The "query_string" column stores the actual search expression string used to execute the search in the database. It is of type TEXT and does not allow NULL values ​​(NOT NULL).

[0085] The "creation_datetime" column stores a timestamp (DATETIME) indicating the date and time this search expression was created by the system, and NULL values ​​are not allowed (NOT NULL).

[0086] The "List of Candidate IDs Based on Keywords (Optional)" (based_on_keywords_ids) is a column that stores a list of candidate IDs from the search criteria table 2023 that formed the basis for creating this search expression. It can be of type TEXT or JSON. It can be used to track which candidate search criteria were used to generate the search expression, and its setting is optional.

[0087] The "Evaluation Status (Optional)" column stores the status indicating the evaluation stage of this search expression, and is of variable-length string type (VARCHAR(50)). For example, it may store values ​​such as "Not Evaluated," "System Evaluated," or "User Evaluated." Storing information in this column is optional.

[0088] Figure 11 shows a specific example of data stored in the search expression table 2024. As shown in Figure 11, multiple search expression records may be generated for a single investigation case (e.g., investigation case ID "1"). For example, search expression ID "1" is a version 1 search expression for the first database (DB1) and indicates that it has been system-evaluated. Search expression ID "2" is a version 1 search expression for the second database (DB2) and indicates that it has been user-evaluated. Search expression ID "3" is a version 2 search expression for the second database (DB2) that may have been recreated after user evaluation and indicates that it has not yet been evaluated (not evaluated).

[0089] (Search execution table) The search execution log table 2025 is a table for recording the history (execution date and time, target DB, search expression ID used, number of hits, etc.) of database searches performed by the first search execution module 2035 and the second search execution module 2039 of the control unit 203.

[0090] Figure 12 shows an example of the data structure of the search execution log table 2025 in this embodiment. The search execution log table 2025 is used to record and manage history information when the first search execution module 2035 and the second search execution module 2039 of the control unit 203 perform searches on the first database 50 or the second database 60.

[0091] As shown in Figure 12, the search execution log table 2025 has a "log ID" (log_id) to uniquely identify each search execution log. This column is of integer type (INTEGER) and is set as the primary key (PK) of the table.

[0092] The "query_id" column is an integer type column that indicates the ID of the search expression used in this search execution. It is a foreign key (FK) that references the search expression table 2024 and does not allow NULL values ​​(NOT NULL).

[0093] The "execution_datetime" column stores timestamp information (DATETIME) indicating the date and time the database search was actually executed, and does not allow NULL values ​​(NOT NULL).

[0094] The "search database name" (database_name) is a column that stores the specific name of the database where the search was performed (for example, "J-PlatPat", "googlepatents", "internal literature database", etc.), and is of variable-length string type (VARCHAR(100)).

[0095] The "hit count" column stores the number of hits obtained as a result of the search, and is of integer type (INTEGER).

[0096] The "execution_status" column stores a string (VARCHAR(50)) indicating the execution result, such as whether the search execution completed successfully or if an error occurred. For example, it may store values ​​such as "Success," "Failure," or "Timeout."

[0097] Figure 13 shows a specific example of data stored in the search execution log table 2025. As shown in Figure 13, each record includes a unique log ID, the search expression ID used, the date and time the search was executed, the name of the database searched, the number of hits, and the execution status. For example, the record with log ID "1" indicates that a search was performed on the "Internal Literature DB" on March 29, 2025 at 10:15:00 using search expression ID "1", resulting in 1500 hits and indicating that the execution was successful.

[0098] (System Evaluation Table) The system evaluation table 2026 is a table for storing evaluations (such as the results of the count evaluation) performed by the acquisition evaluation module 2036 of the control unit 203 based on the search results of the first database 50.

[0099] Figure 14 shows an example of the data structure of the system evaluation table 2026 in this embodiment. The system evaluation table 2026 is used to store and manage the evaluation content performed by the acquisition evaluation module 2036 of the control unit 203 based on the search results for the first database 50.

[0100] As shown in Figure 14, the system evaluation table 2026 has a "system evaluation ID" (sys_eval_id) to uniquely identify each system evaluation. This column is of integer type (INTEGER) and is set as the primary key (PK) of the table.

[0101] The "log ID" (log_id) is a column of integer type (INTEGER) that indicates the log ID of the search execution being evaluated. This is a foreign key (FK) that references the search execution log table 2025, and NULL values ​​are not allowed (NOT NULL). Typically, this log ID refers to the log corresponding to the first database search.

[0102] The "evaluation metric" is an identifier that indicates the criteria used for evaluation. It is a variable-length string type (VARCHAR(100)) and does not allow NULL values ​​(NOT NULL). For example, it may store a value such as "hit_count_check" (hit count check).

[0103] The "evaluation_result" column stores a string (VARCHAR(50)) indicating the result of the evaluation based on the evaluation metric. For example, if the evaluation metric is a hit count check, values ​​such as "too_many", "too_few", and "appropriate" may be stored.

[0104] The "Evaluation Details (Optional)" column is a text-type column that stores specific evidence and supplementary information regarding the evaluation results. Storing information in this column is optional.

[0105] As another example of evaluation metrics and evaluation results, the acquisition evaluation module 2036 may evaluate the amount of noise contained in the search results in the first database 50. In this case, an identifier such as "noise_level_check" may be used as the evaluation metric. To perform the evaluation, the acquisition evaluation module 2036 retrieves snippet information (title, part of the abstract, representative claims, etc.) from the first database 50 for some of the hit documents (for example, sampled documents if the number of hits exceeds a predetermined number, or all documents if the number of hits is small). Then, it compares the retrieved snippet information with the research conditions (text to be researched, etc.) stored in the research conditions table 2022 to determine whether each document conforms to the research conditions (whether it is noise or not).

[0106] This determination can be made, for example, by inputting snippets and search conditions into LLM40, calculating a relevance score, and determining that if the score is above a threshold, it is relevant, and if it is below, it is considered noise. Alternatively, it can be done using a predefined rule-based determination logic. Based on the determination result, if the proportion of noise literature is lower than a predetermined standard, the evaluation result (evaluation_result) is stored in the system evaluation table 2026 as "good" (low noise), and if it is higher, it is stored as "bad" (high noise). This evaluation result can be used in the subsequent search condition adjustment (step S118) to determine whether to adjust the search criteria in a way that reduces noise (e.g., by adding more specific keywords or classifications) or in a way that increases relevant literature (by broadening the search range).

[0107] Furthermore, as another example of evaluation details, information that can serve as a clue for adjusting search conditions may be stored, such as the noise evaluation process described above, or when the hit count evaluation (hit_count_check) results are inappropriate (too_many, too_few). For example, the acquisition evaluation module 2036 could be configured to extract or generate promising keyword candidates for search conditions, or conversely, keyword candidates that should be excluded from search results (or used in ANDNOT conditions), from snippet information of search results acquired from the first database 50 (especially the text of documents judged to have a high degree of relevance, or conversely, documents judged to be noise), and store this list as evaluation details. This keyword extraction and generation process can be performed, for example, by inputting a group of snippets into LLM40 and giving instructions such as "List important technical terms common to this group of texts" or "List frequently occurring terms that do not match the search criteria." Alternatively, it can be performed by analyzing the text using methods such as morphological analysis or TF-IDF, removing predefined stop words (overly common words or particles, etc.), and then mechanically extracting words with high frequency and importance. The keyword list thus extracted and generated can be used as a strong candidate when the search criteria adjustment module 2037 adjusts the search criteria.

[0108] The "evaluation_datetime" column stores a timestamp (DATETIME) indicating the date and time the system performed the evaluation. NULL values ​​are not allowed (NOT NULL).

[0109] Figure 15 shows a specific example of data stored in the system evaluation table 2026. As shown in Figure 15, each record includes a unique system evaluation ID, the log ID of the log being evaluated, the evaluation metric, the evaluation result, the evaluation details, and the evaluation date and time. For example, the record with system evaluation ID "1" indicates that log ID "1" (log of the first database search) was evaluated using the metric "hit_count_check," the result was "too_many," and the reason was that "the number of hits (1500) exceeded the target (200)."

[0110] (User rating table) The user evaluation table 2027 is a table for storing evaluations (good / bad, comments, modified search expressions, etc.) of search expressions (mainly second search expressions) received from users by the evaluation reception module 2041 of the control unit 203.

[0111] Figure 16 shows an example of the data structure of the user evaluation table 2027 in this embodiment. The user evaluation table 2027 is used by the evaluation reception module 2041 of the control unit 203 to store and manage evaluation information for search expressions received from the user. This evaluation information mainly serves as feedback provided by the user for search expressions for the second database 60.

[0112] As shown in Figure 16, the user evaluation table 2027 has a "user evaluation ID" (user_eval_id) to uniquely identify each user evaluation. This column is of type integer (INTEGER) and is set as the primary key (PK) of the table.

[0113] The "query_id" column is an integer (INTEGER) column that indicates the ID of the search expression that was evaluated by the user. This is a foreign key (FK) that references the search expression table 2024, and NULL values ​​are not allowed (NOT NULL). Typically, this search_id refers to the search expression corresponding to the second database search.

[0114] The "User ID" (user_id) is a column that stores an ID to identify the user who performed the evaluation, and is of integer type (INTEGER). This column can function as a reference (foreign key, FK) to the separately defined user table 2028 (see Figure 3), and its setting is optional.

[0115] The "evaluation_score" column stores a string (VARCHAR(50)) representing the evaluation level or score assigned by the user to a search query. For example, it may store qualitative evaluations such as "Good," "Needs Improvement," or "Bad," or numerical ratings on a 5-point scale.

[0116] The "evaluation_comment" column stores comments entered by users in a free-form text format during their evaluation. It records areas for improvement in the search query and reasons for the evaluation. Users may also include modified search queries in this "evaluation_comment" column.

[0117] The "evaluation_datetime" column stores a timestamp (DATETIME) indicating the date and time the user's evaluation was entered into the system. NULL values ​​are not allowed (NOT NULL).

[0118] Figure 17 shows a specific example of data stored in the user evaluation table 2027. As shown in Figure 17, each record includes a unique user evaluation ID, the search expression ID being evaluated, the user ID who performed the evaluation, the evaluation score, the evaluation comment, and the date and time of the evaluation. For example, the record with user evaluation ID "1" indicates that user ID "101" evaluated search expression ID "2," with an evaluation score of "Needs Improvement" and the comment "Missing the G06N3 / 00 perspective." This evaluation information can trigger readjustment or recreation of search expressions.

[0119] (User Table) User Table 2028 is a table (optional) for managing information about users who use this system (user ID, name, affiliation, etc.).

[0120] Figure 18 shows an example of the data structure of a user table 2028, which is optionally provided in this embodiment. The user table 2028 is used to manage information about users who use this information processing system.

[0121] As shown in Figure 18, the user table 2028 has a "user ID" (user_id) to uniquely identify each user. This column is of integer type (INTEGER) and is set as the primary key (PK) of the table. This user ID may be referenced from other tables, such as the survey case management table 2021 and the user evaluation table 2027.

[0122] The "username" column stores the user's name, nickname, etc., and is of variable-length string type (VARCHAR(100)).

[0123] The "email address" column stores the email address used for contacting users, and is of variable-length string type (VARCHAR(255)).

[0124] The "Department" column stores the name of the department to which the user belongs, and is of variable-length string type (VARCHAR(100)). Information stored in this column is optional. Additional information may be presented in the search query based on the department. For example, if a user in the Research and Development Department is creating a search query, they can add a commented-out explanation of the patent classification in the search criteria field.

[0125] The "Registration Date and Time" (created_at) column stores timestamp information (DATETIME) indicating the date and time this user was registered in the system.

[0126] Furthermore, in addition to the information mentioned above, the user table 2028 may also be configured to store other information such as the password information set by the user to log in to the system (usually stored in hashed form), the user's permission level for using system functions (e.g., administrator, general user, etc.), or information indicating each user's usage status (e.g., number of search queries created, API usage, etc.).

[0127] Figure 19 shows a specific example of data stored in the user table 2028. As shown in Figure 19, each record includes a unique user ID, along with the username, email address, department, and registration date and time. For example, the record for user ID "101" indicates that a user named "Taro Tanaka" belongs to the "Intellectual Property Department" and was registered in the system on October 1, 2024 at 9:00:00. (Email addresses are omitted from the display in the figure.)

[0128] (Patent Classification Table) Patent Classification Table 2029 is a table (optional) that holds corresponding data for patent classification symbols and titles (descriptions) to retrieve relevant patent classifications from keywords.

[0129] Figure 20 shows an example of the data structure of a patent classification table 2029, which is optionally provided in this embodiment. This type of table is used to store and manage patent classification information, such as the International Patent Classification (IPC), in a manner that reflects the structure of a specific information source.

[0130] As shown in Figure 20, the patent classification table 2029 has multiple columns containing information for identifying each classification entry.

[0131] The "Classification Type" column stores an identifier indicating the patent classification system to which this record belongs (e.g., "IPC"). It is a variable-length string type (VARCHAR(20)) and does not allow NULL values ​​(NOT NULL).

[0132] "Classification #" (classification_num) is a column that stores the unique number or ID of each classification entry in the information source. It is of type integer (INTEGER) and can be set as a primary key (PK).

[0133] The "hierarchy_name" column stores a name (e.g., "section", "class", "subclass", "subgroup") indicating which hierarchical level this classification entry belongs to within the classification system. It is a variable-length string type (VARCHAR(50)) and does not allow NULL values ​​(NOT NULL).

[0134] The "section" (section_info) is a column that stores information about the section to which this classification entry belongs or is related (for example, section code "G" or name "Physics"), and is of type TEXT.

[0135] The "Class" (class_info) column stores information about the class to which this classification entry belongs or is related (for example, the class code "G06" or the name "Calculation or Counting"), and is of type TEXT.

[0136] The "subclass" (subclass_info) column stores information about the subclass to which this classification entry belongs or is related (for example, the subclass code "G06Q" or the name "Information and Communication Technology [ICT] specifically adapted for administrative, commercial, financial, business, or supervisory purposes"), and is of type TEXT.

[0137] The "symbol" column stores the complete classification code or symbol corresponding to this classification entry (e.g., "G", "G06", "G06Q", "G06Q 50 / 00", etc.), is of variable-length string type (VARCHAR(50)), and does not allow NULL values ​​(NOT NULL).

[0138] The "title" column stores the formal name or definition of the technical content indicated by the classification code. It is of type TEXT and does not allow NULL values ​​(NOT NULL).

[0139] Figure 21 shows a concrete example of data stored in the patent classification table 2029, using IPC as an example. As shown in Figure 21, hierarchical information such as section "G" (physics), class "G06" (computation or counting), and subclass "G06Q" (administrative purposes, commercial purposes, etc.) is recorded. Further down the hierarchy, information on subgroups is also stored. For example, under subclass "G06Q," there is a subgroup "G06Q 50 / 00" related to information and communication technology suited to a specific industry, and under that, various subgroups corresponding to different industries are defined. As a concrete example, the figure shows that it includes an entry for "legal services" corresponding to the symbol "G06Q 50 / 18". Such detailed classification information can be effectively used to generate search criteria and refine search expressions.

[0140] The patent classification table 2029 may not only contain general patent classification data as described above, but may also be a table of patent classifications customized to individual users or specific technological trends. For example, it may be a table that contains only the patent classification codes assigned to company applications filed in the past by a particular user, or a table that contains only the patent classification codes assigned to a group of applications (by the company and / or other companies) filed within a certain period in the past in the technological field that the user is primarily involved in or is investigating. Furthermore, in such a customized table, it is also possible to configure it to store statistical information such as the number of applications to which it was assigned and its frequency of appearance (within the table) in association with each patent classification code. In this case, for example, the search condition generation module 2033 and the first search expression creation module 2034 may consider this statistical information (e.g., number of applications and frequency of appearance) to determine the priority of the classification codes, and then preferentially select the classification codes with higher priority as search condition candidates or adopt them in the search expression.

[0141] (Detailed configuration of the control unit) The control unit 203 comprehensively controls the operation of the entire server 20 by having a processor, such as a CPU (Central Processing Unit), execute a program related to this disclosure stored in the memory unit 202. By operating according to the program, the control unit 203 performs the following functions as a functional module, for example.

[0142] The communication control module 2031 is a module that controls data communication between the terminal device 10, the external LLM 40, the first database 50, the second database 60, etc., via the communication unit 201. It may include, if necessary, functions to implement appropriate security functions to ensure the security of communication, such as encryption of the communication path (e.g., TLS / SSL), access control (e.g., firewall or IP address restriction), and user authentication (e.g., password authentication or two-factor authentication), or functions to cooperate with external security mechanisms.

[0143] The survey condition reception module 2032 receives survey conditions (survey target text, detailed settings, etc.) from the terminal device 10 via the communication control module 2031 and stores them in the survey condition table 2022, etc., via the data management module 2043. This survey condition reception module 2032 corresponds to the survey condition reception step described later.

[0144] The search condition generation module 2033, based on the survey target text received by the survey condition reception module 2032, communicates with LLM40 via the communication control module 2031 to generate search condition candidates (keywords, classifications, etc.) and stores them in the search condition table 2023 via the data management module 2043. The search condition generation module 2033 corresponds to the search condition generation step described later.

[0145] For example, when the search condition generation module 2033 analyzes the target text using LLM40, it may perform the following processing. First, it structurally analyzes the content of the target text and extracts descriptions of the technical field to which the technology described in the text is thought to belong, the constituent elements that form the core of the invention, and the effects brought about by those constituent elements (it is possible that no description may be extracted for each item). Next, it identifies several key keywords that form the core of the search from the extracted descriptions of the technical field, constituent elements, and effects. Then, it performs keyword expansion on each identified keyword to achieve a comprehensive search. At this time, the search condition generation module 2033 can give LLM40 a prompt in the format of, for example, "For the following target text, please list keywords, synonyms, related terms, related conceptual terms, and highly relevant patent classifications (IPC, FI, F-terms) in JSON format for the purpose of patent searching in the technical field "(investigation technical field)".

[0146] In this keyword expansion, related terms are generated by considering factors such as alternative spellings of keywords (e.g., "search engine" and "search search engine"), synonyms and related terms (e.g., "extract" and "obtain," "select"), expressions in other languages ​​such as English, simple variations in spelling (e.g., "computer" and "computer"), specific expressions that may be used by particular companies or research institutions, anticipated misspellings, and expressions specific to patent documents (e.g., "means to do ~," "~department"). Furthermore, in order to appropriately broaden or narrow the search scope, more specific sub-concepts are expanded (e.g., "GPT" for "generative AI," "Transformer" for "large-scale language model"), more comprehensive superordinate concepts (e.g., "search query," "search term" for "keyword"), alternative expressions to achieve the same or similar functions (e.g., "present," "output" for "display"), and other highly relevant scientific and technical terms that are likely to be used in patent specifications. The diverse terms extracted and expanded in this manner, along with relevance scores and other factors, are stored in the search criteria table 2023 via the data management module 2043 as candidate search criteria. Keywords may also include applicants and inventors obtained as information based on the snippets described above. These are also stored in the search criteria table 2023 via the data management module 2043.

[0147] Furthermore, when generating candidate search conditions using the search condition generation module 2033, and subsequently creating search expressions using the first search expression creation module 2034 and the second search expression creation module 2038, the "investigation type" stored in the investigation condition table 2022 is taken into consideration. Depending on the investigation type, the instructions (prompts) given to LLM40 are adjusted, the selection and combination of generated candidate search conditions are changed, or the refining conditions (such as period and legal status) added to the search expression are modified. This ensures that a search strategy optimized for each investigation objective is executed. The following describes specific examples of processing for representative investigation types.

[0148] (In the case of investigation type: "Investigation of invalid documents") When the search type is "invalidation search," its purpose is to discover prior art documents (invalidation documents) that can negate the novelty and inventiveness of a specific target patent (the patent to be invalidated). For this reason, in the search condition acceptance step (step S101), specific information about the target patent (patent number, application number, etc.) and dates that serve as criteria for determining its validity (filing date, priority date, etc.) must be entered as search conditions, or the server 20 must obtain them separately.

[0149] The search condition generation module 2033 analyzes the text to be investigated (such as the claims and specifications of the target patent) and generates keywords and classifications that indicate technical features. Then, the first search formula creation module 2034 and the second search formula creation module 2038 combine the generated search condition candidates and add time limit conditions based on publication dates and issue dates to the search formulas so that only documents that were published (made publicly known) before the reference date (documents that can be primary or secondary citations) are extracted from the first database 50 and the second database 60. Furthermore, since inventions described only in embodiments can also be used as grounds for invalidation in this type of search, the search formula is generally configured to search the entire document (full text).

[0150] (In the case of a "prior art literature search" type of survey) When the search type is "Prior Art Document Search," the main purpose is to search existing technical documents (prior art) related to an invention that is planned to be newly filed or a technical theme under consideration, in order to obtain information for determining novelty and inventive step, and to grasp the trends in technology. The generation of search conditions related to the technical content (keywords, classifications, etc.) is performed by the search condition generation module 2033, as in the case of "Invalidation Document Search."

[0151] However, in a "prior art document search," there is usually no need to limit the search to documents prior to a specific reference date; all relevant documents published to date are included. Therefore, the first search formula creation module 2034 and the second search formula creation module 2038 do not, in principle, add time restrictions based on publication or issue dates to the search formula (except when a specific period is specified as a search condition). In this case, the entire document (full text), which is used to understand technological trends and to judge novelty and inventiveness, is generally included in the search.

[0152] (In the case of investigation type: "Infringement prevention investigation") When the investigation type is a "preventive infringement investigation" (also known as a clearance investigation or FTO investigation), its purpose is to confirm whether the products or methods that the company plans to implement (manufacture, sell, etc.) pose a risk of infringing on the valid patent rights of others. Therefore, it is important not only whether technically relevant literature exists, but also whether it constitutes a "valid right."

[0153] Therefore, it is desirable for the search condition generation module 2033 to focus on generating relatively high-level conceptual keywords that are likely to be used in the scope of rights (claims) from the text to be investigated (such as descriptions of the company's products or methods). Furthermore, when combining the generated search condition candidates, the first search expression creation module 2034 and the second search expression creation module 2038 add filter conditions (legal status, status information, etc.) to the search expression for the first database 50 and the second database 60 to limit the search to documents that are likely to have currently valid rights, i.e., patent documents for which the application is still under examination (pending) or for which the right has not expired and the right is still in effect. In addition, since the determination of infringement risk is mainly based on the scope of rights, it is common to configure the search expression to limit the search scope to the claims as a general rule. This enables efficient investigation focused on patent rights that actually pose a risk of infringement. Note that since effects are not described in the claims, the entire document (full text) is included in the search.

[0154] The first search expression creation module 2034 retrieves candidate search conditions from the search condition table 2023 via the data management module 2043, combines these to create a first search expression for the first database 50, and stores it in the search expression table 2024. The first search expression creation module 2034 corresponds to the first search expression creation step described later.

[0155] The first search expression creation module 2034 creates the first search expression by combining search condition candidates (keywords, related words, classifications, etc.) obtained from the search condition table 2023, for example, as follows: First, the keyword candidates generated by the search condition generation module 2033 that belong to the same concept (for example, the original keyword and its variant spellings, synonyms, related words, variations in spelling, related words, etc.) are grouped together, and the keywords within each group are joined using a logical OR operator (for example, "OR" or "+" depending on the database specifications). If a keyword candidate representing a complex concept (e.g., "search expression creation") is generated, it may be divided into its individual constituent words ("search expression", "creation"), and a condition may be created to join them using a proximity operator (for example, "ADJ", "NEAR", "(W)", etc., depending on the database specifications) so that the search targets documents in which these words appear close together within a predetermined number of words. Additionally, a code may be added to specify the item (field) within the documents that will be searched for using the keyword search. For example, a code such as " / TX" is assigned to keywords or keyword groups according to the database specifications, " / AB" to target only the abstract, and " / CL" to target only the claims.

[0156] Next, the first search expression creation module 2034 selects the keyword groups created in this way (groups joined by logical OR or conditions for proximity search) and the generated patent classification candidates according to the search objective, search type, characteristics of the first database 50, or evaluation results in the acquisition evaluation step (described later), and combines these with a logical AND operator (for example, "AND" or "&") to construct a search expression that includes weighting conditions for narrowing down the search target.

[0157] In selecting and combining patent classification candidates, for example, if the target country for the search is Japan, it is conceivable to set both FI (File Index), the Japanese patent classification system, and IPC (International Patent Classification), which is used internationally, as search conditions. If the search condition generation module 2033 generates three highly relevant IPC subclasses (e.g., G06F, G06Q, G06N), the first search expression creation module 2034 may identify the corresponding FI classifications for each by referring to the patent classification table 2029, etc., and create a search expression with a structure such that, for example, "G06F / FI or G06F / IPC or G06Q / FI or G06Q / IPC or G06N / FI or G06N / IPC", a hit is made if each technical aspect is classified as either FI or IPC, and then a search expression is created by logically combining them with logical AND to extract documents that satisfy all of those technical aspects.

[0158] Furthermore, if very detailed classification codes such as IPC subgroups, FI expansion symbols or sub-issue symbols, or F-terms (File Forming Term) are generated as search condition candidates, it is possible to adjust the degree of further refinement (weighting) by keywords when using these detailed classifications. For example, if it is determined that the scope of technology to be investigated can be sufficiently identified by simply specifying a particular detailed classification (e.g., FI G06Q50 / 18,310 is a classification specialized in "intellectual property management in legal practice"), the first search expression creation module 2034 may be instructed to reduce the number of keywords weighted by the AND condition, or to not weight by keywords at all. This is expected to prevent excessive refinement while forming an appropriate search population. The first search expression created in this way is stored in the search expression table 2024.

[0159] The first search execution module 2035, based on instructions from the overall control module 2042, retrieves the first search expression from the search expression table 2024 via the data management module 2043, executes the search in the first database 50 via the communication control module 2031, and retrieves the results (number of hits, snippet information, etc.). The retrieved results are recorded in the search execution log table 2025. The first search execution module 2035 corresponds to a part of the retrieval evaluation step described later.

[0160] The acquisition evaluation module 2036 compares and analyzes the search results (number of hits, snippet information acquired as needed) from the first database 50 acquired by the first search execution module 2035 with the research conditions (target hit limit, etc.) stored in the research conditions table 2022, and evaluates the first search expression. From the perspective of evaluation, in addition to the appropriateness of the number of hits (e.g., "too_many", "too_few", "appropriate"), the noise level of the search results (e.g., "good" or "bad" based on relevance judgments (e.g., performed using LLM40) based on snippet information) may be included. The acquisition evaluation module 2036 stores these evaluation results, as well as information useful for the basis of the evaluation and adjustment of search conditions (e.g., a list of keyword candidates extracted from snippets) as evaluation details in the system evaluation table 2026. The acquisition evaluation module 2036 corresponds to the acquisition evaluation step described later.

[0161] The Search Condition Adjustment Module 2037 is a module that adjusts (adds, deletes, modifies, changes operators, etc.) the candidate search conditions stored in the Search Condition Table 2023 based on the evaluation results from the Acquisition Evaluation Module 2036 (evaluation_result in the System Evaluation Table 2026), the information stored in the evaluation details of the System Evaluation Table 2026 (evaluation_details) (for example, the extracted keyword list mentioned above), and the original survey conditions (Survey Condition Table 2022).

[0162] For example, if the evaluation result of the acquisition evaluation module 2036 determines that the number of hits in the first database 50 is less than the target range (for example, the evaluation result of the system evaluation table 2026 is "too_few"), the search condition adjustment module 2037 will adjust the search conditions in a way that increases the number of hits. Specific adjustment methods include, for example, loosening the relationship between specific keywords and changing the proximity operator (e.g., "(A) N (B)", "(A) W (B)", appearing within a predetermined number of words) to the AND operator (e.g., "(A) AND (B)") so that a hit is achieved simply by co-occurring. Alternatively, adjustments may be made to broaden the search range by adding keyword candidates such as synonyms and related words stored in the search condition table 2023 to the OR condition included in the search expression, similarly adding patent classification candidates combined in the OR condition, or changing the hierarchy of the patent classification used for narrowing down in the AND condition to a higher level (for example, expanding the target from a specific subclass classification to a broader class classification). For example, if the evaluation result is "too_few", the search condition adjustment module 2037 may perform adjustment processing in the following order of priority based on pre-set rules: (1) adding unused synonyms and related words with a relevance score above a threshold to the OR condition, (2) changing the proximity operator between keywords joined by an AND condition to an AND operator, and (3) changing the patent classification joined by an AND condition to a higher hierarchy. In addition, if the evaluation details of the system evaluation table 2026 contain promising keyword candidates extracted from the search results (for example, keywords that match the search conditions but were not included in the search expression), adjustments may also be made to add those keywords to the OR condition.

[0163] Conversely, if the number of hits is determined to be greater than the target range (for example, the evaluation result is "too_many"), the search condition adjustment module 2037 adjusts the search conditions to reduce the number of hits. Specific adjustment methods include, for example, changing the AND operator to a more restrictive proximity operator to limit the search to cases where specific keywords appear closer together. Alternatively, adjustments may be made to strengthen the refinement by adding new keywords or lower-level patent classifications using AND conditions to the entire search expression, deleting some of the keyword or patent classification candidates combined using OR conditions, or changing the hierarchy of the patent classification used for refinement with AND conditions to a lower level (for example, limiting from class classifications to more specific subclass classifications or group classifications). Furthermore, if the evaluation details of the system evaluation table 2026 contain keywords characteristic of noisy documents or keyword candidates to be excluded, adjustments may be made to add those keywords using NOT conditions or to delete less relevant keywords from existing OR conditions.

[0164] These adjustments may be performed automatically by the search condition adjustment module 2037 based on predetermined rules, or they may be performed in a semi-automatic form where the user is presented with adjustment candidates and prompted to make a selection. The adjusted search conditions are used to recreate the first search expression (when returning from step S117:No to step S110 in Figure 23 described later) or to create the second search expression (step S119 in the same figure described later).

[0165] The second search expression creation module 2038 is a module that creates a second search expression for the second database 60 using the search conditions adjusted by the search condition adjustment module 2037, and stores it in the search expression table 2024. The second search expression creation module 2038 corresponds to the second search expression creation step described later. The second search expression creation module 2038 creates the second search expression for the second database 60 by performing the same processing as the first search expression creation module 2034, so a detailed explanation is omitted.

[0166] The second search execution module 2039, based on instructions from the overall control module 2042, retrieves the second search expression from the search expression table 2024 via the data management module 2043, executes the search in the second database 60 via the communication control module 2031, and retrieves the results (bibliography, etc.). The retrieved results are recorded in the search execution log table 2025. The second search execution module 2039 corresponds to a part of the search result display step described later. Since the second search execution module 2039 performs the same processing as the first search execution module 2035 to execute the search for the second search expression in the second database 60, a detailed explanation is omitted.

[0167] The display control module 2040, based on instructions from the overall control module 2042, generates screen data to display the search expression (mainly the second search expression) obtained from the search expression table 2024 and the search results obtained by the second search execution module 2039 (including sorting processing as needed) on the terminal device 10 used by the user, and transmits it via the communication control module 2031. In the sorting processing, LLM40 or the like may be used for scoring relevance. The display control module 2040 corresponds to the search expression display step and the search result display step described later.

[0168] The evaluation reception module 2041 is a module that receives evaluation information (good / bad, comments, edits made by the user if applicable) for the displayed search expression transmitted from the terminal device 10 via the communication control module 2031, and stores it in the user evaluation table 2027, etc. The evaluation reception module 2041 corresponds to the search expression evaluation reception step described later.

[0169] The overall control module 2042 controls the processing order and coordination of the aforementioned function modules 2031-2041 and 2043. In particular, it controls feedback loops that cause the search condition adjustment module 2037 and the second search expression creation module 2038 to be executed again based on the user evaluation received by the evaluation reception module 2041.

[0170] The data management module 2043 is a module that manages access to each table 2021-2028 in the storage unit 202, including reading, writing, updating, and deleting data.

[0171] (Search query creation process)

[0172] Next, the specific flow of the search expression creation process in System 1 will be explained with reference to the flowcharts in Figures 22 to 24. Figures 22 to 24 show consecutive parts of the flowchart related to the search expression creation process. Figure 25 is a schematic diagram showing an example of the search condition input screen. Figure 26 is a schematic diagram showing an example of the search expression display and evaluation screen. Figure 27 is a schematic diagram showing an example of the search results display screen. The screens shown in these schematic diagrams are schematic representations of the screens displayed on the output device 14 (especially the display) of the terminal device 10 in the process shown in Figures 22 to 24.

[0173] First, the user operates the terminal device 10 to launch the application for creating search queries, and logs in as appropriate to enable input of search conditions into the application. The control unit 190 of the terminal device 10 displays a search condition input screen on the output device 14 as shown in Figure 25. On this search condition input screen, the user selects the "search type" (e.g., "prior art document search"), enters text in the "technology to be searched" field (e.g., "Create a search query for patent document search using generation AI"), and sets the necessary search conditions, such as entering an upper limit value (e.g., "300") in the "maximum number of peer review items" field (Figure 22, step S101). This information entered via the input device 13 is received by the operation reception unit 191 of the terminal device 10.

[0174] Once the input of the survey conditions is complete and the user instructs execution, the transmitting / receiving unit 192 of the terminal device 10 transmits the received survey condition data to the server 20 via the communication unit 120 (step S102).

[0175] On server 20, the communication control module 2031 receives survey condition data from terminal device 10. Based on the received data, the survey condition reception module 2032 creates a new survey case record (or updates an existing case) in the survey case management table 2021 via the data management module 2043 and saves the detailed survey conditions to the survey condition table 2022 (step S103). This completes the survey condition reception step.

[0176] Next, the search condition generation module 2033 retrieves the text to be investigated and the investigation type from the investigation condition table 2022 and starts generating search conditions (Figure 22, step S104). At this time, the text to be investigated is divided into multiple parts based on predetermined rules (for example, by sentence, by semantic chunk, or by comma (,)) and preparations are made to analyze each subtext. In addition, the instructions (prompts) to LLM40 may be adjusted according to the investigation type (e.g., "prior art document search"). For example, the text to be investigated can be divided into subtexts by comma (,) and these can be included in the instructions (prompts) and sent to LLM40 sequentially or in parallel. In this way, instead of sending the entire text to LLM40 at once, parts of it are included in the prompts for analysis, which is expected to reduce the risk of highly confidential information being sent to an external LLM40 and prevent information leakage. Alternatively, the text to be investigated can be sent to LLM40 as is without performing such processing.

[0177] The search condition generation module 2033 sends the prepared data and prompts to the LLM40 via the communication control module 2031, requesting text analysis and generation of related information (step S105).

[0178] LLM40 receives a request from server 20 (step S106) and analyzes the received text (or a portion of the text) (step S107). As a result of the analysis, it generates related terms such as technical keywords contained in the text, their synonyms, superordinate and subordinate conceptual terms, and potentially relevant patent classifications (e.g., patent classifications similar to IPC stored in patent classification table 2029).

[0179] LLM40 sends the analysis results, such as the generated related words and patent classifications, to the server 20 (step S108).

[0180] On server 20, the communication control module 2031 receives the analysis results (step S109). The search condition generation module 2033 formats and organizes the received analysis results as needed and stores them as search condition candidates in the search condition table 2023 via the data management module 2043. This series of processes (steps S104 to S109) completes the search condition generation step.

[0181] Next, the first search expression creation module 2034 retrieves the search condition candidates (keywords, related terms, classifications, etc.) stored in the search condition table 2023, and combines them according to predetermined rules (e.g., an OR join-centric approach emphasizing comprehensiveness) to create a first search expression for searching the first database 50 (e.g., Google Patents). The created first search expression is stored in the search expression table 2024 via the data management module 2043 (step S110). This completes the first search expression creation step.

[0182] Next, the first search execution module 2035 retrieves the created first search expression from the search expression table 2024 and requests the first database 50 to execute the search via the communication control module 2031 (step S111).

[0183] The first database 50 receives a search request from the server 20 (step S112) and searches for documents in the database based on the received first search expression (step S113). Once the search is complete, the first database 50 sends at least the number of hits as search results to the server 20 (step S114). If the acquisition evaluation module 2036 performs noise evaluation or the like as described later, it may also send snippet information (title, part of the abstract, etc.) about some or all of the hit documents to the server 20 in addition to the number of hits.

[0184] On server 20, the communication control module 2031 receives the number of hits and other information (step S115). The first search execution module 2035 records the received number of hits, the executed search expression ID, the execution date and time, and other information in the search execution log table 2025 via the data management module 2043.

[0185] Next, as shown in Figure 23, the acquisition evaluation module 2036 in the server 20 evaluates the appropriateness of the first search expression based on the search results in the first database 50 acquired by the first search execution module 2035 (step S116). The main evaluation criterion is the number of hits. The number of hits for the first search expression is obtained from the search execution log table 2025 and compared with the investigation conditions stored in the investigation conditions table 2022 (in this example, a maximum of 300 peer-reviewed items is used as the baseline value, or the lower and upper limits of the target number of hits) to determine whether the number of hits is within the target range (e.g., "too_many", "too_few", "appropriate"). Note that the acquisition evaluation module 2036 may be configured to evaluate the first search expression from other perspectives in addition to, or as an alternative to, the evaluation of the number of hits. For example, as mentioned above, the noise level can be evaluated using the search result snippet information and LLM40 (including the local LLM built into server 20), or the comprehensiveness and specificity of the search expression components (keywords and classifications), and the estimated search cost can be evaluated. These evaluation results (and the evaluation details mentioned above) are recorded in the system evaluation table 2026 via the data management module 2043. The acquisition evaluation step is executed through the search execution (steps S111 to S115) and evaluation (step S116) up to this point.

[0186] The acquisition and evaluation module 2036 (or the overall control module 2042) determines whether the evaluation result meets the conditions (step S117). If the condition is not met, such as if the number of hits is too many (e.g., "too_many") or too few (e.g., "too_few"), the search condition adjustment module 2037 is activated. The search condition adjustment module 2037 adjusts the search condition candidates stored in the search condition table 2023 based on the evaluation results and evaluation details (extracted keyword candidate list, etc.) of the system evaluation table 2026 (e.g., if the number of hits is too many, it adds exclusion candidate keywords included in the evaluation details with a NOT condition, or excludes keywords with low relevance) (step S118). After adjustment, the process returns to step S110, the first search expression is recreated based on the adjusted search conditions, and the search and evaluation in the first database 50 are performed again. This loop is repeated until the evaluation result meets the conditions.

[0187] If the evaluation result meets the conditions (step S117: Yes), the second search expression creation module 2038 is activated. Based on the search conditions that have been adjusted through the loop or that were evaluated as appropriate from the beginning, the second search expression creation module 2038 creates a more accurate second search expression that conforms to the specifications (available operators, fields, etc.) of the second database 60 (e.g., a specific commercial patent database). The created second search expression is stored in the search expression table 2024 (step S119). This completes the second search expression creation step.

[0188] Next, the display control module 2040 retrieves the created second search expression from the search expression table 2024 and transmits it to the terminal device 10 via the communication control module 2031 (step S120).

[0189] The terminal device 10 receives the second search expression data via the communication unit 120 and the transmitting / receiving unit 192 (step S121). The display control unit 193 displays the received second search expression on the output device 14, as shown in Figure 26, on a search expression display / evaluation screen. This screen displays the name of the database corresponding to the second database 60 to be searched (e.g., "○○ Search"), the generated search expression, reference information (e.g., estimated number of hits "300"). This completes the search expression display step.

[0190] The user can review the second search expression displayed in Figure 26 and evaluate its contents (step S122). They can select either the "OK" or "NG" button on the screen, and if "NG" is selected, they can enter a comment in the "Reason for Inappropriateness" field. Alternatively, the generated search expression may be displayed so that the user can edit it, and if "NG" is selected, the user can edit the search expression and receive it as evaluation information. The entered evaluation information is received by the operation reception unit 191. In this way, the terminal device 10 accepts evaluations from the user (step S123).

[0191] When the user enters an evaluation (e.g., by pressing the "OK" button), the transmitting / receiving unit 192 of the terminal device 10 transmits the evaluation information to the server 20 via the communication unit 120 (step S124).

[0192] On server 20, the communication control module 2031 receives the evaluation information (step S125). The evaluation reception module 2041 stores the received evaluation information in the user evaluation table 2027 via the data management module 2043. This completes the search expression evaluation reception step.

[0193] Next, as shown in Figure 24, the overall control module 2042 (or evaluation reception module 2041) on the server 20 determines the received user evaluation (step S126). If the user evaluation is "NG" (step S126: No), the process returns to step S118. At this time, the search condition adjustment module 2037 readjusts the search conditions, taking into account the comments from the user received by the evaluation reception module 2041 and the content of the search expression edited by the user, and the evaluation loop for the first search expression is repeated. Alternatively, the process may be repeated starting from the recreation of the second search expression.

[0194] If the user evaluation is "OK" (step S126: Yes), or if the setting is not to perform a user evaluation, the overall control module 2042 instructs the second search execution module 2039 to perform a search in the second database 60. The second search execution module 2039 retrieves the second search expression from the search expression table 2024 and requests the second database 60 to perform a search via the communication control module 2031 (step S127).

[0195] The second database 60 receives a search request (step S128) and searches for documents within the database based on the second search expression (step S129). Once the search is complete, the second database 60 sends a list of documents found as search results (including information such as document number, applicant, title, abstract, and patent classification for each document) to the server 20 (step S130).

[0196] On server 20, the communication control module 2031 receives the bibliography list information (step S131). The second search execution module 2039 verifies the received information and records it in the search execution log table 2025 along with information such as the executed search expression ID and execution date and time.

[0197] Next, the display control module 2040 performs a sorting process on the received list of documents (step S132). Specifically, the display control module 2040 compares and evaluates the research conditions (texts to be researched) in the research conditions table 2022 with the information of each document (title, abstract, patent classification, etc.), and sorts the list so that documents considered to be more suitable for the research conditions are displayed at the top. LLM40 or similar may be used for scoring relevance.

[0198] The display control module 2040 generates search result data including the second search expression used for execution and the sorted list of literature, and transmits it to the terminal device 10 via the communication control module 2031 (step S133).

[0199] The terminal device 10 receives search result data via the communication unit 120 and the transmitting / receiving unit 192 (step S134). The display control unit 193 displays the received second search expression and sorted list of documents on the output device 14, as shown in Figure 27. This screen displays the search conditions, the search expression used, the number of hits (e.g., "290"), along with a list of documents sorted by relevance (score order) (including document number, title of invention, abstract, etc.). The user can check the list of documents displayed in order of relevance on this screen and proceed with the search (processing ends in step S134). This completes the search result display step.

[0200] In the above-described flow, after creating the second search expression (step S119), the process of first presenting it to the user and requesting their evaluation (steps S120-S125) was explained. However, instead, immediately after creating the second search expression (step S119), prior to the presentation and evaluation process to the user (steps S120-S127), the second search execution module 2039 may execute a search in the second database 60 using the second search expression (corresponding to steps S127-S130) and obtain at least the number of hits (corresponding to step S131). In this case, the display control module 2040 transmits the created second search expression along with the actual number of hits in the second database 60 (or the number of hits plus snippets of some of the search results, etc.) to the terminal device 10 (a variation of step S120), allowing the user to evaluate not only the content of the search expression but also the specific number of hits obtained (a variation of steps S121 and S122). This makes it easier for users to judge the validity of the search expression. And if the user evaluation is "OK" (step S125: Yes), instead of performing a search again in the second database 60, the sorting process (step S132) and subsequent steps may be performed on the search results already obtained (or the complete search results obtained again if necessary).

[0201] <Effects of this embodiment> The information processing system 1, server (information processing device) 20, and programs and information processing methods executed therein, as described above, provide the following advantages.

[0202] According to this embodiment, the server 20's investigation condition receiving module 2032 receives investigation conditions including text related to the technology to be investigated from the terminal device 10 (investigation condition receiving step), the search condition generation module 2033 has the received text analyzed by the LLM (Large-Scale Language Model) 40 to generate candidate search conditions (search condition generation step), the first search expression creation module 2034 combines the generated candidate search conditions to create a search expression (first search expression) in the first database 50 (first search expression creation step), the first search execution module 2035 and the acquisition and evaluation module 2036 perform a search in the first database 50 using the created first search expression, acquire and evaluate the search results (number of hits, etc.) (acquisition and evaluation step), the search condition adjustment module 2037 adjusts the search conditions based on the evaluation results and the original investigation conditions, and the second search expression creation module 2038 creates a search expression (second search expression) in the second database 60 using the adjusted search conditions (second search expression creation step). This allows for the introduction of an objective analysis using LLM and a step-by-step approach involving preliminary searching, evaluation, and adjustment in the first database, in the search formula creation process, which previously often relied on the experience and intuition of the researcher. As a result, it becomes possible to create high-quality search formulas that match the research objectives efficiently and effectively, while reducing reliance on the researcher's skills.

[0203] In particular, preliminary evaluations such as the number of hits can be performed using the first database 50 (for example, a database that can be accessed via API and is expected to provide fast responses, or an internal database tailored to a specific purpose), and the search conditions can be repeatedly adjusted using LLM based on the results, thereby efficiently optimizing the search scope. Furthermore, high-quality search expressions suitable for the second database 60 (for example, a public database like J-PlatPat), which is the final target of the investigation, can be generated, significantly reducing the time and cost of the entire investigation and improving search accuracy.

[0204] Furthermore, in the survey condition acceptance step, the survey condition acceptance module 2032 is configured to accept detailed survey setting conditions such as survey type, survey technology field, target number of search results in the second database 60 (e.g., maximum number of peer-reviewed items), survey importance, and confidentiality settings, in addition to the text to be surveyed (see Figure 25). This allows users to communicate more detailed requests to the system. The server 20 generates, evaluates, and adjusts search conditions considering these detailed setting conditions, enabling it to automatically build and execute a search strategy that more accurately reflects the user's intent and matches their purpose.

[0205] Furthermore, in the search condition generation step, the search condition generation module 2033 divides the text related to the technology under investigation into multiple parts, and then has the LLM40 analyze each of the divided parts individually. This configuration is expected to extract not only an overview of the entire text, but also a variety of search condition candidates (keywords, synonyms, classifications, etc.) based on detailed technical elements and nuances of each part. This improves the comprehensiveness and accuracy of the search conditions, leading to the creation of higher-quality search formulas. In addition, since the text related to the technology under investigation is not sent to the LLM40, it is expected to have the effect of preventing information leakage.

[0206] Furthermore, in the acquisition evaluation step, the acquisition evaluation module 2036 evaluates the number of hits extracted by searching the first database 50 based on the search conditions stored in the search conditions table 2022 (e.g., the maximum number of peer-reviewed items), allowing for an objective understanding of the breadth of the search scope of the first search expression. If it is determined that the number of hits is too many or too few, the search condition adjustment module 2037 automatically or semi-automatically adjusts the search conditions and executes the evaluation loop of the first search expression, optimizing the search scope to an appropriate size before proceeding to the main search in the second database 60. This reduces the burden of subsequent literature reviews and significantly improves the overall efficiency of the research.

[0207] This is because, for example, by using a database like Google Patents as the first database 50, which allows for high-speed searching of a wide range of documents via an API (e.g., using SerpAPI), the number of hits can be quickly grasped and adjusted, while even when using a database without an official API, such as J-PlatPat, as the second database 60, the number of trial-and-error steps in the second database 60 can be significantly reduced by providing the user with an optimized search formula. Furthermore, by recording promising keyword candidates and keyword candidates to be excluded extracted from search result snippets, etc., during evaluation as evaluation details and using this for adjustment, it becomes possible to more accurately narrow down and expand search conditions, further improving the quality of the search formula and the efficiency of the research.

[0208] In addition, the display control module 2040 is configured to display the generated second search expression on the terminal device 10 (see Figure 26), allowing the user to confirm the content of the search expression actually executed in the second database 60. This increases the transparency of the process and reduces the risk of unintended searches being performed for the user.

[0209] Furthermore, the second search execution module 2039 searches the second database 60 using the second search formula, and the display control module 2040 displays the search results (bibliography list) on the terminal device 10 along with the search formula used for execution (see Figure 27). This configuration allows the user to confirm the search results and the search formula that underpinned them, making it easier to judge the validity of the results and to proceed with further investigation.

[0210] In particular, during the search results display step, the display control module 2040 sorts and displays the extracted list of literature using scores calculated based on their relevance to the research conditions (such as the researched text) (see the score column in Figure 27). This allows users to prioritize reviewing literature that is estimated to be highly relevant. As a result, the time spent on literature review is reduced, and the risk of missing important information is minimized.

[0211] Furthermore, the evaluation reception module 2041 receives user evaluations (OK / NG, comments, or direct editing by the user, etc.) for the displayed second search expression (see Figure 26), and if the evaluation is NG, the overall control module 2042 has a feedback loop that causes the search condition adjustment module 2037, etc., to run again. This configuration allows for the integration of AI-based automation and support with the user's expert knowledge. Users can further improve the quality of the search expression by giving correction instructions in the comments or directly editing the search expression generated by the system, ultimately leading to more satisfying search results.

[0212] As described above, this embodiment has the effect of significantly improving the efficiency and quality of search formula creation in searching for patent documents, etc., through the use of LLM, step-by-step evaluation and adjustment, and an interactive improvement process with the user.

[0213] (Basic computer hardware configuration) Figure 28 is a block diagram showing the basic hardware configuration of computer 90. Computer 90 includes at least a processor 94, memory 95, storage 96, and a communication interface 99. These are electrically connected to each other by a bus.

[0214] The processor 94 is hardware for executing the instruction set written in a program. The processor 94 consists of an arithmetic unit, registers, peripheral circuits, etc.

[0215] Memory 95 refers to a type of memory used to temporarily store programs and data processed by those programs. For example, it can be volatile memory such as DRAM (Dynamic Random Access Memory).

[0216] Storage 96 refers to memory devices used to store data and programs. Examples include flash memory, HDDs (Hard Disc Drives), magneto-optical disks, CD-ROMs, DVD-ROMs, and semiconductor memory.

[0217] A communication IF99 is an interface for inputting and outputting signals for communication with other computers via a network using wired or wireless communication standards.

[0218] A network consists of various mobile communication systems built using the internet, LANs, wireless base stations, etc. For example, a network includes 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks that can connect to the internet via designated access points (e.g., Wi-Fi®). When connecting wirelessly, communication protocols include, for example, Z-Wave®, ZigBee®, and Bluetooth®. When connecting via a wired connection, the network also includes connections made directly via USB (Universal Serial Bus) cables, etc.

[0219] Furthermore, by distributing all or part of each hardware configuration across multiple computers 90 and connecting them to each other via a network, a computer 90 can be virtually realized. Thus, the concept of computer 90 includes not only a computer 90 housed in a single enclosure or case, but also a virtualized computer system.

[0220] (Basic Functional Configuration of Computer 90) The functional configuration of the computer realized by the basic hardware configuration of computer 90 shown in Figure 28 will be explained. The computer comprises at least one functional unit: a control unit, a memory unit, and a communication unit.

[0221] Furthermore, the functional units of computer 90 can also be realized by distributing all or part of each functional unit across multiple computers 90 interconnected via a network. The concept of computer 90 includes not only a single computer 90 but also a virtualized computer system.

[0222] The control unit is realized when the processor 94 reads various programs stored in the storage 96, loads them into the memory 95, and executes processing according to those programs. The control unit can realize various functional units that perform information processing depending on the type of program. In this way, the computer is realized as an information processing device that performs information processing.

[0223] The storage unit is implemented by memory 95 and storage 96. The storage unit stores data, various programs, and various databases. The processor 94 can also allocate storage areas corresponding to the storage unit in memory 95 or storage 96 according to the program. The control unit can also cause the processor 94 to perform addition, update, and deletion operations on data stored in the storage unit according to the various programs.

[0224] A database, specifically a relational database, is used to manage and link data sets called tables, which are structurally defined by rows and columns. In a database, tables are called tables, the columns of a table are called columns, and the rows of a table are called records. In a relational database, relationships can be established and linked between tables.

[0225] Typically, each table has a key column to uniquely identify records, but setting a key on a column is not mandatory. The control unit can instruct the processor 94 to add, delete, or update records in specific tables stored in the memory unit according to various programs.

[0226] The communication unit is implemented by the communication IF99. The communication unit provides the functionality to communicate with other computers 90 via the network. The communication unit can receive information transmitted from other computers 90 and input it to the control unit. The control unit can cause the processor 94 to perform information processing on the received information according to various programs. The communication unit can also transmit information output from the control unit to other computers 90.

[0227] The functions realized by the components described herein may be implemented in a circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), CPUs (a Central Processing Unit), conventional circuits, and / or combinations thereof, programmed to realize the functions described herein. A processor includes transistors and other circuits and is considered a circuitry or processing circuitry. A processor may be a programmed processor that executes a program stored in memory.

[0228] In this specification, circuits, units, and means are hardware programmed to perform or execute the functions described herein. Such hardware may be any hardware disclosed herein, or any hardware known to be programmed to perform or execute the functions described herein. If the hardware is a processor that is considered to be a type of circuit, then the circuit, means, or unit is a combination of hardware and software used to constitute the hardware and / or processor.

[0229] While several embodiments of this disclosure have been described above, these embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications are permitted without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents.

[0230] <Variation> The above embodiments illustrate just one example of the present disclosure, and various modifications are possible. Several modifications are shown below.

[0231] (Manual search performed in database 60) It is conceivable that the second database 60 in the above-described embodiment may not provide a direct search execution interface (API) from an external program, such as the Japanese Patent Information Platform (J-PlatPat). In such a case, the automatic search by the second search execution module 2039 is not performed, and instead, user intervention is possible through the following procedure. First, the display control module 2040 displays the search expression string (e.g., a logical expression) for the second database 60 (J-PlatPat in this example), created by the second search expression creation module 2038, on the screen of the terminal device 10 (a screen similar to that in Figure 26). The user obtains the displayed search expression string by means of copying or other means, and manually enters it into the search interface of the second database 60 (J-PlatPat) using a web browser or the like to perform a search.

[0232] Next, the user checks the search results (e.g., number of hits) obtained on the second database 60 (J-PlatPat). The user determines whether this number of hits is appropriate compared to the target range set as a search condition (e.g., upper limit of target number of hits or upper limit of number of peer-reviewed items). If the user determines that the number of hits is inappropriate (too many or too few) or that there seems to be a lot of noise, the user inputs this evaluation (e.g., pressing the "NG" button and entering a comment in Figure 26) into the system via the terminal device 10. The evaluation reception module 2041 receives this evaluation, and the overall control module 2042 can then prompt the system to execute the search condition adjustment step (S118) and the second search expression creation step (S119) again to generate a more appropriate search expression.

[0233] On the other hand, if the user determines that the number of hits is appropriate, the user can retrieve the bibliography from the second database 60 (J-PlatPat) and proceed to the subsequent peer review process. In this way, even if the user's manual operation is involved in executing the search in the second database 60 and retrieving and evaluating the results, it remains within the scope of the technical concept of this disclosure, which is to provide an optimized search formula after generating search conditions using a large-scale language model and conducting preliminary evaluation and adjustments in the first database.

[0234] (Example when using the second database 60, which is accessible via API) The above modified example (manual search execution in the second database 60) assumes that the second database 60 does not provide an API, etc. However, if the second database 60 provides a search execution interface (API) from an external program, the configuration described in this embodiment, in which the second search execution module 2039 of the server 20 automatically executes a search using the second search expression, obtains the results (bibliography list, etc.) and displays them on the terminal device 10 (see steps S127 to S134 in Figure 24), can be applied as is.

[0235] (Variations in database configuration) Furthermore, if a business providing patent search database services becomes the implementer of the technology disclosed herein, the system 1 and server 20 related to this disclosure may be configured to use a literature database independently constructed, owned, and operated by the business as the first database 50 and / or the second database 60. In this case, for example, by using a literature database specialized in a particular technical field or a database with unique additional information (evaluation score, citation information, etc.) as the first database 50 and the second database 60, it may be possible to provide a more specialized and high-value-added search formula creation support service. This configuration corresponds to the form in Figure 1 where the first database 50 and the second database 60 are managed and operated by the business providing the server 20 itself.

[0236] (Variations of text splitting methods) In the above embodiment, the rules for dividing the text to be investigated in the search condition generation step (particularly step S104) are not limited to comma-based division, sentence-based division, or semantic chunk division. Various methods can be employed.

[0237] One variation of text segmentation methods is n-gram segmentation. n-gram segmentation is a technique that divides text into fixed-length (n-character) units, with or without character overlap. This method allows for analysis based not only on specific words or phrases, but also on more subtle character arrangement patterns, potentially leading to the discovery of unexpected keyword candidates.

[0238] Another variation of text segmentation methods is stop word segmentation. Stop word segmentation is a technique that divides text using particles (such as "no," "ga," and "wa") or other predefined stop words, and then extracts and divides the remaining parts as meaningful units. This technique has the effect of enabling analysis that focuses on content words, excluding function words.

[0239] Another variation of text segmentation is fixed-character segmentation. Fixed-character segmentation is a method that simply divides text into predetermined groups of characters (e.g., 100 characters). In this method, as an option, some characters may be overlapped between segmentation units to prevent information loss at the segmentation boundaries. This configuration has the advantage of making the size of the text input units for LLM40 uniform.

[0240] Another variation is input unit segmentation. Input unit segmentation is a method in which, when a user intentionally uses specific delimiters or line breaks when inputting the text to be investigated, the input unit (for example, each paragraph) is used directly as the segmentation unit. This method allows analysis to be performed while maintaining the information grouping intended by the user.

[0241] These various segmentation methods can be configured to be selectable, allowing users to specify the optimal segmentation method according to the purpose of the research and the nature of the text. Changing the segmentation method alters the granularity and focus of the analysis by LLM40, and is expected to have the effect of adjusting the nature of the resulting search condition candidates (extensiveness, specificity, etc.).

[0242] (Variations of processing according to confidentiality) It is also conceivable that the text segmentation method and the location where the analysis process is executed may be changed depending on the value of the "confidentiality setting flag" (confidentiality_flag in the investigation conditions table 2022) accepted as an investigation condition, or according to the separately set confidentiality level.

[0243] One variation of processing based on confidentiality is adjusting the granularity of the splitting. When the information is deemed highly confidential (e.g., the confidentiality setting flag is set to True), the text is split into smaller units that make it difficult to guess the meaning, such as n-gram splitting or short, fixed-character splitting, before being sent to LLM40. This configuration reduces the confidentiality of the information sent externally at once, further reducing the risk of leakage. Conversely, when the information is less confidential, it is split into relatively large units, such as sentences, which encourages LLM40 to perform analysis that takes context into account more carefully.

[0244] Another variation of processing depending on the level of confidentiality is the use of a local LLM. In this configuration, when dealing with particularly confidential texts under investigation, the control unit 203 of the server 20 performs text analysis (the processing corresponding to step S107) using a local large-scale language model (local LLM) located within the server 20 or on a secure internal network, rather than using an external LLM 40. This configuration ensures extremely high information security because the content of the text under investigation never leaves the organization's control. In this case, the local LLM may be implemented as software running on the processor 29 of the server 20, or it may be incorporated into the server 20 as dedicated AI processing hardware.

[0245] (Examples of modifications using applicant information, etc.) When generating and adjusting search criteria, it is also possible to utilize information about applicants related to the research case (such as the user themselves or competitors).

[0246] One possible modification utilizing applicant information is the filtering and weighting of search criterion candidates. This involves the search criterion generation module 2033, after receiving search criterion candidates from LLM40 (after step S109), excluding candidates that are less relevant to the technical fields typically handled by the applicant identified as a search criterion (e.g., obtained from user table 2028, etc.) (e.g., estimated from information such as patent classification table 2029), or conversely, increasing the importance (priority when creating a search formula) of highly relevant candidates. This process is expected to enable the creation of more efficient search formulas that focus on the technological trends of specific applicants.

[0247] One variation that utilizes other applicant information is prompt adjustment to the LLM. This method involves including information about relevant applicants and their primary technical fields in the prompt along with the text to be investigated when sending the analysis request to the LLM40 in step S105. This may guide the LLM40 to generate keywords and classifications that are likely to be important in the context of that applicant more accurately.

[0248] (Variations regarding the division of labor) In the above embodiment, each functional module within the server 20 (for example, the search condition adjustment module 2037, the first search expression creation module 2034, the second search expression creation module 2038, the display control module 2040, etc.) was described as executing processing based on specific algorithms and rules. However, some or all of these processes may be performed by an external or internal LLM 40.

[0249] For example, in the acquisition evaluation step (step S116, etc.), if the number of hits falls outside the target range, instead of the search condition adjustment module 2037 generating specific adjustment proposals for how to adjust the search conditions (e.g., adding or deleting keywords, changing operators, changing the classification hierarchy, etc.), or as an auxiliary function, a configuration could be considered in which the evaluation results and current search conditions are input to the LLM40 and requested. Alternatively, instead of the first search expression creation step (step S110) and the second search expression creation step (step S119), which combine candidate search conditions (keywords, classifications, etc.) to match the search syntax of a specific database (first database 50 or second database 60), being executed by the first search expression creation module 2034 and the second search expression creation module 2038, a configuration could be considered in which the relevant candidate search conditions and database specification information are directly instructed to the LLM40 to generate the search expression string. Furthermore, in the process of sorting the search results (list of references) obtained from the second database 60 based on their relevance to the research conditions (step S132), instead of the display control module 2040 performing the calculation of the relevance score for each reference using its internal logic, it is also conceivable to configure the LLM40 to evaluate the information on each reference and the research conditions. This may simplify the implementation on the server 20 side, and by utilizing the LLM40's advanced language comprehension and reasoning capabilities, it is possible to achieve more contextual adjustments to search conditions, the construction of optimal search formulas, and highly accurate relevance evaluations.

[0250] (Variations regarding the use of no-code / low-code platforms) In the above embodiment, the server 20 was described as having a configuration with specific functional modules, but some or all of these functions and processing flows may be implemented using a so-called no-code or low-code AI application development platform. For example, a platform such as LangGenius's Dify could be used.

[0251] When using such a platform, the functions corresponding to the control unit 203 of the server 20 are realized by workflows and logic defined on the platform. The user (or system builder) can construct all or part of the search formula creation process according to the present invention (see Figures 22 to 24) by visually combining the creation of prompt templates, setting context variables (search condition text, search type, target number of hits, etc.), arranging LLM (Large-Scale Language Model) call nodes, setting external API (corresponding to the first database 50, second database 60, etc.) linkage functions such as database searches, conditional branching (processing branching based on evaluation results, etc.), and data conversion processing via a graphical user interface (GUI), etc. Furthermore, the screens shown in Figures 25 to 27 can also be easily constructed using templates, etc.

[0252] This configuration has the effect of making it possible to relatively easily build, test, and modify the information processing device and information processing system according to the present invention without performing complex programming. Furthermore, by utilizing the functions provided by the platform, it may be possible to flexibly integrate with various LLMs and external tools.

[0253] In variations utilizing this no-code / low-code platform, the large-scale language models used in steps such as the search condition generation step (e.g., the LLM call corresponding to step S107), the search condition adjustment step (e.g., part of step S118), or the search result sorting process (e.g., relevance scoring in step S132) may be external LLM40s that the platform supports by default or can integrate with (e.g., API services such as OpenAI's GPT series, Google's Gemini series, or Anthropic's Claude series).

[0254] Alternatively, similar to the use of local LLMs described above, from the standpoint of confidentiality, the system may be configured to use a local LLM that is embedded within the platform environment itself or hosted on a secure server environment accessible from the platform. Examples of usable local LLMs include, but are not limited to, Google's Gemma series (e.g., Gemma 3), DeepSeek's DeepSeek series, and Alibaba's Qwen series. If the platform supports hosting and calling these local LLMs, users can selectively use or combine configurations that utilize external APIs and configurations that utilize local LLMs, depending on the confidentiality of the research case. This ensures the advantage of being able to perform the processing of the present invention without sending highly confidential research texts, etc., to external services, while enjoying the ease of no-code / low-code development.

[0255] These variations may be used individually or in combination.

[0256] Furthermore, this technology can take the following configuration. (1) A program for operating a computer comprising a processor and memory, wherein the program is configured to operate the processor, A survey conditions acceptance step that accepts survey conditions including text related to the technology to be surveyed, A search condition generation step involves inputting the subject technology to be investigated, received in the aforementioned investigation condition acceptance step, into a large-scale language model for analysis and generating search conditions, and A first search expression creation step involves creating a search expression in the first database by combining the search conditions generated in the search condition generation step, A search is performed in the first database using the search expression created in the first search expression creation step, and the search results are obtained and the search expression is evaluated in an acquisition evaluation step. A second search expression creation step involves creating a search expression for the second database by adjusting the search conditions generated in the search condition generation step based on the search conditions received in the aforementioned search condition acceptance step, A program that executes the command. (2) The program described in (1) above, wherein in the step of receiving the survey conditions, the program further accepts detailed survey conditions, such as the type of survey, the field of the survey, the maximum number of results in the search population as search results in the second database, the maximum number of peer-reviewed items, the importance of the survey, and confidentiality settings. (3) The program according to (1) or (2) above, wherein in the search condition generation step, the text relating to the subject technology received in the survey condition reception step is divided into multiple parts, and then the divided multiple parts are individually analyzed by the large-scale language model. (4) A program according to any one of the above items (1) to (3), which performs a search in the first database in the acquisition evaluation step and evaluates the number of results extracted as the search results based on the survey conditions. (5) The program according to any one of (1) to (4) above, which causes the processor to further execute a search expression display step that displays the search expression in the second database generated in the second search expression creation step. (6) The program according to any one of the above items (1) to (4), wherein the program causes the processor to perform a search in the second database with respect to the search expression generated in the second search expression creation step, and to further perform a search result display step in which the search results are displayed together with the generated search expression in the second database. (7) The program described in (6) above, which, in the search result display step, rearranges and displays the individual documents extracted in the search results in an order more suitable to the research conditions. (8) The program according to (6) above, that causes the processor to execute a search expression evaluation reception step of receiving an evaluation for the search expression displayed in the search expression display step, and to re-execute at least the first search expression creation step, the acquisition and evaluation step, and the second search expression creation step. (9) A method executed by a computer comprising a processor, the method comprising: the processor: a survey condition reception step of receiving a survey condition including text related to a target technology for survey; a search condition generation step of inputting the target technology for survey received in the survey condition reception step into a large language model for analysis to generate a search condition; a first search expression creation step of combining the search conditions generated in the search condition generation step to create a search expression for a first database; an acquisition and evaluation step of performing a search in the first database using the search expression for the first database created in the first search expression creation step to obtain a search result, and evaluating the search expression; a second search expression creation step of adjusting the search conditions generated in the search condition generation step based on the survey condition received in the survey condition reception step, to create a search expression for a second database; executing the above steps. (10) An information processing apparatus comprising a control unit, the control unit: a survey condition reception step of receiving a survey condition including text related to a target technology for survey; a search condition generation step of inputting the target technology for survey received in the survey condition reception step into a large language model for analysis to generate a search condition; a first search expression creation step of combining the search conditions generated in the search condition generation step to create a search expression for a first database; A search is performed in the first database using the search expression created in the first search expression creation step, and the search results are obtained and the search expression is evaluated in an acquisition evaluation step. A second search expression creation step involves creating a search expression for the second database by adjusting the search conditions generated in the search condition generation step based on the search conditions received in the aforementioned search condition acceptance step, An information processing device that performs the following actions. (11) A means for receiving survey conditions, which includes texts relating to the technology under investigation, A search condition generation means generates search conditions by inputting the subject technology to be investigated, received by the aforementioned investigation condition reception means, into a large-scale language model for analysis, and A first search expression creation means that creates a search expression in the first database by combining the search conditions generated by the search condition generation means, An acquisition and evaluation means that performs a search in the first database using the search expression created by the first search expression creation means, obtains the search results, and evaluates the search expression; A second search expression creation means adjusts the search conditions generated by the search condition generation means based on the search conditions received by the search condition receiving means to create a search expression for the second database, A system that is equipped with [the following].

[0257] The aspects of this disclosure are not limited to the individual embodiments described above, but include various modifications that a person skilled in the art could conceive, and the effects of this disclosure are not limited to those described above. In other words, various additions, modifications, and partial deletions are possible, as long as they do not depart from the conceptual idea and spirit of this disclosure derived from the claims and their equivalents. [Explanation of Symbols]

[0258] 1. Information Processing System 10 Terminal devices 12, 22, 99 Communication IF 13 Input device 14 Output device 15, 25, 95 memory 16, 26, 96 storage 19, 94 processors 20 Servers (Information Processing Devices) 23 Input / Output Interfaces 29 processors 40. LLM (Large-Scale Language Model) 50 First Database 60 Second Database 80 Networks 90 Computer 120 Communications Department 180 Storage section 181 User Information 190 Control Unit 191 Operation Reception Section 192 Transceiver Unit 193 Display Control Unit 201 Communications Department 202 Storage section 203 Control Unit 2021 Survey Project Management Table 2022 Survey Conditions Table 2023 Search Criteria Table 2024 Search Query Table 2025 Search Execution Log Table 2026 System Evaluation Table 2027 User Rating Table 2028 User Table 2029 Patent Classification Table 2031 Communication Control Module 2032 Survey Conditions Acceptance Module 2033 Search Criteria Generation Module 2034 First Search Formula Generation Module 2035 First Search Execution Module 2036 Acquisition Evaluation Module 2037 Search Criteria Adjustment Module 2038 Second Search Formula Creation Module 2039 Second Search Execution Module 2040 Display Control Module 2041 Evaluation reception module 2042 Overall control module 2043 Data management module

Claims

1. A program for operating a computer comprising a processor and memory, wherein the program is configured to operate the processor, A survey conditions acceptance step that accepts survey conditions including text related to the technology to be surveyed, A search condition generation step involves inputting the subject technology to be investigated, received in the aforementioned investigation condition acceptance step, into a large-scale language model for analysis and generating search conditions, and A first search expression creation step involves creating a search expression in the first database by combining the search conditions generated in the search condition generation step, A search is performed in the first database using the search expression created in the first search expression creation step, the search results are obtained, and the search expression is evaluated in an acquisition evaluation step. A second search expression creation step involves creating a search expression for the second database by adjusting the search conditions generated in the search condition generation step based on the search conditions received in the aforementioned search condition acceptance step, A program that executes the command.

2. In the aforementioned survey condition acceptance step, the survey conditions further accept detailed survey setting conditions, which may include the type of survey, the field of study, the maximum number of results in the search population in the second database, the maximum number of peer-reviewed items, the importance of the survey, and confidentiality settings. The program according to claim 1.

3. In the search condition generation step, the text relating to the target technology received in the survey condition reception step is divided into multiple parts, and then each of the divided parts is individually analyzed by the large-scale language model. The program according to claim 1.

4. In the acquisition evaluation step, the number of items extracted as search results from the first database is evaluated based on the survey conditions. The program according to claim 1.

5. The aforementioned processor, A search expression display step is used to display the search expression in the second database generated in the second search expression creation step, The program described in claim 1, which is to be executed further.

6. The aforementioned processor, A search result display step is performed to execute a search in the second database using the search expression generated in the second search expression creation step, and to display the search results, which are the results of the search, along with the generated search expression in the second database. The program described in claim 1, which is to be executed further.

7. In the search results display step, the individual documents extracted in the search results are rearranged and displayed in an order that is more suitable to the research conditions. The program according to claim 6.

8. The aforementioned processor, The search expression evaluation acceptance step is executed to receive an evaluation of the search expression displayed in the search expression display step, and the first search expression creation step, the acquisition evaluation step, and the second search expression creation step are executed at least again. The program according to claim 6.

9. A method performed by a computer equipped with a processor, The aforementioned processor, A survey conditions acceptance step that accepts survey conditions including text related to the technology to be surveyed, A search condition generation step involves inputting the subject technology to be investigated, received in the aforementioned investigation condition acceptance step, into a large-scale language model for analysis and generating search conditions, and A first search expression creation step involves creating a search expression in the first database by combining the search conditions generated in the search condition generation step, A search is performed in the first database using the search expression created in the first search expression creation step, the search results are obtained, and the search expression is evaluated in an acquisition evaluation step. A second search expression creation step involves creating a search expression for the second database by adjusting the search conditions generated in the search condition generation step based on the search conditions received in the aforementioned search condition acceptance step, How to do it.

10. An information processing apparatus comprising a control unit, The control unit, A survey conditions acceptance step that accepts survey conditions including text related to the technology to be surveyed, A search condition generation step involves inputting the subject technology to be investigated, received in the aforementioned investigation condition acceptance step, into a large-scale language model for analysis and generating search conditions, and A first search expression creation step involves creating a search expression in the first database by combining the search conditions generated in the search condition generation step, A search is performed in the first database using the search expression created in the first search expression creation step, the search results are obtained, and the search expression is evaluated in an acquisition evaluation step. A second search expression creation step involves creating a search expression for the second database by adjusting the search conditions generated in the search condition generation step based on the search conditions received in the aforementioned search condition acceptance step, An information processing device that performs the following actions.

11. A means for accepting survey conditions, which includes text related to the technology under investigation, A search condition generation means generates search conditions by inputting the subject technology to be investigated, received by the aforementioned investigation condition reception means, into a large-scale language model for analysis, and A first search expression creation means that creates a search expression in the first database by combining the search conditions generated by the search condition generation means, An acquisition and evaluation means that performs a search in the first database using the search expression created by the first search expression creation means, obtains the search results, and evaluates the search expression; A second search expression creation means adjusts the search conditions generated by the search condition generation means based on the search conditions received by the search condition receiving means to create a search expression for the second database, A system that is equipped with [the following].

Images