Real estate information extraction system, real estate information extraction method, and real estate information extraction program using multiple large-scale language models

Abstract

A real estate information extraction system, method, and program are provided that, when realizing automatic extraction of real estate information, stably extracts information while absorbing variations in the accuracy of large-scale language models (LLMs). [Solution] A real estate information extraction system that uses real estate documents as input and has multiple LLMs 1 to 3 recognize strings and match candidates for each item, then scores and selects the estimated results. The cloud server is equipped with a reading instruction means 210 that recognizes strings written in the real estate documents from the user terminal for each item and obtains read values, a master value estimation means 220 that presents a master candidate list corresponding to each item and allows the estimation of a master value based on the read values ​​and obtains the estimated results, an evaluation means 230 that evaluates the master values ​​for each item obtained from the multiple LLMs by assigning a basic score set for each model and for each item, and an selection means 240 that determines an adopted master value for each item based on the evaluation.

Description

[Technical Field]

[0001] The present invention relates to a technology for automatically extracting property information such as location, price, structure, zoning, and facilities listed in property documents used in real estate business, and in particular to a real estate information extraction system, real estate information extraction method, and real estate information extraction program that extract real estate information by utilizing a large-scale language model. [Background technology]

[0002] In real estate brokerage, real estate property information known as "Mysoku" is widely used to distribute information between real estate companies. Mysoku is a document that compiles information such as location, price, structure, zoning, floor plan, and facilities on a single sheet of paper, PDF, or image data, and is still often exchanged via fax or email. Registering the information contained in this Mysoku into a company's property management system or website is a fundamental and important task in real estate brokerage.

[0003] Several studies and patent applications have been proposed in the past regarding the automatic extraction of such real estate information.

[0004] Non-Patent Document 1 (Information Extraction from MySoku PDFs Using Large-Scale Language Models) reports a method of extracting property information by OCR-processing MySoku PDFs and inputting the text into a large-scale language model (GPT-3.5 or GPT-4). This research shows that extraction is possible with a certain degree of accuracy (macro average 0.936) even for MySokus with complex layouts.

[0005] Non-Patent Document 2 (Improving Information Extraction from MySoku PDFs Using Large-Scale Language Models) examines a method for extracting item information end-to-end by inputting MySoku PDFs directly into a multimodal LLM without using OCR. In this research, comparative experiments were conducted using Gemini and Claude, and cases were reported in which the accuracy exceeded that of the OCR+LLM method.

[0006] Patent Document 1 (JP 2024-136122 A) discloses an information reading system equipped with a mechanism for allowing the user to approve values ​​corresponding to predetermined items for character strings recognized by OCR. This method describes a mechanism for obtaining predetermined item information through a UI that highlights the recognition results and accepts approval or correction.

[0007] Patent Document 2 (Patent Publication No. 7430437) describes an information processing device that generates prompts for a large-scale language model based on character strings acquired by OCR, inputs them, and extracts specified items. This method discloses a prompt design that outputs "unknown" for unknown items, multiple types of judgment, completion by web search, and the inclusion of a table data structure in the input. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Application Laid-Open No. 2024-136122 [Patent Document 2] Patent No. 7430437 [Non-patent literature]

[0009] [Non-Patent Document 1] Information Extraction from MySoku PDFs Using Large-Scale Language Models, Proceedings of the 30th Annual Conference of the Association for Natural Language Processing (March 2024), Hongo et al. [Non-patent document 2] Improving information extraction from MySoku PDFs using large-scale language models, The 39th Annual Conference of the Japanese Society for Artificial Intelligence, 2025, Higuchi et al. Summary of the Invention [Problem to be solved by the invention]

[0010] As such, in the prior art, two methods have been considered: one in which character strings obtained by OCR are input into an LLM to extract items, and the other in which character strings are extracted by directly inputting PDFs or images into a multimodal LLM. However, the former depends on the accuracy of the OCR, and the latter depends on the accuracy of the multimodal LLM, and both methods have a fundamental problem in that they are dependent on the performance of the recognition processing that is their premise.

[0011] Therefore, in order to realize automatic extraction of real estate information, such as MySoku, at a practical level, a system is required that can extract information stably while absorbing variations in accuracy, without relying on the accuracy of OCR or a single LLM.

[0012] Therefore, the present invention aims to realize a real estate information extraction system, a real estate information extraction method, and a real estate information extraction program that can stably extract information while absorbing variations in the accuracy of LLM when automatically extracting real estate information. [Means for solving the problem]

[0013] The real estate information extraction system of the present invention is configured to use real estate documents (PDF or image) as input, have multiple large-scale language models (LLMs) recognize character strings and match candidates for each item, and evaluate and select the estimated results. Specifically, it includes: (i) a reading instruction means for recognizing character strings written in the real estate documents for each item (location, price, structure, zoning, facilities, etc.) and obtaining read values, (ii) a master value estimation means for presenting a master candidate list corresponding to each item, having the system estimate a master value based on the read values, and obtaining the estimated results, (iii) an evaluation means for assigning a basic score set for each model and each item to the master values ​​for each item obtained from the multiple LLMs and evaluating them, and (iv) an selection means for determining an adopted master value for each item based on the evaluation.

[0014] The evaluation means adds up the basic scores assigned to the values ​​when multiple LLMs return the same master value, and if there are no identical master values, determines the selected candidate based on the maximum basic score of each candidate. Furthermore, the system includes a score update means that adds a score to the basic score based on the history of user adoptions or corrections (correction history) during operation to improve the evaluation. The score update means assigns an additional value according to the number of times each master value has been adopted in the correction history, and can reduce the influence according to the elapsed time of the history.

[0015] For items such as equipment and features that have a large number of candidates and a variety of expressions, in addition to LLM estimation, it is possible to apply at least one of an external library means that inputs the read value into an external similarity calculation library to determine a master value, or a dictionary means that compares the read value with a dictionary and normalizes it to a standard notation.

[0016] The system is provided with a difference notification means for, when the master values ​​estimated by the multiple LLMs differ, sending a difference notification indicating the discrepancy to the user terminal and displaying the estimated master values ​​of each LLM side by side. Also, a correction history storage means is provided for storing the user's selection result (adoption or correction) in response to the side-by-side display as a history, and the evaluation means uses this correction history to evaluate the master value.

[0017] The present invention is also realized as a method and a program corresponding to the above system configuration. That is, the present invention is a real estate information extraction method including the steps of: inputting real estate documents, acquiring readings for each of a plurality of items, estimating a master value based on a master candidate list, evaluating the estimated results using basic scores for each model and item, and determining an adopted master value based on the evaluation; and a program for causing a computer to execute each step of the method.

[0018] Furthermore, as a modified example, it is also possible to omit returning the read values ​​and estimate and obtain only the master values ​​for each item. In this case, for items for which a candidate list is defined (structure, zoning, etc.), the master values ​​are estimated based on the master candidate list, and for items for which no candidate list is provided (location, price, etc.), the evaluation and selection process is carried out while completing the missing information by means of user input, completion using an external library or dictionary, or treating as unknown. These configurations are realized independently as each aspect of the system, method, and program. [Effects of the Invention]

[0019] According to the present invention, it is possible to provide a real estate information extraction system, a real estate information extraction method, and a real estate information extraction program that can stably extract information while absorbing variations in the accuracy of LLM when realizing automatic extraction of real estate information.

[0020] That is, as described in claim 1, by issuing read instructions to multiple large-scale language models and estimating read values ​​and master values, it becomes possible to compare the estimation results of multiple models without relying on the accuracy of a single model, and variations in extraction accuracy can be absorbed.

[0021] Furthermore, as described in claims 2 and 3, by assigning a basic score to each model and item and determining the adopted master value based on the sum of identical values ​​or comparison of the maximum score, the agreement between multiple models is reflected as reliability, and even if there are discrepancies, it becomes possible to select the most appropriate value rationally. This improves the stability of the extraction results.

[0022] Furthermore, as described in claims 4 to 6, by providing a score update means that adds scores based on the history of user adoptions or corrections, it is possible to dynamically adjust the evaluation parameters by successively reflecting feedback from actual operation. This allows the correct master value to be stably selected based on the correction history even when spelling variations or incorrect mappings occur repeatedly. Furthermore, erroneous learning can be avoided by reducing the degree of influence over time.

[0023] Furthermore, as described in claims 7 to 9, for items such as equipment and features that have a large number of candidates, by configuring the system to use external libraries and dictionaries, it becomes possible to make fast and stable judgments even for items that are unstable when using LLM alone.

[0024] Furthermore, as described in claims 10 to 12, when differences occur between multiple models, a difference notification is generated and displayed side by side on the user terminal, and the user's selection results are saved as a history. This allows the system to automatically extract data and the user to confirm and correct it, thereby improving extraction accuracy and practical reliability.

[0025] Furthermore, as described in claims 15 to 17, by adopting a modified example in which only the master value is estimated without obtaining a read value, the master value can be efficiently estimated for items for which a candidate list exists, and flexible operation is possible in which the items for which a candidate list does not exist can be left to user input or external completion.

[0026] As described above, this invention evaluates the estimation results of multiple LLMs based on base scores and history weighting, and combines discrepancy notification and feedback from user corrections to achieve "stable automatic extraction while absorbing variations in accuracy," which was difficult with conventional technology. This improves the efficiency of real estate document registration work and the reliability of extracted data.

[0027] The problem with reading methods using a single large-scale language model is that it is not possible to determine the accuracy of the recognition results for each item in advance, and users are forced to visually check all items. In contrast, in this invention, multiple large-scale language models analyze the same data in parallel, and the estimation results are compared and evaluated, scored, and displayed. If there are any differences, the different items are displayed so that they can be easily identified, allowing the reliability of each item to be visually grasped, thereby resolving this issue.

[0028] The objects, features, configurations, operations, and effects of the present invention and embodiments of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0029] [Figure 1] FIG. 1 is a functional block diagram showing the overall configuration of a real estate information extraction system according to one embodiment of the present invention. [Figure 2A] FIG. 2A is an example of a screen (basic information tab) displayed on the user terminal 100. [Figure 2B] FIG. 2B is an example of a screen (surrounding environment tab) displayed on the user terminal 100. [Figure 2C] FIG. 2C is an example of a screen (transport tab) displayed on the user terminal 100. [Figure 2D] FIG. 2D is an example of a screen (facilities tab) displayed on the user terminal 100. [Figure 2E] FIG. 2E is an example of a screen (management information tab) displayed on the user terminal 100. [Figure 3] FIG. 3 shows an example of a difference notification displayed on the user terminal 100. [Figure 4] Figure 4 shows an example of the LLM basic scores set for each LLM and for each item. [Figure 5] Figure 5 shows an example of the correction history and the total score after history weighting. [Figure 6] FIG. 6 is a flowchart showing the overall operation of the real estate information extraction system according to one embodiment of the present invention. [Figure 7] FIG. 7(a) shows a conventional example of displaying the results recognized by a single LLM, and FIG. 7(b) shows an example of display using a multiple LLM configuration according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0030] (Overall system configuration) Figure 1 is a functional block diagram showing the overall configuration of a real estate information extraction system according to one embodiment of the present invention. In Figure 1, the real estate information extraction system includes a user terminal 100, a cloud server 200, and a large-scale language model group 300, all of which are communicatively connected via a network (not shown).

[0031] The user terminal 100 is an information processing device such as a personal computer, a tablet terminal, or a smartphone. Cloud server 200 is an information processing device such as a server device or a storage device, and may be made up of one or more information processing devices. The large-scale language model group 300 includes multiple large-scale language models. Each large-scale language model may be provided by the same operator or by different operators. Here, LLM1, LLM2, and LLM3 are illustrated as examples, but any number of models may be used as long as they are multiple. Typically, the cloud server 200 can exchange prompts and responses with each large-scale language model by communicating with each large-scale language model using an API.

[0032] (Configuration of user terminal 100) The user terminal 100 includes at least an input unit 101 and a display unit 102. The input unit 101 has a function of transmitting a PDF file or an image file, which is real estate information (hereinafter, sometimes referred to as "my information"), to the cloud server 200.

[0033] As shown in FIGS. 2A to 2E, the display unit 102 displays at least (a) a MySock preview area, (b) a list area of ​​read values, and (c) a list area of ​​registered data on the screen.

[0034] Here, the terms used in this specification will be explained. The "reading value" refers to the recognition result for each item extracted from MySoc by the cloud server 200 using each large-scale language model.

[0035] "Items" are items such as location, price, structure, zoning, and facilities listed on MySoku. There are two types of "items": items for which you select and register values ​​from pre-determined candidates (structure, zoning, facilities, etc., referred to as "item A"), and items for which you freely register values ​​(location, price, area, etc., referred to as "item B"). Some items (parking lots, surrounding facilities, etc.) contain both types of items A and B at the same time.

[0036] "Registered data" refers to values ​​registered in an item. Values ​​registered in item A include candidate values ​​estimated from candidates predetermined by multiple large-scale language models based on the read value, candidate values ​​estimated by similarity search using dictionaries or external libraries, and values ​​adopted from the candidates. Values ​​registered in item B include read values ​​extracted by multiple large-scale language models and values ​​adopted from the read values.

[0037] "Master value" is a concept that comprehensively indicates the value (information) registered in an item, and includes not only the value that is actually registered, but also the value that is estimated or extracted as a candidate. Note that a master value estimated by a large-scale language model, a dictionary, or an external library may also be referred to as an estimated master value, a master value candidate, a candidate master value, or an adopted master value, depending on the context. As mentioned above, there are two types of items, item A and item B, but in this invention, the value registered in either item will be called a "master value."

[0038] A "master candidate list" is a list of values ​​that can be used as a "master value." Depending on the context, it may also be called a "candidate list." The "master candidate list" is used for item A.

[0039] 2A to 2E show examples of images displayed on the display unit 102. FIG. In Figure 2A, the items under the "Basic Information" tab include property type, location, price, land area, building area, floor plan, construction date, structure, roof structure, number of floors, main lighting surface, building confirmation number, renovation, private road burden, land rights, land use, city planning, zoning, building coverage ratio, floor area ratio, and parking. In this example, property type, floor plan, structure, roof structure, main lighting surface, land rights, land use, city planning, and zoning are Item A. Other items are Item B.

[0040] In Figure 2B, the items under the "Surrounding Environment" tab include elementary school district, junior high school district, surrounding facilities, and road access. In this example, the elementary school district and junior high school district are item A. For surrounding facilities and road access, the area where you can select government offices, financial institutions, convenience stores, etc. is item A, and the actual names are item B.

[0041] In Figure 2C, the items under the "Transportation" tab are Line and Station 1, Line and Station 2, and Line and Station 3. In this example, the line and station name is item A. The rest are item B.

[0042] In FIG. 2D , the items under the "Facilities" tab include bathroom / toilet, balcony / garden, kitchen, heating / cooling, storage, equipment / functions, security, TV / communications, and parking / bicycle parking (these are "facility" items in the narrow sense), as well as features. In this example of items, all of the items are item A. As will be described later, in this embodiment, for the items shown in FIG. 2D , multiple large-scale language models are not used to compare these items with a master candidate list, but rather master values ​​are estimated using a similarity search of a dictionary or external library. However, for facilities and features, multiple large-scale language models may also be used to compare these items with a master candidate list to estimate candidate master values.

[0043] In Figure 2E, the items in the "Management Information" tab are Current Status, Delivery, Original Contractor, and Memo (Confidential). In this example, Current Status and Delivery are Item A. The rest are Item B.

[0044] The display unit 102 can present candidate master values ​​estimated by multiple large-scale language models for each item (location, price, structure, zoning, facilities, etc.) side by side. If multiple candidate master values ​​differ, a discrepancy notice is displayed to indicate this (whether there are differences and the number of differences), and each candidate master value for the item is presented in a selectable format (e.g., radio buttons, pull-down menus, or selection lists). Based on this presentation, the user can select a candidate that differs from the master value adopted by the cloud server 200. Figure 3 shows an example of a discrepancy notice. In the example of Figure 3, the discrepancy notice displays the relevant item surrounded by a bold frame, and candidates extracted by other LLMs are displayed as a selection list.

[0045] If the user selects a different candidate, the display unit 102 accepts the selection, and the user terminal 100 transmits the selection result (correction information) to the cloud server 200. The cloud server 200 stores the received correction information as a correction history, and can update the parameters for scoring by the evaluation means 230 (described later) based on the correction history as needed. This allows for continuous improvement of registration accuracy through actual operation.

[0046] If the user finds an error in the read value or the extracted master value on the display unit 102, the registered data for that item may be directly corrected. Such correction input is also sent to the cloud server 200 as the above-mentioned correction information, and is used to store the correction history and update the score.

[0047] The specific UI configuration of the display unit 102 can be realized using well-known web browser technology, and the zooming in and out of the MySock preview image, page switching, highlighting of candidate master values, highlighting of different items, registration buttons, item display formats, etc. can be designed arbitrarily.

[0048] In addition, the user terminal 100 may be implemented on a smartphone or tablet using a dedicated application, and in this case, the basic functions of (a) previewing myosoku, (b) displaying readings, (c) displaying registered data, and (d) accepting registrations by the user remain the same.

[0049] (Configuration of cloud server 200) Cloud server 200 is a device that determines a master value for each item of real estate documents based on the real estate documents received from user terminal 100, registers the master values ​​in property database 295, and returns display data to user terminal 100. Cloud server 200 includes at least read instruction means 210, master value estimation means 220, evaluation means 230, selection means 240, correction history storage means 250, score update means 260, external library means 270, dictionary means 280, and registration / output unit 290.

[0050] The reading instruction means 210 prompts a group of multiple large-scale language models 300 to extract "reading values" of specified items from real estate documents (PDF or images) uploaded from the user terminal 100, and receives the results.

[0051] The master value estimation means 220 presents a master candidate list corresponding to each item along with a prompt to read the read value, and executes the prompt to multiple large-scale language models 300 to estimate a "master value" based on matching the read value with the candidate list, and receives an estimated master value for each item.

[0052] The prompt to extract a read value and the prompt to estimate a master value by matching candidates may be executed in a single prompt. That is, a single prompt can instruct the LLM to (i) extract a read value, (ii) estimate a master value by matching it with a master candidate list, and (iii) return both the read value and the master value in a predetermined output format (e.g., JSON). The above prompts (i) to (iii) may be divided into two or more parts.

[0053] For items that do not have a candidate list, such as location or price, or items with many candidate options, such as facilities or features, the LLM returns only the readings for those items.

[0054] An example of a prompt used by the reading instruction means 210 and the master value estimation means 220 is as follows:

[0055] (Example prompt) You will be asked to enter a PDF file (real estate property information = MySoku). <Task> 1) Extract the "reading value" for each specified item from the PDF contents. 2) Check the extracted readings against the given "master candidate list" and select the "master value" for each item. 3) Return both the reading value and the master value (or only the reading value depending on the item) according to the specified output format example. <Examples of target items> Location (no master candidate) / Price (no master candidate) / Structure / Land use area / Facilities (no master candidate) <Master candidate list> Structure: ["wooden", "steel frame", "reinforced concrete", "SRC", "other"] Zoning: ["Type 1 Low-Rise", "Type 2 Low-Rise", "Type 1 Residential", "Type 2 Residential", "Semi-Residential", "Commercial", "Neighborhood Commercial", "Semi-Industrial", "Industrial Zone", "Industrial Only", "Other"] <Example of output format> { "Location":"Aobadai, Aoba Ward, Yokohama City", ”Price”:”5500”, "Structure (reading value)": "Wooden, two-story building", "Structure (master value)": "Wooden", "Use district (reading value)": "Type 1 low-rise", "Use district (master value)": "Type 1 low-rise", “Equipment”:[ "City gas", "Waterworks", "Sewer" ] }

[0056] The above prompt example is an example of one embodiment of the present invention, and the specific appearance of the prompt statement expression and output format can be changed as desired. For example, it can be configured to return the read value and master value of each item in a nested JSON format, or to return "Unknown" if no entry exists. The present invention also encompasses these variations.

[0057] The evaluation means 230 has a function of assigning a score set for each model and item to the estimated master values ​​of each item obtained from multiple large-scale language models 300 and evaluating them. That is, for each model, an independent score is set for each item, such as location, price, structure, zoning, and facilities, and the reliability of the estimated master value (or the read value if the read value is used as is without estimating the master value; the same applies below) is calculated based on the score. Figure 4 shows an example of scores (basic scores) set for each model and item. For example, for the item: structure, a basic score of 0.7 is assigned to LLM1 and a basic score of 0.4 is assigned to LLM2. This means that in the preliminary study stage, the reliability of structure was higher for LLM1 than for LLM2.

[0058] The selection means 240 determines the selected master value for each item based on the score assigned by the evaluation means 230. That is, if the same master value is obtained from multiple models, the candidate with the highest combined score is selected, and if the same master value does not exist, the candidate with the highest single score is selected. The selection means 240 can generate discrepancy notification data for items where discrepancies have occurred along with the selection result and return it to the user terminal 100.

[0059] A specific example will be given below. (Example 1: Three-way agreement) Location is an item for which no candidate list is provided, and the read value returned by each model is evaluated as a candidate. If LLM1, LLM2, and LLM3 all return the same read value, "Aobadai, Aoba Ward, Yokohama City," the evaluation means 230 adds up the scores set for each model and item for that read value (LLM1: 0.7, LLM2: 0.4, LLM3: 0.4) to calculate a combined score of 1.5. The selection means 240 determines that read value as the selected master value for location. In this case, the read value may be determined as the selected master value based on the fact that the three values ​​match, without calculating a combined score.

[0060] (Example 2: LLM2 and LLM3 match, but only LLM1 differs) If LLM1 returns the reading "Aoyamadai, Aoba-ku, Yokohama," and LLM2 and LLM3 match and return the reading "Aobadai, Aoba-ku, Yokohama," the evaluation means 230 determines that the score of "Aoyamadai, Aoba-ku, Yokohama" (the value of LLM1) is 0.7, and the combined score of "Aobadai, Aoba-ku, Yokohama" (the matched values ​​of LLM2 and LLM3) is 0.4 + 0.4 = 0.8. The selection means 240 determines the latter reading "Aobadai, Aoba-ku, Yokohama" as the selected value for the location based on the fact that the combined score of 0.8 is greater than the score of 0.7.

[0061] (Example 3: When a master candidate list exists - structure) The structure is an item for which a master candidate list is defined in advance. For example, the master candidate list may include "wooden," "steel frame," "reinforced concrete," "SRC," and "other." In this case, if LLM1 extracts the reading value "two-story wooden building" and returns the master value "wooden" as a result of matching with the candidate list, a score of 0.7 is assigned. If LLM2 extracts the reading value "wooden building" and returns the master value "wooden", a score of 0.4 is assigned. If LLM3 extracts the reading value "steel frame" and returns the master value "steel frame", a score of 0.4 is assigned. The evaluation means 230 compares the total score 1.1 (0.7+0.4) of the master value "wooden" with the score 0.4 of the master value "steel-framed", and the selection means 240 determines "wooden" as the selected master value of the structure.

[0062] (Example 4: When a master candidate list exists - Land Use Zone) The use zone is an item that is pre-populated in the master candidate list and includes "Type 1 low-rise," "Type 2 low-rise," "Type 1 residential," "commercial area," "industrial area," "other," etc. For example, if LLM1 extracts the reading "Class 1 Low-Rise Residential Zone" and returns the master value "Class 1 Low-Rise" as a result of matching with the candidate list, a score of 0.7 will be assigned. If LLM2 similarly returns "Class 1 Low-Rise" as the master value "Class 1 Low-Rise", a score of 0.4 will be assigned. If LLM3 extracts the reading "Commercial Zone" and returns the master value "Commercial Zone", a score of 0.4 will be assigned. The evaluation means 230 compares the combined score 1.1 (0.7+0.4) of the master value "Class 1 low-rise" with the score 0.4 of the master value "commercial area", and the selection means 240 determines "Class 1 low-rise" as the selected master value for the use area.

[0063] The correction history storage means 250 has a function of storing the selection as a correction history when a candidate different from the master value determined by the selection means 240 is selected by the user in the user terminal 100. The correction history records at least (a) the date and time of the correction, (b) the item name, (c) the read value (before the correction), (d) the system-determined value, and (e) the user-selected value (after the correction) (see the example shown in FIG. 5(a)). Here, the objects to be recorded are limited to master values, and do not include freely input values ​​such as prices. A distinctive feature of this embodiment is that the correction history is managed using the read value (before correction) as a key. In other words, the system associates the read value with the correction content and stores it, in line with the operation in which the user performs correction operations based on the read value actually confirmed on the screen. If no correction is made, no history is registered, since there is no need to make corrections to values ​​that are correctly read.

[0064] The score update means 260 has a function of referring to the correction history stored by the correction history storage means 250 and updating the score used by the evaluation means 230. Specifically, for a certain read value X, the score update means 260 tally up the past correction history and calculates the history addition based on the corrected value (user-selected value). For example, as shown in Figure 5(a), suppose the system-determined value (the value estimated by LLM) for the read value "Type 1 Low-Rise Residential" is "Type 1 Residential" or "Type 1 Low-Rise," and the user selects "Type 1 Low-Rise" (correct correction) or "Type 1 Residential" (incorrect correction). In this case, as shown in Figure 5(b), for the read value "Type 1 Low-Rise Residential," a history addition of 0.4 is assigned to "Type 1 Low-Rise" which the user selected more often, and a history addition of 0.2 is assigned to "Type 1 Residential" which was selected only a few times. In this way, by using the read value as a key (i.e., by linking it to the read value) and referring to past correction history, it is possible to correct the tendency for the same misrecognition to be repeated in a learning manner, and encourage the adoption of the correct master value.

[0065] The score update means 260 tally up the correction history over a certain period (for example, the past 90 days) and add a value obtained by multiplying the number of times of adoption by a predetermined coefficient (for example, 0.2) as the history addition. As a result, for a certain reading value, a master value that has been repeatedly corrected by the user will have a higher score, while a value that has been incorrectly corrected will have a relatively lower reliability. In addition, the history addition can be attenuated over time, so that the influence of old history does not continue.

[0066] The evaluation means 230 assigns a base score set for each model and item to the candidate values ​​for each item obtained from multiple large-scale language models 300, and calculates a total score for each candidate by adding the history additions calculated by the score update means 260. If multiple models return the same master value, the base scores for that master value are added together, and the history additions for that master value are also added to calculate the total score. This makes it possible to ultimately adopt a stable master value based on past correction records, even if there are orthographic variations or incorrect mappings.

[0067] The above history addition is an example, and the coefficient (0.2), aggregation period (e.g., 90 days), presence or absence of time decay, and function form (stepwise decay, exponential decay, etc.) can be changed as appropriate depending on the operational conditions. Also, if multiple models return the same candidate value, adding the history addition to the combined basic score for that candidate value will enable the selection to reflect actual operational preferences.

[0068] In this method, the read value is used as a key, and the master value that the user finally selects is recorded as the "correct answer," and a history is added according to the number of times it is selected.

[0069] In the workplace, there are many variations in the way myths are written, but users often choose the correct expression in accordance with business rules and laws. Therefore, if a large number of myths are processed over a long period of time, the correct master value will be selected repeatedly and recorded in the history more often than the incorrect value. As a result, the history addition will naturally be given more to the correct master value, and misrecognition by each LLM can be eliminated on the cloud server 200 side.

[0070] In the short term, incorrect values ​​may prevail due to chance or temporary miscorrections. However, this method can limit the history weighting to a certain period of time (e.g., the past 90 days), set an upper limit on the history weighting, or decay the weighting over time. This prevents temporary errors from remaining for long periods of time, and ultimately the weighting will be concentrated on the correct master value that is repeatedly selected.

[0071] Furthermore, if multiple LLMs return the same value, the base scores are added together, so if a correct value is supported by multiple models, it will have an even greater advantage. This mechanism ensures that a statistically correct master value is consistently adopted over the long term.

[0072] The external library means 270 has a function of determining a master value for a given item using an external similarity calculation library instead of the master value estimated by the large-scale language model group 300. For example, for an item with many candidates, such as equipment or features, the "edit distance" with the read value is calculated, and the read value is matched to the item name with the closest "edit distance." This enables fast and stable judgment even for items with a huge number of candidates, such as equipment. The similarity calculation library for edit distance is an example, and other similarity libraries such as vectorization libraries may also be used.

[0073] The dictionary means 280 has a function of determining master values ​​for predetermined items (equipment, features, etc.) based on predetermined dictionary data. This makes it possible to absorb variations in spelling and input, and improve the consistency of registered data.

[0074] The registration / output unit 290 has a function of registering the master values ​​of each item determined by the selection means 240 in the property database 295 and returning display data to the user terminal 100. The display data includes the read values, the selected master values, the candidate values ​​estimated by each model, assigned score information, and discrepancy notifications. The registration / output unit 290 may also store the uploaded real estate documents in the document store 291 and use them to match candidate values ​​by referring to the master candidate store 292.

[0075] With reference to FIG. 6, the overall processing operation according to one embodiment of the present invention will be described. S1 (receive materials) The user terminal 100 transmits real estate documents (PDF or image files) from the input unit 101 to the cloud server 200. The cloud server 200 receives the documents and stores them in the document store 291 as needed. This ensures that the documents can be referenced in subsequent processing.

[0076] S2 (single prompt processing) The cloud server 200 issues a single prompt to multiple large-scale language models 300, inputs the real estate documents in a format that the models can accept (e.g., PDF or page images), executes the following processes in a batch, and receives the results (hereinafter referred to as S2 results). (i) Extraction of readings for specified items (location, price, structure, zoning, facilities, etc.) from the relevant real estate documents; (ii) Selection (estimation) of master values ​​based on matching with a master candidate list defined for each item (excluding items for which no candidates are provided), (iii) Return in a specified format (e.g., JSON) including the read value and master value (or only the read value depending on the item).

[0077] The prompts used in this case may include at least the following points: (a) Strictly observe output standards: Returns must be in the specified format only, and no explanatory or supplementary text must be added. (b) Candidate Matching Rules: Select a master value by normalizing to a term in the master candidate list. If no candidate applies, return "unknown" or "other." If multiple matches are equally likely, indicate this or select the best match. (c) Items without candidates: For items without candidates, such as location and price, only the read value should be returned (master values ​​should not be output). For location, prefecture, city / ward / town / village name, street address, and building name should each be returned as separate items, and only the read value should be returned. For prices, etc., a numerical value (in 10,000 yen) should be returned. (d) Countermeasures against variations in notation: Read values ​​should be faithful to the original description, and master values ​​should be unified to the standard notation of the candidate (e.g., "Type 1 low-rise residential area" → master value "Type 1 low-rise"). (e) Prohibition of excessive speculation: Do not create content that is not based on the documents, and mark illegible or unspecified information as "unknown." Do not use external knowledge or searches. (f) Exception handling: Strings outside the master candidate list are not returned as master values ​​(strings outside the master candidate list are considered invalid as master values ​​and are retained as read values).

[0078] The S2 results are used for scoring in the subsequent evaluation means 230, so each item is formatted as a pair (read value, master value (if applicable)). Items that do not have candidates (location, price, etc.) include only read values, while items that have candidates (structure, zoning, etc.) include read values ​​and master values ​​normalized to the candidate list.

[0079] S3 (Handling of items with no candidates) For items such as location and price that do not have a pre-defined master candidate list, the read values ​​returned from each large-scale language model 300 in S2 are treated as candidate values. In other words, for these items, the master values ​​are not normalized, and the read values ​​are input directly to the subsequent evaluation means 230, where they are subjected to the assignment of a basic score and the selection process.

[0080] In this case, the prefecture, city, town, street address, and building name are stored as read values ​​for location, and the number (in 10,000 yen) is stored as read value for price. These candidate values ​​are used as match values ​​if multiple models return the same value, and are treated as difference candidates if they return different values.

[0081] In this way, for items that do not have a candidate list, by directly comparing and evaluating the read values, highly accurate selection processing can be achieved even for information that does not require normalization.

[0082] S4 (branch of master value determination process) Based on the result obtained in S2, the cloud server 200 branches the process as follows depending on the nature of the item. 1. Items with no candidates (e.g. location, price, etc.) Since there is no master candidate list, the readings returned from the LLM are used as the evaluation target. For example, for location, the read values ​​of prefecture, city, town, street address, and building name are treated as candidates, and for price, the read value is treated as a number (10,000 yen). In this case, there is no master value and the candidate value in the evaluation means 230 is the "read value." 2. Items with candidates (e.g., structure, zoning, etc.) Since a master candidate list is defined, the estimated master value returned by the LLM after matching is used as the evaluation target. For example, if the read value "Type 1 low-rise residential area" returns a master value of "Type 1 low-rise," then this "Type 1 low-rise" will be the evaluation target. In this case, the master value is always present and the readings remain as references. 3. Special items (e.g., facilities, features, etc.) For items with an extremely large number of candidates and frequent variations in spelling or unknown expressions, the estimated master value of LLM alone may not be accurate enough. For this reason, the cloud server 200 can use the following auxiliary means.

[0083] (a) External library means 270: A read value is input, the similarity with a set of candidates is calculated, and the candidate with the highest similarity is determined as the master value. For example, in the equipment section, If the read value is "separate bathroom and toilet," and if "separate bathroom and toilet" is included in the candidate set, the similarity between the two is determined to be high, and the master value is unified to "separate bathroom and toilet." Also, if the reading is "reheat bath," if the candidate set includes "reheat function," it will be unified with this. The following processing can be performed.

[0084] (b) Dictionary means 280: Based on a predefined dictionary, the readings are normalized to a standard representation to determine a master value. For example, in the equipment section, - "Auto Lock" and "Automatic Lock" will be unified to "Auto Lock" - Expand the summary notation "water supply and sewerage" into two facility items, "water supply" and "sewerage," based on a dictionary. · "WIC" corresponds to "walk-in closet"; The following processing can be performed.

[0085] Either the external library means 270 or the dictionary means 280 can be selected and used depending on the item.

[0086] S5 (rating) The evaluation means 230 evaluates the candidate values ​​obtained in S3 and S4 (read values ​​for items without candidates, estimated master values ​​returned by the LLM for items with candidates, and master values ​​determined by the external library means 270 or dictionary means 280 for special items) in the following procedure. 1. Basic score assignment A base score preset for each large-scale language model 300 and for each item is assigned to the value returned by that model. For example, for the structural item, a base score of 0.7 is set for LLM1, 0.4 for LLM2, and 0.4 for LLM3. 2. Adding up the matching values If multiple models return the same master value (or reading), their base scores are summed, which reflects the confidence of the multiple model matches. 3. History addition The score update means 260 tally the most recent history recorded in the correction history storage means 250, and calculates a history addition for each value according to the number of times it has been adopted over a certain period (e.g., the past 90 days) using the read value as a key. The evaluation means 230 adds this history addition to each candidate value. The history addition may be given by a coefficient of, for example, the number of times of selection x 0.2, so that old history is attenuated and its influence is reduced. 4. Calculation of total score A total score is calculated for each candidate by combining the basic score and the history addition. This total score becomes the basis for determining the selected master value for each item in the subsequent selection means 240.

[0087] In this way, by combining the "basic reliability of each model" with "historical addition based on operational performance," S5 realizes a system in which values ​​that have repeatedly been found to be correct in practice are stably and highly evaluated, even if there are variations in notation or incorrect mapping.

[0088] S6 (adopted) The selection means 240 determines the selected master value for each item based on the total score of each candidate value calculated in S5. 1. When the same candidate is returned from multiple models The basic scores of the candidate are added together, so if the combined score plus the historical weighting exceeds the other candidates, the candidate is selected. 2. If there is no identical candidate The overall scores of each candidate are compared and the candidate with the highest score is selected. 3. In the case of an antagonistic state If multiple candidates are close with almost the same score, the system will decide on the provisionally adopted value as is, but will treat it as a target for discrepancy notification in the subsequent discrepancy determination (S7).

[0089] In this way, S6 uses a total score that combines a basic score set for each LLM with a historical addition based on actual operation, enabling highly reliable selection that reflects the consistency of multiple models and the user's track record.

[0090] S7 (Difference judgment) The cloud server 200 determines whether or not there is a difference between the adopted master value determined in S6 and other candidates. 1. If there is a match If all large-scale language models 300 return the same value and there are no candidates other than the adopted master value, it is determined that no discrepancy has occurred. In this case, the process proceeds to S10 (registration process) without receiving a discrepancy notification. 2. In case of mismatch If there is a difference between the candidates returned for each large-scale language model 300 and a candidate other than the adopted master value exists, it is determined that a difference has occurred. In this case, the process proceeds to S8 (notification of difference). 3. When the score is tied Even if the total scores of multiple candidates are close and there is no clear superiority or inferiority as to which value should be adopted, it is determined that a difference exists.

[0091] In this way, S7 not only determines the adopted master value, but also detects differences in the estimation results between models and rivalry between candidates, and can determine whether user confirmation is required.

[0092] S8 (difference notification) If it is determined in S7 that there is a difference, the cloud server 200 generates difference notification data indicating the content of the difference and returns it to the display unit 102 of the user terminal 100. 1.Notification content The discrepancy notification data may include at least the following: (a) The name of the item where the discrepancy occurs (e.g., zoning, structure, etc.), (b) the candidate master values ​​returned by each large-scale language model 300; (c) score information assigned by the evaluation means 230 (optional); (d) The currently selected master value determined by the selection means 240. 2. Presentation on the user interface The display unit 102 highlights the items with differences (by bolding or highlighting, etc.) and displays each candidate value alongside them. The candidate values ​​are presented in the form of radio buttons, a pull-down list, or a selection list, allowing the user to select a candidate that is different from the adopted value. 3. Provide additional information (optional) If necessary, supplemental information such as the history of each candidate and the frequency of past corrections may be displayed, which makes it easier for the user to determine the validity of the candidate.

[0093] In this way, in S8, for items where there is a discrepancy between models or where scores are at odds, a discrepancy notification is sent so that the user can check and correct the items as necessary.

[0094] S9 (User selection acceptance and correction saving) When the user selects a candidate that differs from the adopted master value from among the different candidates presented in S8 on the display unit 102 of the user terminal 100, the selection result is transmitted to the cloud server 200. 1.Selective reception The user can select a desired candidate from the options on the display unit 102. The selection format can be configured as desired, such as radio buttons, a pull-down list, or text correction input. 2. Saving revision history The correction history storage means 250 stores the selected content as a correction history. The stored information can include at least the following: (a) Date and time of correction (b) Item name (c) Readings (before correction) (d) System-Determined Value (e) User-selected value (corrected) 3. Limitation of scope The objects to be recorded here are master values, and freely input values ​​such as prices may be excluded from the correction history.

[0095] In this way, in S9, the content corrected by the user is saved as correct answer data and can be used for subsequent learning by the score update means 260, thereby improving accuracy through operation.

[0096] S10 (Registered) The registration / output unit 290 registers the adopted master value determined in S6 or the value corrected by the user in S9 in the property database 295 as the final registered value. 1. Registration process The determined value for each item (the selected master value or the corrected value) is stored in a predetermined field in the property database 295. This allows the data to be stored in a format that can be used in an in-house property management system or on a website for external disclosure. 2.Return of display data The registration / output unit 290 returns the registration result to the user terminal 100. This display data includes at least the following: (a) Readings for each item (b) Confirmed adopted master value 3.Conservation treatment If desired, the original uploaded property documents and the finalized registration data may be associated and stored for future audits or reprocessing.

[0097] In this way, in S10, the information extracted and judged by the system is registered in the official property database and is also fed back to the user's terminal, allowing them to check the registration results or use them for the next task.

[0098] S11 (score update) The score update means 260 updates the parameters of the score given by the evaluation means 230 based on the correction history stored in S9. 1. Target of data collection The correction history saved over a recent period (for example, the past 90 days) is targeted, and the number of times each item and value was ultimately adopted by the user is tallied using the read value as the key. 2. Calculation of historical addition Based on the aggregation results using the reading value as a key, the history addition is calculated using, for example, "number of times selected x coefficient (e.g., 0.2)". The more times a value is selected, the larger the history addition is given, and the weight in the total score increases. 3. Adjusting the impact It is possible to set an upper limit on the history addition, or to decay it over time, which prevents temporary erroneous corrections or old history from having an excessive effect. 4. Applying the updated results The calculated history addition is added to the basic score in the next evaluation process (S5) and thereafter, and is reflected in the total score of each candidate value.

[0099] In this way, in S11, by sequentially adding history that reflects the user's correction results, the correct master value is more likely to take precedence throughout operation, and the accuracy of the entire system is continuously improved.

[0100] S12 (End) In the cloud server 200, the registration / output unit 290 registers the final value in the property database 295 in S10, and the score update means 260 updates the history addition in S11, after which the process ends. 1. End of single process The entire process of extracting, evaluating, selecting, registering, and updating scores for one real estate document is completed here. 2.Transition to next treatment If multiple real estate documents are to be processed consecutively, the process returns to S1 to receive the next document. 3. Save when finished (optional) If necessary, processing logs and statistical information on adoption results can be saved and used for operational audits and future analysis.

[0101] In this way, in S12, the end of processing a single material or the transition to the next processing is clearly defined, and the processing cycle of the entire system is completed.

[0102] (S2 variation: two-stage prompt format) In this embodiment, the extraction of the readings and the estimation of the master values ​​are performed in one step using a single prompt in S2, but as a modification, the process may be divided and performed using a two-step prompt. 1. First prompt (extract reading) The cloud server 200 inputs real estate documents into each large-scale language model 300 and issues instructions to read and return the character strings in the documents for each item such as "location," "price," "structure," "zoning," and "facilities." The output of this prompt is a "readout" that is faithful to the original text. 2. Second prompt (Master value estimation) Next, cloud server 200 inputs the read value obtained in the first prompt, and also presents a master candidate list corresponding to each item, and instructs the user to "match the read value with the candidate list and return the closest master value." As a result, for example, if the read value "Class 1 low-rise exclusive residential area" is input, the master value "Class 1 low-rise" is returned. 3. Items with no candidates For items that do not have a candidate list, such as location or price, a second prompt is not necessary, and the reading extracted in the first prompt is used as the candidate.

[0103] This two-stage prompt format has the advantage that the reading process and the master value estimation process can be clearly separated, making it easier to analyze the cause of errors and optimize the prompts independently.However, it may result in higher processing costs and response times than the single-prompt format.

[0104] (Variation: A form in which the read value is omitted and only the master value is estimated) In the above-described embodiment, the present invention has been described with reference to an example in which a master value is estimated after a read value is acquired, but the present invention is not limited to this. As a modified example, a configuration in which only a master value is estimated without acquiring a read value may be adopted.

[0105] In this configuration, the cloud server 200 receives real estate documents (PDF or image) as input and causes multiple large-scale language models 300 to return only classification (estimation) results based on a master candidate list for each specified item. That is, in a prompt, (i) the contents of the entire document or each page are input, (ii) a master candidate list for each item (for items for which candidates are provided) is presented, and (iii) only the most suitable master value is returned in a specified format. For items for which candidates are provided (e.g., structure, zoning, etc.), the estimated master value is input directly to the evaluation means 230.

[0106] On the other hand, for items that do not have candidates (e.g., location, price, etc.), omitting the read value may result in the master value of the item not existing. In such cases, the missing information may be supplemented by one of the following methods. (a) User input: An input field for the item is presented on the display unit 102, and the user manually inputs the information. (b) Estimation by other means: The value of the item is estimated or normalized by auxiliary means such as the external library means 270 or dictionary means 280, or by external database reference. (c) Unknown: If no basis can be obtained, mark the item as "unknown" and supplement it in the subsequent registration process or operation flow.

[0107] In this modification, the evaluation means 230 also treats estimated master values ​​for items with candidates, and values ​​of the items (complemented as necessary) for items without candidates as candidates, and calculates a total score by assigning basic scores for each model and item, adding up matching values, and adding up the history calculated in S11. The selection means 240 determines the selection value for each item based on the total score.

[0108] In this modified example, the reading instruction means 210 and the master value estimation means 220 may be implemented in a form that is promptly designed to omit returning the read value and return only the master value. That is, the read value may be treated as "not returned" or "unknown", and subsequent processing (evaluation, adoption, discrepancy notification, history storage, score update) may be performed only on the master value.

[0109] In this way, even in a form in which the read values ​​are omitted and only the master value is estimated, the real estate information extraction system of the present invention can obtain stable normalization results (estimated master values) for items for which candidates are provided, and for items for which candidates are not provided, by using user input or completion by other means in combination, it is possible to complete processing while satisfying practical operational requirements.

[0110] (Further benefits from using multiple LLMs) In an embodiment of the present invention, any items that differ between models are notified of the differences and clearly identified (for example, grayed out or highlighted) on the user terminal, allowing the user to see at a glance which items need to be checked. This eliminates the need to check every item one by one, significantly reducing the time and psychological burden of the checking work.

[0111] Fig. 7(a) shows a conventional example of displaying the results recognized by a single LLM, and Fig. 7(b) shows a display example using a multiple LLM configuration according to the present invention. In Fig. 7(a), the accuracy of each item cannot be determined in advance, and all items must be checked equally, whereas in Fig. 7(b), items that differ between models are visually distinguished, allowing the user to focus on checking those items (of course, this does not mean that other items do not need to be checked).

[0112] In this way, the present invention not only improves accuracy but also brings substantial improvements in both work efficiency and usability through the "visualization of discrepancies" achieved by using multiple LLMs in combination. This reduces the burden of confirmation during registration work, shortens the work time of on-site personnel, and prevents variations in input accuracy.

[0113] (Variation: Score update method using correction history) The score update means 260 of the present invention is not limited to the above embodiment, and can be implemented by changing the method of using the correction history and the unit of score update in various ways. Examples are shown below.

[0114] (1) Score addition method (current implementation) This method uses the read value as a key, refers to the corresponding correction history, and calculates an additional value according to the number of times the master value selected by the user for that read value has been adopted. In this method, by repeatedly adding history to the basic score, the score of the repeatedly corrected master value increases, and the score of values ​​that are easily misrecognized decreases relatively.

[0115] (2) Score correction method (addition / subtraction balance adjustment) This method does not simply add up the correction history, but also uses the read value as a key to use as a correction coefficient to subtract scores from master values ​​or models that have been corrected many times. This makes it possible to statistically suppress the tendency for erroneous determination of a particular model or master value. It should be noted that addition includes the addition of negative values, and therefore history addition also includes this subtraction. The score is updated by adding and subtracting.

[0116] (3) Basic score correction method (long-term adjustment) This method involves compiling correction history over a certain period of time and updating the basic score for each item or model. For example, if a certain model has high accuracy in the zoning category, the basic score of that model is increased, and conversely, the score of a model with a high correction rate is decreased. With this method, the basic score is automatically adjusted throughout operation, allowing for stable selection accuracy to be maintained over the long term.

[0117] (4) Weight learning method (integrated score optimization) This method integrates multiple model outputs as features and uses correction history as training data to calculate the optimal weight vector for each model and item. In this method, the scoring itself is achieved by a linear combination or machine learning model, and is automatically optimized through the correction history.

[0118] (5) In-session correction method (real-time update) If a correction is made within the same session or within the same document, the system automatically detects the readings with the same notation and temporarily increases the score for the same session only. This allows the user to minimize operations when variations in spelling occur repeatedly within the same document.

[0119] In this way, the score update means 260 is not limited to a method of directly reflecting the correction history in the score, but may be configured to dynamically optimize the score through modification of the basic score, correction by model, weight vector learning, etc. All of these variations have in common the fact that they use the correction history to update the score system of the evaluation means 230.

[0120] (Variation: Configuration in which correction history is reflected in prompts of a large-scale language model) The present invention is not limited to the above embodiment, and the correction history by the user may not only be used to update the score on the server side, but also may be configured to reflect the correction history in the master value estimation process of the large-scale language model (LLM) itself.

[0121] In this configuration, the cloud server 200 extracts representative correction patterns that have occurred within a predetermined period (e.g., the past 90 days) from the correction history accumulated by the correction history storage means 250, and provides the correction patterns to the large-scale language model group 300 as prompt information. The correction pattern is defined as a data structure that associates, for example, (a) the read value, (b) the system-determined value, and (c) the user-selected value (after correction). The cloud server 200 converts these into a natural language format and incorporates it into the beginning of the input prompt for each LLM. The correction pattern is extracted due to limitations on the number of input tokens and analysis accuracy, so depending on the number of input tokens, all of the correction history may be used.

[0122] An example of a prompt instruction is shown below. <Example prompt> You are an AI that extracts real estate information. Below are some of the past corrections. Please refer to these and unify the same expressions or typos to the correct ones. "Type 1 low-rise residential area" → "Type 1 low-rise" ·"RC" → "RC construction" "Separate bathroom and toilet" → "Separate bath and toilet" Taking these factors into consideration, please estimate the master values ​​for each item from the real estate documents shown below. Real Estate Information.pdf

[0123] In this way, the correction history reflecting means dynamically generates prompt templates in the prompt generation processing unit based on the history information stored in the correction history storage means, and transmits the generated prompt templates to each large-scale language model. Each model can refer to the presented correction history, automatically correct spelling variations and abbreviations, and estimate master values ​​based on the candidate list.

[0124] With this configuration, correction trends are reflected through natural language inference within the model without the need to explicitly add or subtract scores on the server side, thereby improving the accuracy of correcting typos and standardizing notation and preventing users from repeatedly making the same correction operations. The cloud server may also be configured to manage correction histories on a per-model basis and learn correction tendencies specific to each model. In this case, an embedding vector of the correction history is generated for each model, and the history with the highest similarity is preferentially selected when generating a prompt.

[0125] Furthermore, it is possible to embed and vectorize the correction history and store it in a database, and add an API that references the "correction example database" in prompts. With this configuration, each LLM can search the database when a prompt is executed, and automatically obtain and reference correction patterns for similar errors.

[0126] This configuration can be used in conjunction with existing score addition methods and basic score correction methods. In other words, by using statistical information on correction history to update scores on the server side and incorporating representative correction patterns into prompts, a two-tiered self-learning system can be realized that combines server-side correction and in-model correction.

[0127] In this way, the correction history reflecting means can continuously improve the accuracy of master value estimation of the entire system by directly reflecting past correction trends in the large-scale language model in a learning manner. This configuration can be realized by using the prompt design function in the current large-scale language model API and can be implemented by auxiliary logic on the server side.

[0128] This configuration is not limited to the case where multiple large-scale language models are used in combination, but may also be applied to a single large-scale language model (at least one large-scale language model). In a single-model configuration, by providing the correction history to the model as prompt information, the model itself can estimate the master value while referring to past correction trends, which similarly improves correction accuracy and output stability.

[0129] (Variation: Extracting property information from a web page) The present invention is not limited to the real estate documents (PDF or images) in the above-described embodiment, but can also be applied to property information posted on a web page. That is, the cloud server 200 may be configured to access a URL specified by the user terminal 100, acquire the web page as a document, and provide it for subsequent processing (S2 and thereafter).

[0130] Since current large-scale language models (LLMs) are not good at directly analyzing HTML structures and scripts, it is desirable for the cloud server 200 to convert the acquired web page into a PDF or image and then treat it as "real estate information." This allows the layout and text information of the entire page to be input stably, and the previously described single prompt processing (S2) and two-stage prompt format can be applied as is.

[0131] In the future, if the LLM is able to directly understand HTML structures, the cloud server 200 may be configured to analyze the contents of a web page directly from a URL and extract property information. In this case, the prompt can instruct the user to "extract property details from the page of the specified URL, compare each item (location, price, structure, zoning, facilities, etc.) with the candidate list, and return the master value," thereby achieving processing equivalent to that of the conventional method using PDF documents as input.

[0132] (Example prompt: current) You are an AI that extracts real estate information. Please convert the property page listed at the URL below into a PDF or image, analyze it, and extract information such as location, price, structure, zoning, and facilities. For each item, standardize the notation (master value) based on the candidate list or dictionary and return it in the following format. Output format: JSON - Items not listed should be marked as "Unknown." If there are multiple properties, the highest-ranking property should be used.

[0133] (Example prompt: future) You are an AI that extracts real estate information. Analyze the property page listed at the URL below and extract information such as location, price, structure, zoning, and facilities. For each item, standardize the notation (master value) based on the candidate list or dictionary and return it in the following format. Output format: JSON - Items not listed should be marked as "Unknown." If there are multiple properties, the highest-ranking property should be used.

[0134] According to this modification, the cloud server 200 can automatically acquire and convert property information on the web and integrate it into the existing extraction, evaluation, and selection process flow. Therefore, similar effects to those of the present invention can be obtained for property information on online portal sites as well as paper media and PDFs, such as collaborative analysis of multiple LLMs, discrepancy notification, and reflection of correction history.

[0135] (Variation: Reading MySock images) The present invention can be applied to acquiring not only text information posted on MySoku but also image information. That is, the cloud server 200 may be configured to extract image data (exterior photos, floor plans, map images, etc.) included in real estate documents and associate and register the images in the property database 295.

[0136] The image extraction process can be achieved using known means, such as extracting images embedded in a PDF, automatically detecting an image area using an image recognition library, or allowing the user to specify and register an area.

[0137] (Variation: Generalized configuration as a document information extraction system) All of the above-described embodiments and modifications are not limited to real estate documents, but can be applied to any document data or image data that includes item information.

[0138] Although the present invention has been described in detail above, the above description is merely illustrative of the present invention in all respects and is not intended to limit its scope. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. Each of the constituent elements of the invention disclosed in this specification is considered to be an independent, stand-alone invention. Inventions that combine the constituent elements in any manner are also included in the present invention. The specific expressions in this specification are merely examples, and the present invention also includes those that conceptualize these exemplary expressions. [Industrial Applicability]

[0139] The real estate information extraction system, real estate information extraction method, and real estate information extraction program of the present invention enable automatic and stable extraction of property information such as location, price, structure, zoning, and facilities listed in real estate documents (so-called MySoku). Therefore, they significantly improve the efficiency of property information registration work in real estate brokerage business, reduce the burden of manual work, and contribute to preventing input errors.

[0140] Furthermore, the technical concept of this invention is not limited to the real estate field. Because it is equipped with a mechanism for comparing and evaluating inference results from multiple large-scale language models and dynamically adjusting them based on history, it can be applied to a wide range of fields that require item extraction from non-standard documents, such as financial product information, medical records, research papers, and contracts.

[0141] Therefore, the present invention has industrial applicability in that it significantly contributes to improving the reliability and efficiency of automatic data extraction and registration work in various information processing fields, including the real estate industry. [Explanation of symbols]

[0142] 100 user terminals 101 Input section 102 Display section 200 cloud servers 210 Reading instruction means 220 Master value estimation means 230 Evaluation Methods 240 Adoption measures 250 Correction history storage means 260 Score Update Method 270 External Library Means 280 Dictionary Means 290 Registration / Output Section 291 Document Store 292 Master Candidate Stores 295 property database 300 Large-scale Language Models (LLM1, LLM2, LLM3)

Claims

1. A real estate information extraction system that extracts property information from real estate documents, a reading instruction means for receiving the real estate document as an input, instructing a plurality of large-scale language models to recognize character strings written in the real estate document for each of a plurality of items, and obtaining a reading value for each of the items; a master value estimation means for providing a master candidate list corresponding to each item to the plurality of large-scale language models, estimating a master value corresponding to each item based on the read value, and acquiring the estimation result; evaluation means for evaluating the master values ​​for each item obtained from the plurality of large-scale language models; a selection means for determining a master value to be selected for each item based on the evaluation by the evaluation means, the evaluation means assigns a base score to each large-scale language model and to each item in accordance with a master value estimated by each large-scale language model, and evaluates the master value based on the base score; When the same master value is estimated from a plurality of large-scale language models, the selection means adds up the basic scores assigned to the master values ​​and selects the master value with the largest total score; A real estate information extraction system characterized in that, if the same master value is not obtained, the basic scores assigned to each master value are compared and the master value with the largest basic score is adopted.

2. 2. The real estate information extraction system according to claim 1, further comprising a score update unit that updates the basic score assigned by the evaluation unit based on a correction history made by the user.

3. the correction history is a history of user adoption and correction of the master value estimated by the master value estimation means, 3. The real estate information extraction system according to claim 2, wherein the score update means updates the score based on the number of times the master value in the correction history has been adopted.

4. 3. The real estate information extraction system according to claim 2, wherein said score update means reduces the degree of influence in accordance with the time that has elapsed since the correction history occurred.

5. 2. The real estate information extraction system according to claim 1, further comprising an external library means for inputting read values ​​into an external similarity calculation library to determine master values ​​for predetermined items.

6. 2. The real estate information extraction system according to claim 1, further comprising dictionary means for determining a master value for a predetermined item based on a dictionary of read values.

7. 7. The real estate information extraction system according to claim 5, wherein the predetermined items are related to facilities or features.

8. The real estate information extraction system of claim 1, further comprising a difference notification means for, when the master values ​​estimated by multiple large-scale language models differ, sending a difference notification indicating the occurrence of such a difference to a user terminal and displaying the estimated master values ​​of each of the large-scale language models side by side on the user terminal.

9. The real estate information extraction system described in claim 8, further comprising a correction history storage means for storing the user's selection results (adoption or correction) as history regarding the master values ​​written on the user terminal by the difference notification means.

10. The real estate information extraction system according to claim 9, wherein the evaluation means evaluates the master values ​​estimated by each large-scale language model using the correction history stored by the correction history storage means.

11. A real estate information extraction method executed by an information processing device that extracts property information from real estate documents, comprising: receiving a real estate document as an input, instructing a plurality of large-scale language models to recognize strings of characters in the real estate document for each of a plurality of fields, and obtaining a reading for each of the fields; a step of providing a master candidate list corresponding to each item to the plurality of large-scale language models, estimating a master value corresponding to each item based on the read value, and obtaining the estimation result; evaluating the master value for each term obtained from the plurality of large-scale language models; and determining a master value to be adopted for each item based on the evaluation, the evaluating step includes assigning a base score to each large-scale language model and each item in accordance with a master value estimated by each large-scale language model, and evaluating the master value based on the base score; In the determining step, when the same master value is estimated from a plurality of large-scale language models, the basic scores assigned to the master values ​​are summed up, and the master value with the largest total score is adopted; A real estate information extraction method executed by the information processing device, characterized in that if the same master value is not obtained, the basic scores assigned to each master value are compared and the master value with the largest basic score is adopted.

12. On the computer, receiving a real estate document as an input, instructing a plurality of large-scale language models to recognize strings of characters in the real estate document for each of a plurality of fields, and obtaining a reading for each of the fields; a step of providing a master candidate list corresponding to each item to the plurality of large-scale language models, estimating a master value corresponding to each item based on the read value, and obtaining the estimation result; evaluating the master value for each term obtained from the plurality of large-scale language models; a step of determining a master value to be adopted for each item based on the evaluation; Execute In the evaluating step, a base score is assigned to each large-scale language model and each item in accordance with a master value estimated by each large-scale language model, and the master value is evaluated based on the base score; In the determining step, when the same master value is estimated from a plurality of large-scale language models, the basic scores assigned to the master values ​​are summed up, and the master value with the largest total score is adopted; If the same master value is not obtained, the program causes the computer to compare the basic scores assigned to each master value and adopt the master value with the largest basic score.

13. A real estate information extraction system that extracts property information from real estate documents, a master value estimation means for receiving the real estate document as an input, presenting a master candidate list to a plurality of large-scale language models based on the content of the real estate document for each of a plurality of items, estimating a master value corresponding to each of the items, and acquiring the estimation results; evaluation means for evaluating master values ​​of each item obtained from the plurality of large-scale language models; and an selecting means for determining a master value to be selected for each item based on the evaluation, the evaluation means assigns a base score to each large-scale language model and to each item in accordance with a master value estimated by each large-scale language model, and evaluates the master value based on the base score; When the same master value is estimated from a plurality of large-scale language models, the selection means adds up the basic scores assigned to the master values ​​and selects the master value with the largest total score; If the same master value is not obtained, the system compares the basic scores assigned to each master value and adopts the master value with the highest basic score.

14. A real estate information extraction method executed by an information processing device that extracts property information from real estate documents, comprising: a step of inputting the real estate document, presenting a master candidate list to a plurality of large-scale language models based on the content of the real estate document for each of a plurality of items, estimating a master value corresponding to each of the items, and obtaining the estimation results; evaluating a master value for each item obtained from the plurality of large-scale language models; and determining a master value to be adopted for each item based on the evaluation, the evaluating step includes assigning a base score to each large-scale language model and each item in accordance with a master value estimated by each large-scale language model, and evaluating the master value based on the base score; In the determining step, when the same master value is estimated from a plurality of large-scale language models, the basic scores assigned to the master values ​​are summed up, and the master value with the largest total score is adopted; A real estate information extraction method executed by the information processing device, characterized in that if the same master value is not obtained, the basic scores assigned to each master value are compared and the master value with the largest basic score is adopted.

15. On the computer, a step of inputting real estate documents, for each of a plurality of items, presenting a master candidate list to a plurality of large-scale language models based on the contents of the real estate documents, estimating master values ​​corresponding to each of the items, and acquiring the estimation results; evaluating a master value for each item obtained from the plurality of large-scale language models; a step of determining a master value to be adopted for each item based on the evaluation; Execute In the evaluating step, a base score is assigned to each large-scale language model and each item in accordance with a master value estimated by each large-scale language model, and the master value is evaluated based on the base score; In the determining step, when the same master value is estimated from a plurality of large-scale language models, the basic scores assigned to the master values ​​are summed up, and the master value with the largest total score is adopted; If the same master value is not obtained, the program causes the computer to compare the basic scores assigned to each master value and adopt the master value with the largest basic score.

16. A real estate information extraction system that extracts property information from real estate documents, a master value estimation means for inputting the real estate document, providing a master candidate list corresponding to each item to a plurality of large-scale language models, estimating a master value corresponding to each item, and acquiring the estimation results; evaluation means for evaluating the master values ​​obtained from the plurality of large-scale language models; a selection means for determining a master value to be selected for each item based on the evaluation by the evaluation means; a correction history reflecting means configured to retain a correction history made by a user, provide the correction history as prompt information to the plurality of large-scale language models, and cause the plurality of large-scale language models to estimate the master value based on the correction history; Equipped with the evaluation means assigns a base score to each large-scale language model and to each item in accordance with a master value estimated by each large-scale language model, and evaluates the master value based on the base score; When the same master value is estimated from a plurality of large-scale language models, the selection means adds up the basic scores assigned to the master values ​​and selects the master value with the largest total score; If the same master value is not obtained, the system compares the basic scores assigned to each master value and adopts the master value with the highest basic score.

Images