Structured processing apparatus, structured processing method, and program

Abstract

This makes it easier to structure text while taking into account named entities found within the text. [Solution] The system includes: an acquisition unit that acquires text information of a target text to be structured; a source text notation processing unit that performs source text notation processing to match the notation of structured elements, which are elements included in first structured data structured by applying a structured processing model to the target text, with the notation of the target text corresponding to the structured elements; and an output unit that outputs second structured data in which the source text notation processing has been performed on the notation of the structured elements by the source text notation processing unit.

Description

Technical Field

[0001] The present invention relates to a structured processing device, a structured processing method, and a program.

Background Art

[0002] In recent years, technologies for realizing various financial transactions using computer systems have been developed. For example, international money transfers are made through an international network via SWIFT (Society for Worldwide Interbank Financial Telecommunication) (registered trademark), and funds are settled through settlement standards operated by central banks of each country. In SWIFT (registered trademark), it is required to describe addresses and the like in a format conforming to ISO. As a technology for checking or converting address data, for example, Patent Document 1 discloses a technology capable of checking address data according to various usage purposes.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, not only address notations but also text sentences may contain many unique expressions. Structuring such a text sentence into a prescribed format while considering the unique expressions described in the text sentence requires a great deal of labor. For example, it is conceivable to map and structure them one by one manually, but it is not realistic to manually convert a huge number of text sentences. Also, although it is possible to perform structuring based on rules, although a certain level of accuracy can be expected, there are problems with the variations of the rules and the amount of information in the dictionary in order to comprehensively handle the unique expressions of each of many countries.

[0005] In view of the above-mentioned problems, the present invention aims to provide a structuring processing device, a structuring processing method, and a program that can facilitate structuring while taking into account named entities described in text. [Means for solving the problem]

[0006] A structured processing device according to one aspect of the present invention includes: an acquisition unit that acquires text information of a target text to be structured; a source text notation processing unit that performs source text notation processing to match the notation of structured elements, which are elements included in first structured data structured by applying a structured processing model to the target text, with the notation of the target text corresponding to the structured elements; and an output unit that outputs second structured data on which the source text notation processing has been performed on the notation of the structured elements by the source text notation processing unit.

[0007] A structuring processing method according to one aspect of the present invention is a structuring processing method performed by a computer-based structuring processing device, wherein an acquisition unit acquires text information of a target text to be structured, a source text notation processing unit performs source text notation processing to match the notation of structured elements, which are elements included in first structured data structured by applying a structuring processing model to the target text, with the notation of the target text corresponding to the structured elements, and an output unit outputs second structured data in which the source text notation processing has been performed on the notation of the structured elements by the source text notation processing device.

[0008] A program according to one aspect of the present invention is a program that causes a computer, which is a structured processing device, to acquire text information of a target text to be structured, to perform a source text notation process to match the notation of structured elements, which are elements included in first structured data structured by applying a structured processing model to the target text, with the notation of the target text corresponding to the structured elements, and to output second structured data on which the source text notation process has been performed on the notation of the structured elements. [Effects of the Invention]

[0009] According to the present invention, it is possible to easily structure text while taking into account named entities described in the text. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing the configuration of the structured processing system 1 according to an embodiment. [Figure 2] This figure shows an example of information stored in the target text data storage unit 104 according to the embodiment. [Figure 3] This figure shows an example of information stored in the first structured data storage unit 105 according to the embodiment. [Figure 4] This figure shows an example of information stored in the second structured data storage unit 106 according to the embodiment. [Figure 5] This is a diagram illustrating the structuring process performed by the structuring processing apparatus 10 according to the embodiment. [Figure 6] This is a diagram illustrating the source text notation processing performed by the structured processing apparatus 10 according to the embodiment. [Figure 7] This is a flowchart showing the processing flow performed by the structured processing apparatus 10 according to the embodiment. [Figure 8] This is a flowchart showing the flow of the structuring process performed by the structuring processing apparatus 10 according to the embodiment. [Figure 9] This is a flowchart showing the flow of the source text notation processing performed by the structured processing device 10 according to the embodiment. [Figure 10] This is a diagram illustrating the effects of the structured processing system 1 according to the embodiment. [Modes for carrying out the invention]

[0011] Embodiments of the present invention will be described below with reference to the drawings. In the following description, the case of structuring an address as text will be used as an example. However, the invention is not limited to this. This embodiment can be applied to structuring any text, not just addresses, such as work reports like daily reports, sales reports, customer service records, diagnostic reports, etc.

[0012] (Regarding Structured Processing System 1) In this embodiment, the structured processing system 1 is a system for structuring natural language (text). Figure 1 is a block diagram showing the configuration of the structured processing system 1 according to this embodiment. In this figure, the structured processing system 1 comprises a structured processing device 10, a carrier server 20, and a structured processing model server 30. In the structured processing system 1, the structured processing device 10 and the carrier server 20 are connected to each other via a communication network such as the Internet. Furthermore, the structured processing device 10 and the structured processing model server 30 are connected to each other via a communication network such as the Internet.

[0013] The service provider server 20 is a computer managed by service providers that offer various services to customers, such as financial institutions, local governments, or private service providers. The service provider server 20 is equipped with a customer master 201. The customer master 201 is a database that stores information about the service provider's customers. The customer master 201 stores various information about customers, such as address, name, age, and occupation, in natural language text.

[0014] The structured processing model server 30 is a computer that structures text using a structured processing model. Structuring here means associating the constituent elements of the text with one of the pre-defined items that are designated as structured items. For example, assume there is an address "1-1-2 Otemachi Chiyoda-ku, TOKYO 100-0005, JAPAN", and as structured items, Country and Country sub division are set. In this case, among the components of the address, the element "JAPAN" is associated with the "Country" as a structured item, and the element "TOKYO" is associated with the "Country sub division" as a structured item, thereby being structured.

[0015] The structuring processing model is, for example, a machine learning model trained to output structured data obtained by structuring the input text by learning the correspondence between the learning text and the structured data obtained by structuring the learning text.

[0016] Alternatively, the structuring processing model is a generative AI that learns patterns and relationships based on training data and generates content based on what has been learned. For example, as content, a generative AI of a natural language model that generates structured data obtained by structuring natural language sentences (text) can be used as the structuring processing model. As the natural language model, for example, a large language model such as an LLM (Large Language Model) can be used.

[0017] The structuring processing device 10 is a computer that generates structured data obtained by structuring text in response to a request from an operator corresponding to the operator server 20. As the structuring processing device 10, a cloud, an on-premises server, a personal computer, etc. can be applied.

[0018] Here, we will explain using an example of a case where a business requests that we structure the customer "address" contained in customer master 201. The address, which is the text to be structured (target text), contains many proper nouns, and even the same place name may be written in various ways. For example, when using English letters, it may be written as "Tokyo" with the first letter capitalized, as "tokyo" with the first letter lowercase, or as "TOKYO" with all capital letters. Also, when using Japanese characters (kanji, hiragana, katakana, etc.), it may be written as "Marunouchi" or as "Marunouchi". The notation of the street number and block may also be written in various ways, such as "1-chome 1-banchi 1-go", "1-chome 1-1", "1-1-1", "1 no 1 no 1", "1-1-1", etc. In addition, it may be written as an abbreviation with part of the place name omitted. Furthermore, if it is an address used for mailing within Japan, the notation "Japan" is often omitted.

[0019] While structured data processing models are generally trained on structured data processing, they are usually not trained to handle the various notations found in text. Therefore, structured data processing models use a unified rule that they have learned to perform structuring. Thus, even if the target text is written using "Tokyo," "tokyo," or "TOKYO," the structured data will create structured data that associates it with a unified notation (for example, "TOKYO" in all capital letters) with a structured item (for example, an item such as a prefecture (Country subdivision)).

[0020] On the other hand, businesses have a need to structure customer addresses according to specific regulations (e.g., ISO 20022 compliant items) while maintaining the unique identifiers included in the customer's address description. For this reason, if the text contains the notation "Tokyo," it is desirable that structured data be created in which the notation "Tokyo" is associated with a structured item (e.g., an item for prefecture (Country subdivision)). Similarly, if the text contains the notation "tokyo," it is desirable that structured data be created in which the notation "tokyo" is associated with a structured item (e.g., an item for prefecture (Country subdivision)). If the text contains the notation "TOKYO," it is desirable that structured data be created in which the notation "TOKYO" is associated with a structured item (e.g., an item for prefecture (Country subdivision)).

[0021] One possible solution is to customize the structured processing model, for example, by training the structured processing model with named entities. However, this requires preparing training data that includes various variations in notation to accommodate natural language variations, misspellings, and various ways of representing named entities, such as abbreviations, common names, and alternative symbols, and training the model with this data, which is time-consuming. Furthermore, because the processing performed by the structured processing model becomes more complex, even if the model is trained with named entities, it is not guaranteed that the model will be able to handle named entities with high accuracy.

[0022] Therefore, in this embodiment, the structured data (first structured data) generated by the structured processing model is subjected to source text notation processing. Source text notation processing is the process of converting the notation of each element in the first structured data back to the notation of the target text (source text). This avoids the effort and complexity of customizing the structured processing model, and makes it possible to create structured data that meets the requirements of businesses while using a general structured processing model as is.

[0023] (Regarding the structured processing device 10) In the example shown in Figure 1, the structured processing device 10 includes a target text acquisition unit 101, a structured processing unit 102, a source text notation processing unit 103, a target text data storage unit 104, a first structured data storage unit 105, a second structured data storage unit 106, and an output unit 107.

[0024] The target text acquisition unit 101 acquires the target text. For example, the business server 20 sends customer address data stored in the customer master 201 to the structured processing unit 10 and requests it to generate structured data by structuring the customer address data. The target text acquisition unit 101 acquires the customer address data notified by the business server 20 as the target text. The target text acquisition unit 101 stores the acquired target data in the target text data storage unit 104.

[0025] The structuring processing unit 102 generates structured data (first structured data) by structuring the target data using a structuring processing model. The specific process by which the structuring processing unit 102 generates the first structured data will be explained in detail later. For example, if the structured processing model is a machine learning model, the target text is input to the structured processing model. Since the machine learning model is trained to output structured data that structures the input text, the structured processing unit 102 outputs structured data that structures the input target text. The structured processing unit 102 acquires the data output from the structured processing model as structured data that structures the target text. Alternatively, if the structured processing model is a generative AI, the structured processing unit 102 generates a prompt instructing the AI ​​to structure the target text. The structured processing unit 102 notifies the generative AI of the structured processing model server 30 of the generated prompt. The generative AI retrieves the structured data in response to the prompt. The structured processing unit 102 retrieves the structured data generated by the generative AI. The structuring processing unit 102 stores the structured data (first structured data) generated using the structured processing model in the first structured data storage unit 105.

[0026] The original text notation processing unit 103 performs original text notation processing on the first structured data. The original text notation processing unit 103 stores the structured data (second structured data) obtained by performing original text notation processing on the first structured data in the second structured data storage unit 106. The specific processes by which the original text notation processing unit 103 performs the original text notation will be explained in detail later. For example, suppose that in the first structured data, the structured data element associated with a structured item (for example, an item related to prefecture (Country subdivision)) is written as "TOKYO". In contrast, if the target text is written as "Tokyo," the structured data element associated with the structured item in the second structured data (for example, the item for prefecture (Country subdivision)) will be written as "Tokyo." Furthermore, if the target text contains the notation "tokyo," the structured data element associated with the structured item in the second structured data (for example, the item for prefecture (Country subdivision)) will also be written as "tokyo."

[0027] The target text data storage unit 104 stores the text data (target text data) of the target text. Figure 2 shows an example of information stored in the target text data storage unit 104. In this figure, the target text data includes information corresponding to both the ID and the target text. The ID is identification information that identifies the target text. The target text is the text data of the target text identified by the ID. In this diagram, the target text identified by identification information ID (001) is the address of Otemachi, which is written as "1-1-2 Otemachi Chiyoda-ku, TOKYO 100-0005, JAPAN". The target text identified by identification information ID (002) is the address of Toyosu, which is written as "Toyosu Center Building Annex, 3-3-9 Toyosu, Koto-ku, Tokyo, Japan".

[0028] The first structured data storage unit 105 stores the first structured data. Figure 3 shows an example of information stored in the first structured data storage unit 105. In this figure, the first structured data includes information corresponding to ISO20022 compliant items and structured information. ISO20022 compliant items are structured items, and here, items such as Sub Department, Street Name, Building Number, ..., Town Name, ..., Country Sub Division, ... are set as items compliant with a specific standard (ISO20022). Structured information is the element of the target text corresponding to the structured item. Here, the element of the target text (1-1-2 Otemachi) is associated with the structured item (Street Name). Also, the element of the target text (CHIYODA) is associated with the structured item (Town Name). Also, the element of the target text (TOKYO) is associated with the structured item (Country Sub Division).

[0029] The second structured data storage unit 106 stores the second structured data. Figure 4 shows an example of information stored in the second structured data storage unit 106. In this figure, the second structured data, like the first structured data, includes information corresponding to both ISO20022 compliant items and structured information. Similar to the first structured data, the ISO20022 compliant items are structured items. The structured information is the element of the target text corresponding to the structured item. Here, while the first structured data associated the structured item (Town Name) with the capital letter (CHIYODA), the second structured data associates it with structured information that has been reverted to the notation (Chiyoda-ku) as described as element TY10 of the target text T.

[0030] Returning to the explanation of Figure 1, the output unit 107 outputs various information. For example, the output unit 107 sends the second structured data generated by the original text notation processing unit 103 to the business server 20 as structured data generated in response to a request from a business corresponding to the business server 20.

[0031] (Regarding structured processing) Here, the specific process by which the structured processing unit 102 generates the first structured data will be explained using Figure 5. Figure 5 is a diagram illustrating the structured processing performed by the structured processing unit 10 according to this embodiment. Here, the case in which a generation AI is used as the structured processing model will be explained as an example.

[0032] In this diagram, the target text acquisition unit 101 of the structured processing device 10 acquires target text T1 as input. Here, it is shown that the target text T1 is an address written as "1-1-2 Otemachi Chiyoda-ku, TOKYO 100-0005, JAPAN".

[0033] The structured processing unit 102 of the structured processing device 10 references external data via an external data API (Application Programming Interface). The external data API is an API for using external data. External data is data that stores information about the target text. By extracting information about the target text from the external data and inputting the extracted data along with the target text into the generating AI, the generating AI can generate answers using the external data in addition to the existing knowledge gained through training, thereby improving the accuracy of the answers.

[0034] In this embodiment, the external data is data useful for structuring addresses, such as geographic data. The geographic data associates attribute information, such as location, country name, and city classification, with various place names on Earth. By using such geographic data, addresses shown in the target text can be associated with arbitrary structured items as a preprocessing step.

[0035] The structuring processing unit 102 generates preprocessed data PD that structures the target text according to arbitrary structured items in accordance with a general geographic system, without limiting it to a specific standard (ISO20022), by referencing external data. In this diagram, the preprocessed data PD includes information corresponding to both non-ISO compliant items and structured information. Non-ISO compliant items are arbitrary structured items that conform to a general geographical system. Here, items such as Country, City-Country Subdivision, Street number, Street name, and POST Code are set as arbitrary structured items that conform to a general geographical system. Structured information consists of elements of the target text that correspond to the structured items. Here, the structured item (Country) in the preprocessed data PD is associated with the element (JAPAN) in the target text. Similarly, the structured item (City-Country Subdivision) is associated with the element (CHIYODA-ku,TOKYO) in the target text. Furthermore, the structured item (Street name) is associated with the element (1-1-2 Otemachi) in the target text.

[0036] The structuring processing unit 102 instructs the generation AI, which acts as a model ML, to generate structured data in accordance with a specific standard (ISO20022). The structuring processing unit 102 uses preprocessing data PD to generate a prompt instructing the generation AI to generate structured data structured according to the specific standard (ISO20022) from the target text. The structuring processing unit 102 sends the generated prompt, along with the target text and preprocessing data PD, to the generation AI. The generation AI structures the target text according to the specific standard (ISO20022) using preprocessing data PD in response to the prompt notified by the structuring processing unit 102. The generation AI outputs structured data KD1, which structures the target text according to the specific standard (ISO20022), to the structuring processing unit 10. As a result, the structuring processing unit 102 of the structuring processing unit 10 obtains the structured data KD1 from the generation AI. In this diagram, the structured data KD1 generated by the generation AI corresponds to the first structured data described above. Specifically, in structured data KD1, the element (1-1-2 Otemachi) from the target text T1 is associated with the structured item (Street Name) as an ISO20022 compliant item. Furthermore, although the structured data KD1 associates the element (CHIYODA) from the target text T1 with the structured item (Town Name) as an ISO20022 compliant item, it includes instances where the notation does not perfectly match the corresponding element in the target text T1 (Chiyoda-ku).

[0037] (Regarding the processing of the original text) Here, the specific processing performed by the original text notation processing unit 103 will be explained using Figure 6. Figure 6 is a diagram illustrating the original text notation processing performed by the structured processing device 10 according to this embodiment.

[0038] The lower part of Figure 6 shows the target text T2. Here, it is indicated that the address used as the target text T2 is "Toyosu Center Building Annex, 3-3-9 Toyosu, Koto-ku, Tokyo, Japan".

[0039] The upper part of Figure 6 shows the first structured data KD2, which corresponds to the target text T2 in Figure 6. Here, the element KY20 (TOKYO,) is associated with the structured item (Country Subdivision).

[0040] The middle section of Figure 6 shows the divided text T2#, which corresponds to the target text T2 in Figure 6. The divided text T2# is the text obtained by dividing the target text T2 using a specific delimiter character contained within the target text T2 as the boundary. Here, the character MK, which corresponds to a space, is used as the delimiter. In this diagram, the example shown is that the target text T2 is divided into the following elements, which are designated as the divided text T2#: element YT20 "Toyosu", element YT21 "Center", element YT22 "Building", element YT23 "Annex,", element YT24 "3-3-9", element YT25 "Toyosu,", element YT26 "Koto-ku,", element YT27 "Tokyo," and element YT28 "Japan".

[0041] The source text notation processing unit 103 of the structured processing device 10 generates segmented text T2# using the target text T2. The source text notation processing unit 103 determines whether a character corresponds to a delimiter character, character by character from the beginning of the target text. The delimiter character can be any character, and for example, all or part of characters and symbols such as spaces, commas, periods, punctuation marks, middle dots, hyphens, single quotation marks, double quotation marks, hash symbols, dollar signs, and combinations thereof can be used.

[0042] The original text notation processing unit 103 calculates the similarity between the elements associated with structured items in the structuring process (structured elements) and the elements of the segmented text T2# (text elements). Similarity is the degree to which two elements (in this case, structured elements and text elements) are similar. For example, similarity is the degree to which all or some of the characteristics of the text, or combinations thereof, are similar, such as the appearance of the text, its pronunciation as a name, its meaning and content as an idea, and the relationship between substitutable characters, such as the relationship between uppercase and lowercase letters, or between hiragana and katakana. Any method can be used to calculate similarity. For example, features extracted from one element and features extracted from the other element can be extracted using a machine learning model or language model trained to extract features from strings. The features extracted from each element are mapped to a feature space formed around each of the features that the machine learning model or language model can extract. The similarity between the two elements can be calculated according to the distance between the features corresponding to each element in the feature space.

[0043] The source text notation processing unit 103 calculates the similarity of a single structured element to each of the text elements that make up the segmented text T2#. Here, the source text notation processing unit 103 calculates the similarity of structured element KY20 to each of the text elements TY20 to TY28 that make up the segmented text T2#. The original text notation processing unit 103 extracts the text element with the highest similarity score among the calculated similarity scores, that is, the text element that is most similar to the structured element. Here, the text element TY27 (Tokyo), which has the greatest similarity to the structured element KY20 (TOKYO,), is extracted. The source text notation processing unit 103 compares the notation of the extracted text element with the notation of the structured element. If the notations of the two differ, the source text notation processing unit 103 changes the notation of the structured element to match the notation of the text element. Here, the notation of the text element TY27 (Tokyo) is compared with the notation of the structured element KY20 (TOKYO,). Since the notations of the two differ ((Tokyo) and (TOKYO,)), the source text notation processing unit 103 changes the notation of the structured element KY20 (TOKYO,) to match the notation of the text element TY27 (Tokyo). With this, the source text notation processing unit 103 performs the source text notation processing.

[0044] Furthermore, if the original text notation processing unit 103 calculates that the similarity of a structured element to any of the text elements is all 0 (zero), meaning that the structured element is not similar to any of the text elements, it determines that there is no text element corresponding to that structured element. In this case, the original text notation processing unit 103 does not change the notation of that structured element.

[0045] (Processing flow performed by the structured processing device 10) Here, the processing flow performed by the structuring processing apparatus 10 will be explained using Figures 7 to 9. Figures 7 to 9 are flowcharts showing the processing flow performed by the structuring processing apparatus 10 according to this embodiment.

[0046] Figure 7 shows the overall flow of processing performed by the structured processing device 10. First, the target text acquisition unit 101 of the structured processing device 10 acquires the target text (step S100). Next, the structured processing unit 102 of the structured processing device 10 generates the first structured data (step S200). Next, the source text notation processing unit 103 of the structured processing device 10 performs source text notation processing (step S300). By performing source text notation processing, the source text notation processing unit 103 generates the second structured data in which the notation of the first structured data is modified to match the notation of the target text. The output unit 107 of the structured processing device 10 outputs the second structured data.

[0047] Figure 8 shows the flow of structuring processing performed by the structuring processing unit 102 of the structuring processing device 10. First, the structuring processing unit 102 acquires the target text from the target text acquisition unit 101 (step S201). Next, as preprocessing, the structuring processing unit 102 structures the target data using external data and arbitrary structured items in line with a general geographic system (step S202). Next, as preprocessing, the structuring processing unit 102 generates a prompt instructing the system to convert the data structured in an arbitrary format (preprocessed data PD) generated in step S202 into the format of a standard (ISO20022). For example, the structuring processing unit 102 generates a prompt instructing the system to generate structured data structured according to a specific standard (ISO20022) from the target text. The structuring processing unit 102 sends the generated prompt, along with the target text and preprocessed data PD, to the generating AI. The structuring processing unit 102 acquires the response corresponding to the prompt as first structured data (step S204). The structuring processing unit 102 acquires the structured data generated by the generating AI in response to the prompt notified by the structuring processing unit 10 as the first structured data.

[0048] Figure 9 shows the flow of the source text notation processing performed by the source text notation processing unit 103 of the structured processing device 10. First, the source text notation processing unit 103 obtains the target text T (step S301). For example, the source text notation processing unit 103 obtains the target text T by reading the target text T from the target text data storage unit 104. Next, the source text notation processing unit 103 generates divided text T# by dividing the target text T based on a delimiter (step S302). Furthermore, the original text notation processing unit 103 acquires the first structured data KD (step S303). For example, the original text notation processing unit 103 acquires the first structured data KD by reading it from the first structured data storage unit 105. The original text notation processing unit 103 calculates the similarity between the structured element KY of the first structured data KD and all the text elements TY that make up the segmented text T# (step S304).

[0049] The source text notation processing unit 103 identifies the text element TY of the segmented text T# that corresponds to the structured element KY of the first structured data KD based on the calculated similarity (step S305). For example, the source text notation processing unit 103 identifies the text element TY27, which is written as (Tokyo), as the text element that corresponds to the notation (TOKYO,) of the structured element KY20. The original text notation processing unit 103 determines whether the notations match (step S306). The original text notation processing unit 103 determines whether the notation of the structured element KY, for which similarity was calculated, matches the notation of the text element TY, which was identified in step S304.

[0050] If the notation does not match in step S306, the source text notation processing unit 103 changes the notation of the structured element KY to match (step S307). For example, the source text notation processing unit 103 changes the notation of structured element KY20 (TOKYO,) to (Tokyo) to match the notation of text element TY27 (Tokyo). The source text notation processing unit 103 uses the modified structured element KY as the structured element of the second structured data. The source text notation processing unit 103 determines whether the notation matches for all structured elements in the first structured data KD (step S308). If there are any structured items in the first structured data KD that do not match the notation of the target text, the process returns to step S304 and performs source text notation processing for the structured items that do not match.

[0051] In step S308, if all structured elements of the first structured data KD match the notation of the target text, the structured data (second structured data) that matches the notation of the target text is stored in the second structured data storage unit 106 (step S309).

[0052] As described above, according to this embodiment, the structured processing device 10 comprises a target text acquisition unit 101, a source text notation processing unit 103, and an output unit 107. The target text acquisition unit 101 is an example of an acquisition unit and acquires text information of the target text to be structured. The source text notation processing unit 103 performs source text notation processing. Source text notation processing is the process of matching the notation of structured elements, which are elements included in the first structured data, with the notation of the target text corresponding to the structured elements. The first structured data is data structured by applying a structured processing model (e.g., a generation AI) to the target text. The output unit 107 outputs the second structured data. The second structured data is structured data on which source text notation processing has been performed on the notation of the structured elements by the source text notation processing unit 103. As a result, the structured processing device 10 of this embodiment can return the notation of structured elements to the notation of the source text (target text) even if they have been converted to a general notation and structured during the structuring process. Therefore, it becomes easier to structure the text into a prescribed format while taking into account named entities described in the text.

[0053] Furthermore, according to the embodiment, the structured processing device 10 has a source text notation processing unit 103 that calculates the similarity between the text element TY and the structured element KY. The text element TY is a component of the segmented text T# obtained by dividing the target text T with a delimiter (e.g., a space) contained in the target text T. The source text notation processing unit 103 identifies the text element TY in the target text T that corresponds to the structured element KY according to the calculated similarity. The source text notation processing unit 103 changes the notation of the structured element KY to match the notation of the identified text element TY. As a result, the source text notation processing unit 103 performs the source text notation processing. As a result, the structured processing device 10 of the embodiment can accurately identify the text element TY corresponding to the structured element based on the similarity, and can restore the notation of the structured element to a notation faithful to the source text (target text).

[0054] Furthermore, according to this embodiment, in the structured processing device 10, the delimiter is a character corresponding to a space or a period included in the target text. As a result, the structured processing device 10 of this embodiment can divide the target text into elements that are easily corresponded to structured items by using a description format that is commonly used in customer master data 201, etc.

[0055] Furthermore, according to the embodiment, in the structured processing device 10, the structured processing model is either a language processing model or a machine learning model. The machine learning model is a model that has been trained to perform the processing of structuring the target text by learning the correspondence between the training text and the structured data obtained by structuring the training text. As a result, in the structured processing device 10 of the embodiment, structured data can be generated using any model that has been trained to perform structuring processing.

[0056] Furthermore, according to this embodiment, in the structured processing device 10, the target text T includes an address. The source text notation processing device 103 performs source text notation processing by converting the notation of the structured element KY, which has been converted to a standard place name during the structuring process, back to the expression of the place name written in the target text T. As a result, in the structured processing device 10 of this embodiment, even if an address containing a named entity is converted to a standard place name during the structuring process, such as by structuring it using a structured processing model that has learned standard notations, the notation of the structured element KY can be converted back to the expression of the place name written in the target text T.

[0057] Furthermore, according to the embodiment, the structured processing device 10 further comprises a structured processing unit 102. The structured processing unit 102 structures the target text T. Specifically, the structured processing unit 102 inputs a prompt to the generating AI (language processing model) instructing it to structure the target text T. The structured processing unit 102 obtains a response generated from the language processing model in response to the prompt. As a result, the structured processing unit 102 structures the target text T. Thus, in the structured processing device 10 of the embodiment, the generating AI can generate structured data, and the text can be structured by simple processes such as generating and sending prompts.

[0058] Furthermore, according to one embodiment, in the structured processing device 10, the target text T includes an address. The structured processing device 102 inputs a prompt to the generating AI (language processing model) instructing it to generate first structured data, which is the target text structured according to a specific format using preprocessed data PD. The preprocessed data PD is data in which the target text T is structured using external data, including geographic data. The structured processing device 102 obtains the response generated in response to the prompt from the language processing model. As a result, the structured processing device 102 obtains first structured data in which the target text T is structured according to a specific format. As a result, the structured processing device 102 can structure the target text T according to a specific standard (e.g., ISO20022). By generating preprocessed data PD using external data and notifying the generating AI, the generating AI can structure using the preprocessed data PD, thereby enabling the generation of structured data that conforms to the standard format (e.g., ISO20022) with greater accuracy.

[0059] Here, the effects of this embodiment will be explained using Figure 10. Figure 10 is a diagram illustrating the effects of the processing performed by the structured processing system 1 according to this embodiment. As described in the above embodiment, the operator server 20 transmits the target text T to the structured processing device 10. The structured processing unit 10 receives the target text T from the carrier server 20. The structured processing unit 10 obtains the target text T by reading the plain text address received from the carrier server 20. The structured processing unit 10 makes a structured request to structure the target text by sending the obtained target text T to the structured processing model server 30. The structured processing model server 30 generates first structured data KD, which is the structured data of the target text T, using the structured processing model. The structured processing model server 30 sends the generated first structured data KD to the carrier server 20. Here, in the first structured data KD, elements of the target text T are associated with structured fields, but the associated elements may not be a faithful representation of the notation in the target text T, but rather converted to commonly used standard notation. For example, the target text T is written as "Japan," but the structured field (Ctry) in the first structured data KD is associated with the uppercase notation "JAPAN." The structured processing unit 10 receives the first structured data KD from the structured processing model server 30. If the notation of the structured element KY in the first structured data KD received from the structured processing model server 30 does not match the notation of the corresponding text element TY, the structured processing unit 10 performs a notation change to change the notation of the structured element KY to match the notation of the text element TY. As a result, the structured data processing device 10 can generate second structured data HKD in which the notation of structured element KY matches the notation of the corresponding text element TY. Therefore, the structured data processing device 10 can meet the requirements of businesses that want to structure customer addresses while maintaining named entities.

[0060] (Modification of Embodiment 1) Herein, we will describe a modification 1 of the embodiment. This modification differs from the embodiment described above in that it uses the determination result obtained by determining the country corresponding to the address shown in the target text T.

[0061] It is generally known that the way addresses are written differs from country to country. For example, in English, addresses are written starting with the smallest district, such as "1-1-2 Otemachi Chiyoda-ku, TOKYO 100-0005, JAPAN." On the other hand, in Japanese, addresses are written starting with the larger district, such as "Tokyo, Chiyoda-ku, Otemachi 1-1-2." Thus, it is known that addresses contain various unique expressions due to differences in historical and cultural backgrounds in different countries and regions. Furthermore, since standard notation refers to the notation used by the majority of people in a country, it is reasonable to assume that standard notation may differ from country to country.

[0062] To accommodate these differences based on country, region, and language, this modified version includes a country determination process that determines the country corresponding to the address shown in the target text T. For example, the structured processing unit 102 generates a prompt instructing the AI ​​to determine the country corresponding to the address shown in the target text T, and sends the generated prompt and the target text T to the generating AI. The generating AI generates a response in response to the prompt notified by the structured processing unit 102 and sends the generated response to the structured processing unit 102 of the structured processing device 10. If the response from the generating AI includes a country name, the structured processing unit 102 determines that the country corresponds to the address shown in the target text T.

[0063] Alternatively, the structured processing unit 102 may perform country determination processing before performing structured processing using the structured processing model. In this case, the structured processing unit 102 generates preprocessed data PD as shown in Figure 5. The structured processing unit 102 determines the country corresponding to the address shown in the target text T based on the elements associated with the structured item (Country) corresponding to "Country" in the preprocessed data PD. For example, as shown in Figure 5, if the structured element (JAPAN) is associated with the structured item (Country) in the preprocessed data PD, the unit determines that the country corresponding to the address shown in the target text T is Japan.

[0064] The structuring processing unit 102 generates a prompt corresponding to the country of the address shown in the target text T. For example, the structuring processing unit 102 generates a prompt that indicates that the target text T contains an address, that the country corresponding to that address is the country determined by the country determination process (in this case, Japan), and that the preprocessed data PD should be used to structure the data according to the standard (ISO20022). This allows the generating AI to be explicitly notified of the country name corresponding to the address shown in the target text T, thereby improving the accuracy of the response.

[0065] As described above, in the structured processing device 10 according to the modified embodiment 1, the structured processing unit 102 determines that the target text T includes an address. The structured processing unit 102 performs a country determination process to determine the country name corresponding to the address shown in the target text T. The structured processing unit 102 generates a prompt according to the determination result of the country determination process. As a result, the structured processing device 10 according to the modified embodiment 1 can perform structured processing according to the country corresponding to the address shown in the target text T, and can perform structuring that takes into account named entities specific to that country and language.

[0066] (Modified embodiment 2) Herein, we will describe a modified example of the embodiment 2. This modified example differs from the embodiment described above in that it performs error processing when the second structured data contains errors. Examples of errors that may occur in the second structured data are as follows: Error E1: No text elements are classified into a specific structured field (missing elements). Error E2: Classification error where an incorrect text element is classified into a specific structured field. Error E3: A specific structured field contains an element with an incorrectly set border for the target text. ("Border Setting Error") Error E4: "Character addition" - A structured element contains characters that are not present in the text element. Error E5: The structured element does not contain any characters present in the text element. ("Character Deletion")

[0067] Furthermore, experience has shown that some countries are more prone to errors E than others. For example, English-speaking countries tend to have a higher incidence of errors E1-E3, while errors E4-E5 are less likely to occur. On the other hand, Asian countries, especially those with Chinese characters, tend to have a higher incidence of errors E1-E3 as well as E4-E5. In particular, Japanese-speaking countries are prone to error E4, while Chinese-speaking countries tend to have a higher incidence of both errors E4-E5.

[0068] In light of these trends, this modified version implements error handling tailored to the country (or language). For example, if the target text T corresponds to an English-speaking region, errors E1 to E3 tend to occur more frequently for certain structured items. Therefore, the system determines whether those structured items are missing, whether there are classification errors, or whether the boundaries of the target text are incorrectly set. For example, regarding error E2, "classification error," it is possible to determine whether or not error E2 occurred by using table information that associates a specific structured item with the text element that was incorrectly classified. If error E2 occurred, error processing for error E2 can be performed using table information that associates the incorrectly classified text element with the correctly classified text element, and the correct text element can be reclassified to the specific structured item.

[0069] On the other hand, when the target text T corresponds to the Asian region, errors E4 to E5 tend to occur more frequently for certain structured items. Therefore, the similarity and character count are calculated for those specific structured items. If the similarity is above the threshold and the character count of the structured element is greater (compared to the text element), it is determined that error E4 has occurred. Conversely, if the similarity is above the threshold and the character count of the structured element is less (compared to the text element), it is determined that error E5 has occurred. In this case, for example, it is determined whether the text corresponding to the structured element contains multiple words. Whether or not multiple words are present can be determined using methods such as morphological analysis. If the text corresponding to the structured element contains multiple words, for example, a specific character (e.g., a comma) is added between the words. The string with the specific character (e.g., a comma) added to the structured element is then split into individual words, and the similarity of each split word to the structured element is calculated. Of the similarities calculated for each split word, the one with the greatest similarity to a certain text element is selected as the structured element corresponding to that text element. In this case, the notation of the structured element is changed to match the notation of that text element. Alternatively, if error E4 occurs, the system determines whether there are any characters that are likely to be deleted (for example, whitespace characters or commas) among the characters that make up the structured element. If the characters that make up the structured element contain characters that are likely to be deleted, and the text element corresponding to that structured element does not contain those characters (characters that are likely to be deleted), the original text notation processing unit 103 deletes the characters (characters that are likely to be deleted) that make up the structured element. In this case, the original text notation processing unit 103 recalculates the similarity between the element after the character (character that is likely to be deleted) has been removed from the structured element and each of the text elements, and uses the recalculated similarity to identify the text element that corresponds to the structured element.

[0070] As described above, in the structured processing device 10 according to the modified embodiment 2, the target text T includes an address. The original text notation processing device 103 performs error processing. Error processing occurs when an error is found in a structured element for a specific structured item set in the second structured data. The structured processing device 102 performs country determination processing to determine the country name corresponding to the address shown in the target text T. The original text notation processing device 103 performs error processing according to the determination result of the country determination processing. As a result, the structured processing device 10 according to the modified embodiment 2 can efficiently perform error processing by handling errors that are likely to occur in the country and language corresponding to the address shown in the target text T, while avoiding error processing that is difficult to perform in some countries and languages.

[0071] The structured processing apparatus 10 in the above-described embodiment may be implemented in whole or in part by a computer. In that case, the program for implementing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be loaded into a computer system and executed. The term "computer system" here includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and storage devices such as hard disks built into a computer system. Furthermore, the term "computer-readable recording medium" may also include those that dynamically hold programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or communication lines such as telephone lines, and those that hold programs for a certain period of time, such as volatile memory inside a computer system that acts as a server or client in such cases. The program may be for implementing a part of the functions described above, or it may be a program that can implement the functions described above in combination with a program already recorded in the computer system, or it may be implemented using a programmable logic device such as an FPGA.

[0072] While embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and includes designs and the like that do not depart from the spirit of this invention. [Explanation of Symbols]

[0073] 1…Structured Processing System 10…Structure processing device 101...Target text acquisition unit (acquisition unit) 102...Structured Processing Section 103...Original Text Notation Processing Section 104...Target text data storage unit 105...First Structured Data Storage Unit 106...Second Structured Data Storage Unit 107…Output section 20…Service provider server 201...Customer Master (Target Text) 30…Structured Processing Model Server

Claims

1. An acquisition unit that acquires text information of the target text that is to be structured, A source text notation processing unit performs source text notation processing to match the notation of structured elements, which are elements included in first structured data structured by applying a structured processing model to the target text, with the notation of the target text corresponding to the structured elements. An output unit outputs second structured data in which the original text notation processing has been performed on the notation of the structured elements by the original text notation processing unit, A structured processing device equipped with the following features.

2. The aforementioned original text notation processing unit, The similarity between the text elements obtained by dividing the target text using the delimiter contained in the target text and the structured elements is calculated. According to the calculated similarity, the text element in the target text corresponding to the structured element is identified, The original text notation processing is performed by changing the notation of the structured element to match the notation of the identified text element. The structuring apparatus according to claim 1.

3. The delimiter is a character that corresponds to a space or a period in the target text. The structuring apparatus according to claim 2.

4. The structured processing model is a language processing model, or a machine learning model trained to perform the processing of structuring the target text by learning the correspondence between the training text and structured data obtained by structuring the training text. The structuring apparatus according to claim 1.

5. The aforementioned target text includes an address, The aforementioned source text notation processing unit performs the source text notation processing by converting the notation of the structured elements, which have been converted to standard place names during the structuring process, back to the expression of the place names written in the target text. The structuring apparatus according to claim 1.

6. The system further includes a structuring processing unit that inputs a prompt to a language processing model instructing it to structure the target text, and then structures the target text by obtaining a response generated from the language processing model in response to the prompt. The structuring apparatus according to claim 1.

7. The aforementioned target text includes an address, The structured processing unit further includes a prompt that instructs a language processing model to generate first structured data in which the target text is structured according to a specific format, using preprocessed data that has been structured using external data including geographic data, and obtains the first structured data in which the target text is structured according to a specific format by obtaining the response generated in response to the prompt from the language processing model. The structuring apparatus according to claim 1.

8. A structured processing method performed by a structured processing unit, which is a computer, The acquisition unit acquires the text information of the target text that is to be structured, The original text notation processing unit performs original text notation processing to match the notation of structured elements, which are elements included in the first structured data structured by applying a structured processing model to the target text, with the notation of the target text corresponding to the structured elements. The output unit outputs second structured data in which the original text notation processing has been performed on the notation of the structured elements by the original text notation processing unit. A structured processing method.

9. In a structured processing unit, which is a computer, To obtain the text information of the target text that is to be structured, By applying a structured processing model to the target text, a source text notation process is performed to match the notation of the structured elements, which are elements included in the first structured data that has been structured, with the notation of the target text corresponding to the structured elements. The system outputs second structured data in which the original text notation processing has been applied to the notation of the aforementioned structured elements. program.

Images